Center Street PAC calls itself a ‘nonpartisan PAC focused on defeating radical partisan extremists by helping elect rational candidates who promote stable & effective governing policies.’
What we’ve come to expect from these supposed nonpartisan groups preaching about putting country over party is that they are working to elect Democrats and there is nothing at all nonpartisan about them. Especially when you see their polling data and polls broken down by people who actually know what the Hell they’re doing.
Big thanks to G. Elliott Morris for putting in the work:
You may have noticed some peculiar Senate polls from a "nonpartisan" group called Center Street PAC this cycle. Polls like these, where Democrats are doing 10-20 points better than the averages.
I interviewed their pollster and got my hands on their data. Here is what I found:🧵 pic.twitter.com/rcW4aoDvzQ
— G Elliott Morris (@gelliottmorris) October 20, 2022
Get the corn, this is really interesting.
Here is the article if you want to skip the gory details:https://t.co/t0MJHQlaJW
The first thing to do is establish whether CSP is an outlier. So we made a model that measures how far each pollster's data deviates from averages, accounting for partisan bias & a few other vars.
— G Elliott Morris (@gelliottmorris) October 20, 2022
By this measure, Center Street's polls have Democratic margins about 10-12 points higher than expected — the highest difference for any pollsters. Now, we wouldn't count that whole number as the firm's "house effect," but it proves that our suspicion of bias is correct.
— G Elliott Morris (@gelliottmorris) October 20, 2022
Suspicion of bias.
No way.
How could it be?!
But why is the bias there? Is the PAC doing something different than everybody else? Do they have some weird data source or use bad weights? Etc.
I asked the PAC's pollster — an investor and market researcher who has never done political polling before — to explain.— G Elliott Morris (@gelliottmorris) October 20, 2022
… who has never done political polling before.
Alrighty then.
He told me that he didn't know, but has 2 suspicions. First, they do not "push" undecided voters to say whether they are leaning towards a candidate. This is generally standard practice, since partisans with softer preferences who will still turn out can hang out in that category
— G Elliott Morris (@gelliottmorris) October 20, 2022
Center Street's pollster says he does not push leaners because they are "measuring something different than what the other polls are measuring" and has "a substantial philosophical difference" with them. (He has no record to justify that decision.)
— G Elliott Morris (@gelliottmorris) October 20, 2022
Substantial philosophical difference.
HA HA HA HA HA HA HA
Second, the CSP polls come from an internet sample that is weighted only by age, gender and recalled 2020 vote. This means the poll could still undercount white non-college voters, or overcount people who lie about not voting for Trump, or the most pro-Vance Trump voters, etc.
— G Elliott Morris (@gelliottmorris) October 20, 2022
Sounds legit.
Oh wait, it gets better.
There was not much pushback to that. CSP's pollster maintained that weighting by 2020 vote "absorbs" variance from the other predictors, making them unnecessary. (This is wrong.) He then asked me to explain how other pollsters weight data. (I also consider this a red flag…)
— G Elliott Morris (@gelliottmorris) October 20, 2022
Serious red flag.
This begs two questions. First, what would happen if we re-weighted Center Street PAC's data to be representative by age, sex, the interaction of race + education, and 2020 vote? And what if we push undecided voters to pick a side? Do the wildly pro-Democratic margins disappear?
— G Elliott Morris (@gelliottmorris) October 20, 2022
Center Street PAC was transparent enough to share all the data from their Ohio polls with The Economist, so we re-engineered the results using their data.
We created 4 different versions with 3 diff weighting schemes, & 1 pushing undecideds (toward their 2020 vote).
Results: pic.twitter.com/pcrq7xbUFm
— G Elliott Morris (@gelliottmorris) October 20, 2022
Our analysis shows that if Center Street had taken steps to weight their data properly and ask the horse-race question in the best-practice format, then their margin for Tim Ryan in OH-Sen would have fallen from 8 points among registered voters (11 among LVs) to just 2-3 points.
— G Elliott Morris (@gelliottmorris) October 20, 2022
You know, more in line with the average numbers we’re all seeing.
Now, do we buy that 2-3 points? Probably not. I think even these adjustments are not good enough.
Center Street PAC gets its data from an opaque online survey provider which fields its surveys to respondents via a collection of different platforms. This is a problem because…— G Elliott Morris (@gelliottmorris) October 20, 2022
Even these adjustments are not good enough.
Wow.
… bc polls fielded online tend to have higher shares of Democrats—even within demographic groups. A good online pollster (YouGov, for example) adjusts for this using some proxy of political engagement. But the CSP data offer no method for correction. It is a biased sample.
— G Elliott Morris (@gelliottmorris) October 20, 2022
It is a biased sample.
This raises 3 important points:
1, about polling in an era of low response rates and new technologies:– You cannot weight your way out of bad data.
Pollsters must first design good polls, and cannot rely only on weighting & algos to solve all their problems. (2020, anyone?)
— G Elliott Morris (@gelliottmorris) October 20, 2022
2, about uncertainty:
– The true margin of error in a survey is much wider than its sampling error. In this example, we moved the topline vote shares for Tim Ryan and JD Vance BY UP TO TWELVE PERCENTAGE POINTS just by changing how the data was processed. (True fx cd be larger.)
— G Elliott Morris (@gelliottmorris) October 20, 2022
and 3, about intention and partisan bias in aggregates:
– Center Street PAC's pollster told me that they use their surveys to "show viability" of the candidates they support. He says the polls get sent to "high net-worth, kind of significant donors" to convince them to give $$.
— G Elliott Morris (@gelliottmorris) October 20, 2022
So in reality, Center Street PAC is partisan and actively using polls to raise money for those candidates they support.
Gosh, that sounds pretty damn sleazy.
The point here is that we should not blindly trust numbers from pollsters and just throw them in the average. In this case, the data-generating process for CSP's "polls" is heavily polluted, by both dramatically subpar methods decisions and very severe conflicts of interest.
— G Elliott Morris (@gelliottmorris) October 20, 2022
Heavily polluted.
That’s putting it nicely.
So, in sum:
1. Center Street releases opaque "polls" showing candidates _that they have endorsed_ doing 10-12 pts better than the averages of other polls.
3. This is almost entirely due to bad choices of methods.
4. Those choices are overlooked bcuz outliers raise more money.— G Elliott Morris (@gelliottmorris) October 20, 2022
THAT THEY HAVE ENDORSED.
One takeaway is to be careful with polls from this particular PAC, and maybe even all PACs in general. I have always been more than a bit nervous about aggregating publicly available polling from partisan sources. The DGP can break down in very predictable ways, as we see here.
— G Elliott Morris (@gelliottmorris) October 20, 2022
Nonpartisan our FOOT.
But more importantly, I hope this shows readers how easily polls can go wrong,
How every decision, even the seemingly minor ones, changes the final numbers that you and I see flash across our screens.
& abt the value of getting smarter w polls.
Uncertainty is really really high.— G Elliott Morris (@gelliottmorris) October 20, 2022
Here is the rebuttal from Center Street's pollster.
My response is: Given that you see huge differences when pushing undecideds gently, I think that suggests the survey industry best practice (to push them) should also apply here.
Let me make another pthttps://t.co/tSVKzxTbnL
— G Elliott Morris (@gelliottmorris) October 20, 2022
Heh.
We told you it gets better.
When I was talking w @kurtjetta I got the sense that he is a straight shooter who simply had no one to help him interrogate his assumptions about methods.
This is a danger in any research setting, but especially in _political_ research; Bias and small sample sizes don't mix. 1/2— G Elliott Morris (@gelliottmorris) October 20, 2022
2/2 This contrasts w the umbrella Super PAC machinery that oversees his polling. Maybe, with no one working with him, he only gets input from the hyper-partisan operatives working the fundraising arm of C St
& that's where it gets messy to take public Super PAC "polls" seriously
— G Elliott Morris (@gelliottmorris) October 20, 2022
Don’t worry, we’re not taking Center Street PAC polls seriously.
jfc
— Geoffrey Skelley (@geoffreyvs) October 20, 2022
In short: These are not polls at all. They're frauds designed with an end result in mind.
— Titan (@BOOB_level) October 21, 2022
I asked one of center street pacs' employees to explain why they 1) don't weigh by education or party and 2) weigh the results to the 2020 election.
His response was to call me a domestic terrorist.
They are grifting frauds.https://t.co/CCWzJPyFUF
— JiggleMyJello (@jello_jiggle) October 20, 2022
They’re about as nonpartisan as this editor … so you know, like not at all.
Heh.
***
Related:
***
Editor’s note:
Help us keep owning the libs and join Twitchy VIP! If you use promo AMERICAFIRST you’ll save 25%
Join the conversation as a VIP Member