Nate Silver (@fivethirtyeight) September 13, 2013
As Twitchy reported Wednesday, the Public Policy Polling Company, a firm that conducts polls for Democrats, has come under fire from FiveThirtyEight editor Nate Silver and others for withholding the results of some of its polls.
Yesterday PPP came in for a second round of thrashing after Nate Cohn of The New Republic raised serious questions about PPP’s fly-by-the-pants approach to polling:
After examining PPP’s polls from 2012 and conducting a lengthy exchange with PPP’s director, I’ve found that PPP withheld controversial elements of its methodology, to the extent it even has one, and treated its data inconsistently. The racial composition of PPP’s surveys was informed by whether respondents voted for Obama or John McCain in 2008, even though it wasn’t stated in its methodology. PPP then deleted the question from detailed releases to avoid criticism. Throughout its seemingly successful run, PPP used amateurish weighting techniques that distorted its samples—embracing a unique, ad hoc philosophy that, time and time again, seemed to save PPP from producing outlying results. The end result is unscientific and unsettling.
PPP’s opacity and flexibility goes too far. In employing amateurish weighting techniques, withholding controversial methodological details, or deleting questions to avoid scrutiny, the firm does not inspire trust. We need pollsters taking representative samples with a rigorous and dependable methodology. Unfortunately, that’s not PPP.
Once again, Silver took to Twitter to blast PPP:
Silver will continue to incorporate PPP’s questionable polls in its much-heralded “poll of polls.” But don’t worry because he is going to come up with a weighting method that is “punative” to PPP and other pollsters using sketchy methodologies.
In other words: Garbage in, garbage out.
Editor’s note: The headline of this post has been changed because we thought of a better one.