Over the weekend, the WaPo’s Dana Milbank wrote an op-ed that was embraced by the White House accusing the media of treating Biden “as badly as — or worse than — Trump” using a data tool called “sentiment analysis” as his proof:
Submitted for your consideration:https://t.co/h5lHm0sZAp
— Ronald Klain (@WHCOS) December 4, 2021
Except, using “sentiment analysis” of media articles to compare ones about Trump to ones about Biden is “complete crap,” writes data expert Nate Silver:
To this good thread explaining why the "sentiment analysis" cited in the @milbank WaPo article this weekend is complete crap—the analysis was used to make the claim that the press is just negative toward Biden as Trump—I'll also add a couple of comments based on their data. 1/ https://t.co/V8pPJoM13p
— Nate Silver (@NateSilver538) December 6, 2021
“It’s just totally random,” Silver tweeted. “Lots of stories about the stock market” and “Many have nothing to do with Biden at all”:
These are the articles the algorithm found, out of more than 40K stories, were the most favorable toward Biden. It's just totally random. Lots of stories about the stock market. Many have nothing to do with Biden at all ("Haiti President Assassinated At Home, Wife Wounded"). 2/ pic.twitter.com/ubKvmvaMHi
— Nate Silver (@NateSilver538) December 6, 2021
Silver added, “What’s probably happening here is that words like ‘high’ and ‘rise’ are viewed as favorable by the algorithm, even if they’re taken completely out of context (e.g. ‘higher taxes’ or ‘stock futures rise’)”:
What's probably happening here is that words like "high" and "rise" are viewed as favorable by the algorithm, even if they're taken completely out of context (e.g. "higher taxes" or "stock futures rise"). 3/
— Nate Silver (@NateSilver538) December 6, 2021
This is happening with the articles least favorable to Biden, too:
Here are the articles that the algorithm deems *least* favorable to Biden, by contrast. These are a little bit more on the mark, but still, mediocre at best. Again, lots of stuff about stock or commodity prices. And lots of polling stories, e.g. about Biden's approval rating. 4/ pic.twitter.com/SQY7FlMpIN
— Nate Silver (@NateSilver538) December 6, 2021
Heckuva job, Dana:
I know some of you would say the media should have fewer polling-driven stories. I mostly disagree, I guess. But, I don't think it reflects some sort of bias if Biden's approval rating drops 5 points and the news outlet writes about that, and the framing is mostly negative. 5/
— Nate Silver (@NateSilver538) December 6, 2021
Silver ran one of his own articles — “exactly the sort of story that @Milbank says there should be more of” — through the algorithm to test it. . .
I also looked at how FiveThirtyEight stories were classified, and again it's fairly random. Here is a story we did on Trump/GOP efforts to undermine democracy, for example. Exactly the sort of story that @Milbank says there should be more of (I agree). 6/https://t.co/EgipxruBBf
— Nate Silver (@NateSilver538) December 6, 2021
. . .and found that an article calling out Trump for attacking democracy only came back as “neutral”:
How was the story scored? As being *neutral* to Trump (-0.0176). Now I think the story was fair and accurate (it's accurate to report that Trump poses big threats to democracy). Still, if this is the sort of story you want more of, the algorithm isn't helping to ID them. 7/
— Nate Silver (@NateSilver538) December 6, 2021
In other words, it’s “complete crap”:
Designing good algorithms is hard, but this is an especially bad one.
And as a news consumer, you should be extremely wary of statistical methodologies you don't understand but that confirm your priors. 8/8
— Nate Silver (@NateSilver538) December 6, 2021
***
Related:
Ron Klain appreciates Dana Milbank & Margaret Sullivan’s takes on how the media has failed Biden https://t.co/6arWIR8sVA
— Twitchy Team (@TwitchyTeam) December 4, 2021
