Nate Silver admitted that his new site FiveThirtyEight is “making plenty of mistakes,” so he’s “happy to forgive one of” Paul Krugman’s. Silver won’t win any medals for humility. (More)
Squirrels are big on data, as I explained last week, so I’d really like to see Silver’s new data journalism site do well. Except for his latest forecast for the Senate midterms. Democratic activists are already working hard to prove him wrong on that, and I’m firmly in their corner.
He’ll update that prediction regularly as the election gets closer and state-level polls come in, and he may get a chance to change his mind. Or maybe he’ll stick to his analysis and be proven wrong. That would be good for the country, as a GOP-held Senate would cripple President Obama’s last two years. It might also be good for Silver, who could use a lesson in humility.
In his manifesto titled What the Fox Knows, Silver praises original reporting but condemns pundits:
My disdain for opinion journalism (such as in the form of op-ed columns) is well established, but my chief problem with it is that it doesn’t seem to abide by the standards of either journalistic or scientific objectivity. Sometimes it doesn’t seem to abide by any standard at all.
True enough, or at least it’s often true, but he went even further in a New Yorker interview:
Plenty of pundits have really high IQs, but they don’t have any discipline in how they look at the world, and so it leads to a lot of bullshit, basically.[…]
The op-ed columnists at the New York Times, Washington Post, and Wall Street Journal are probably the most hedgehoglike people. They don’t permit a lot of complexity in their thinking. They pull threads together from very weak evidence and draw grand conclusions based on them. They’re ironically very predictable from week to week. If you know the subject that Thomas Friedman or whatever is writing about, you don’t have to read the column. You can kind of auto-script it, basically.
It’s people who have very strong ideological priors, is the fancy way to put it, that are governing their thinking. They’re not really evaluating the data as it comes in, not doing a lot of [original] thinking. They’re just spitting out the same column every week and using a different subject matter to do the same thing over and over.
Okay, I still agree. Sort of. Here’s Leon Wieselter’s defense of pundits at The New Republic:
The state of American punditry is not strong. A lot of it is lazy, tendentious, and lost to style. But Silver’s outburst is nonetheless a slander. There are all sorts of pundits just as there are all sorts of quants. The editorial pages of the Washington Post in particular are regularly filled with analytical and empirical seriousness. But Silver wishes to impugn not only the quality of opinion journalism, he wishes to impugn also its legitimacy. The new technology, which produces numbers the way plants produce oxygen, has inspired a new positivism, and he is one of its princes. He dignifies only facts. He honors only investigative journalism, explanatory journalism, and data journalism. He does not take a side, except the side of no side. He does not recognize the calling of, or grasp the need for, public reason; or rather, he cannot conceive of public reason except as an exercise in statistical analysis and data visualization. He is the hedgehog who knows only one big thing. And his thing may not be as big as he thinks it is.
Numbers can’t answer every or even most political questions, Wieselter emphasizes. No equation can determine whether we should recognize marriage equality, or our social responsibility to the needy … or at least no equation that does not embed moral values – Wieselter calls them “beliefs” – in its analysis. He continues:
The intellectual predispositions that Silver ridicules as “priors” are nothing more than beliefs. What is so sinister about beliefs? He should be a little more wary of scorning them, even in degraded form: without beliefs we are nothing but data, himself included, and we deserve to be considered not only from the standpoint of our manipulability. I am sorry that he finds George Will and Paul Krugman repetitious, but should they revise their beliefs so as not to bore him? Repetition is one of the essential instruments of persuasion, and persuasion is one of the essential activities of a democracy. I do not expect Silver to relinquish his positivism – a prior if ever there was one – because I find it tedious.
But you can’t be an effective fox just by letting the data speak for itself – because it never does. You use data to inform your analysis, you let it tell you that your pet hypothesis is wrong, but data are never a substitute for hard thinking. If you think the data are speaking for themselves, what you’re really doing is implicit theorizing, which is a really bad idea (because you can’t test your assumptions if you don’t even know what you’re assuming.)
Timothy Egan joins the chorus of those dismayed by Nate Silver’s new FiveThirtyEight. I’m sorry, but I have to agree: so far it looks like something between a disappointment and a disaster.
But I’d argue that many of the critics are getting the problem wrong. It’s not the reliance on data; numbers can be good, and can even be revelatory. But data never tell a story on their own. They need to be viewed through the lens of some kind of model, and it’s very important to do your best to get a good model. And that usually means turning to experts in whatever field you’re addressing.
Unfortunately, Silver seems to have taken the wrong lesson from his election-forecasting success. In that case, he pitted his statistical approach against campaign-narrative pundits, who turned out to know approximately nothing. What he seems to have concluded is that there are no experts anywhere, that a smart data analyst can and should ignore all that.
Quartz’s Allison Schrager makes a similar point:
In a perfect world the data would just speak for itself, but that’s never the case. Interpreting and presenting data requires making judgments and possibly mistakes.
Schrager goes on to make some claims that I dispute. For example, I don’t think data journalism should exclude regression analyses merely because lay readers lack the training to replicate them. Instead, good data journalism should explain – in clear, non-jargon language – what a regression analysis is and why one was needed for that data. I’m less worried about data journalists using obscure math than about them using obscured models … hiding their ideological biases behind ‘simple’ calculations.
In his letter to Talking Points Memo, Silver agrees with some critics but disagrees with Krugman, arguing that he has experts writing for FiveThirtyEight and that Krugman himself routinely mocks the Very Serious People – often economists – who dominate the Beltway dialogue. Silver concludes:
I haven’t been surprised by Krugman’s criticism because I’ve fired some shots at the New York Times editorial page, of which he’s a member. Krugman was full of praise for FiveThirtyEight while it was part of the New York Times.
In short, I think this was a piece Paul wrote with his pundit hat on – it wasn’t making an attempt to be fair. But we’re making plenty of mistakes and I’m happy to forgive one of his.
That’s almost a case study in humble-brag, as was his “it wasn’t all that hard” explanation after reminding readers that he forecast every state correctly in the 2012 presidential elections. If someone gave Silver a medal for humility, they’d have to take it back or he’d wear it to every press interview and TV appearance.
And his humility medal would clash with mine.
Good day and good nuts