In defence of Nate Silver and Data Journalism

Leon Wieselter accuses Nate Silver of “intimidation by quantification.” I accuse Leon Wieselter of idiocy.

Note: this was a post I drafted at the time of the time FiveThirtyEight launched. For various reasons I never got around to publishing it. The time that has passed has left me convinced that Silver’s new website is nowhere near as good as his old subsite at the New York Times was. However, I think that is because of problems with the execution and I stand this defence of what he is trying to achieve.

Since its launch a week ago: Nate Silver’s new data driven news site FiveThirtyEight has acquired plenty of detractors. I’m a big fan of Silver – my Kindle is signed by him – so unsurprisingly I’m supportive of his new project. However, at least some of these criticisms are valid. For example, Jonathan Chait’s warning that “Silver’s great added value was to bring basic statistical literacy to the fields of political forecasting and sports commentary, which are dominated by old-line hacks who rely on horse sense and either disdain data in any form or use data very badly. The new FiveThirtyEight tries to expand this revelatory contribution to other fields. The trouble is that many of those fields, like economics and climate science, already have real experts.”

That said there have also been some very weak arguments made. Chief among them those by Leon Wieselter, the literary editor of the New Republic and “one of the ideas men of the liberal intelligentsia.” He accuses Silver of a “slander” against Op-Ed journalism for his claim in an interview with the New York magazine that it “leads to a lot of bullshit.”

The contrasting approaches of the two men essentially represent another phase in the ‘Two Cultures’ debate. Like the coiner of the phrase, the scientist turned novelist C.P. Snow, worries that the ‘two cultures’ of science and the humanities don’t inform each other enough. For example, he says in his New York interview that “a weakness of conventional journalism [is] that you have beautiful English language skills and fewer math skills, and we hope to rectify that balance a little bit.” Wieselter by contrast follows his fellow literary critic and Snow’s nemesis F.R Leavis who believed that the application of science to areas traditionally studied by the humanities was degrading. He was written that “in the struggle to establish accurate and respectful relations between the sciences and the humanities, I am for a two-state solution.”

So what are Wieselter’s problems with data journalism?

1. He thinks Silver wants to make all journalism quantitative

he [Silver] cannot conceive of public reason except as an exercise in statistical analysis and data visualization.

This is transparently not true. Silver does not think all journalism has to be quantitative and you can see this elsewhere in Wieselter’s piece where he says that Silver “honors only investigative journalism, explanatory journalism, and data journalism.” Two of these are primarily qualitative.

2. He thinks data journalism can’t cope with questions of values

Many of the issues that we debate are not issues of fact but issues of value. There is no numerical answer to the question of whether men should be allowed to marry men, and the question of whether the government should help the weak, and the question of whether we should intervene against genocide.

This is true – trite indeed – but the reverse is also true. How likely the Republicans are to take control of the Senate in the Mid Terms is a factual question and Silver’s data driven approach has shown itself better able to predict such things than the prognosticators of the Op-Ed pages.

3. He seems to have a problem with objectivity

The intellectual predispositions that Silver ridicules as “priors” are nothing more than beliefs. What is so sinister about beliefs?…..He should be a little more wary of scorning them, even in degraded form: without beliefs we are nothing but data, himself included, and we deserve to be considered not only from the standpoint of our manipulability.

And why would one boast of having no interest in the great disputations about injustice and inequality? Neutrality is an evasion of responsibility, unless everything is like sports. Like Ezra Klein, whom he admires, Nate Silver had made a success out of an escape into diffidence. What is it about conviction that frightens these people?

What worries them is not convictions per se. Rather it is the risk of facts being bent for the sake of supporting a conviction.

Unfortunately this is how the brain works

Ezra Klein writes here at some lengths about the psychology of confirmation bias and how this screws up our political decision-making. And a famous study of the predictive abilities of political experts found that those “who had one big idea they were certain would reveal what was to come were handily beaten by those who used diverse information and analytical models, were comfortable with complexity and uncertainty and kept their confidence in check.”

People like Silver and Klein are alive to these risks. Silver discusses the Tetlock study at length in his book the Signal and the Noise. Thus what they are doing is not coming up with ways to banish convictions. Rather, they devise methodologies to ensure that convictions follow from facts and not the reverse. And anyone interested in an informed debate about public policy should wish them good luck in that.

A broader view

Appropriately enough, Wieselther’s hostility to data journalism is part of his broader convictions. He recently had a debate in the New Republic with the psychologist Stephen Pinker about the relationship between science and the humanities. Wieselther’s position was essentially that the methodologies of the natural sciences are never appropriate for studying human affairs. An argument which Pinker ably rebutted by among other things pointing out that subjects like archaeology and linguistics could never have made the progress they have without a host of scientific insights and methods.

Probably the most relevant aspect of this science/humanities debate to considering Silver’s new website is Wieselter’s absolute paranoia that once scientific methods are allowed into the humanities; they are all that will be allowed. He seems incapable of imagining a situation in which traditional approaches can co-exist with new ones borrowed from the natural sciences. This despite the fact that Pinker provides several examples of how the humanities have informed his own scientific work.

Wieselter seems similarly incapable of envisaging that data journalism and traditional journalism can co-exist. He therefore winds up attacking Nate Silver for something he has no intention nor prospect of doing. Far from making a convincing case for the merits of op-ed journalism, Wieselter winds up producing a misleading and overwrought piece that illustrates the potential deficiencies of his favoured medium.