Nate Silver, the audacity of maths and the innumeracy of political commentary

So maths works. Who knew? For weeks, the statisticians Nate Silver and Sam Wang have been predicting an Obama victory, on the strength of proper statistical models that include all relevant state polls. This morning, they’ve been utterly vindicated.

Assuming Florida indeed breaks for Obama, as returns indicate right now, Silver will have called all 50 states correctly. Wang will have 49 out of 50. Both have effectively gone 100 per cent — they each pinned Florida as a coin toss, but one took heads and the other tails.

As the US election drew nearer, Silver, in particular, has had to contend with some frankly awful political punditry that betrayed dreadful statistical innumeracy. Right up until yesterday, most of the mainstream media was still declaring the race too close to call. It wasn’t. Mitt Romney still had a chance of winning, particularly if the Silver/Wang assumption that all the polls were not systematically underestimating his support was incorrect. But Obama was overwhelmingly likely to win.

I think the hostility of many media pundits to Silver and Wang, and the way so much of the media essentially ignored their predictions and stuck with the “too close to call” narrative, says something interesting and important about political commentary. It’s a theme that plays a significant part in the Geek Manifesto’s chapter on the media. And it’s that the media is systematically unwilling to grant mathematics, statistics and science a significant role in politics.

Hardly anyone in frontline politics, or in the upper echelons of the media, has a background in mathematics or the natural sciences. Editors and newspaper columnists, on both sides of the Atlantic, tend to be graduates in politics and the humanities. They think instinctively that political commentary should mostly be done on feel. Hard numbers and evidence don’t generally come into it. A more scientific approach to political questions is somewhat alien.*

You can see this in the way Silver was derided by many pundits who simply didn’t understand the predictions he was making. And you can see it, too, in the choices that TV and newspaper editors make about their political commentators. Scientists are hardly ever asked for their perspective on political events: while cultural figures such as Will Self and Bonnie Greer are regular guests on Question Time, scientists, mathematicians and statisticians hardly ever feature.

In the past two years, Melanie Phillips has been on Question Time more often than all scientists put together. As far as I can make out, not a single scientist has appeared on the BBC’s flagship political discussion show this year. That says something.

The success of Silver and Wang shows that geeks have something rather important to contribute to political analysis, which is rather more valuable than much punditry. I hope editors take note.

*I’m generalising of course. There are political pundits who get this. My old Times colleague Danny Finkelstein, for example, is great on statistics and what they do and don’t mean.