Predicting the Bern

Even though Bernie Sanders is leading Hillary Clinton in recent primary polls, conventional news and political structures have been reluctant to recognize Bernie Sanders as a legitimate candidate. Most likely it’s because Bernie Sanders doesn’t fit the standard profile of a politician. What many analysts are missing, however, is that is what people want.

There is a fatigue with the old way of doing things. Legacy systems of reporting and gathering data seem slow to catch on to this, however. As technology advances, it is becoming more obvious that old methods of predicting elections are unreliable. This was highlighted when 538’s Nate Silver predicted the 2012 election with 98% accuracy. His accuracy showed a new method which predicted results more precisely than conventional methods.

What 538 showed the world in 2012 was the rise of predictive analytics, i.e. using big data and advanced algorithms to draw accurate conclusions about future events. What separates the 538 method is going beyond identity politics and seeing deeper into real trends.

Previous templates for gathering predictive data would make assumptions about subcultures, not simply to be biased or lazy, but because enormous amounts of information wasn’t available to assimilate and analyze.

For example, data used to be collected via phone surveys. “Likely voters” were called on old landlines and asked a series of questions about how they felt about the candidates. In 2008, this method was shown to have major flaws in seeing Obama’s popularity with youth voters, because they largely had cell phones. The youth and minority voters were also generally ignored because of their history of not voting. Both assumptions created a faulty data pool that incidentally created poor predictions. This was because they ignored how popular Obama was in those categories even though it was widely known. Now, it is much easier to gather quantitative passive data via social media that reflects the mood of the electorate.

When we use a greater share of available data, we see where certain biased assumptions are wrong. A good example of this is in the race between Bernie Sanders and Hillary Clinton for the Democratic nomination. There is a presumption that the Hillary has the women’s vote sewn up by virtue of being a woman. But recent analysis has shown that young feminists are shying away from Hillary.

The more women learn about Bernie Sanders, the less they are compelled by identity politics. The thought is that, although it would be great for women and the glass ceiling to see a woman become President, Bernie Sanders’ message of social and economic justice better represents the issues that are important to women.

Historical predictors like identity politics are not going away. But analytics recognize them for what they are, a piece of the pie. As we watch this election unfold, those who will be surprised by a Bernie Sanders victory are the ones ignoring the data.