On November 3, two oppositional forces went head to head and the results were…divisive. With commentators and pundits still reeling from the poor performance of US election pollsters, it seems fitting to ask — can AI (ultimately) solve a problem like election prediction? 

At least this time around, the answer seems to be no, not really. But not necessarily for the reasons you might think. 

Here’s how it went wrong according to Venturebeat:

Firms like KCore Analytics, Expert.AI, and Advanced Symbolics claim algorithms can capture a more expansive picture of election dynamics because they draw on signals like tweets and Facebook messages…KCore Analytics predicted from social media posts that Biden would have a strong advantage — about 8 or 9 points — in terms of the popular vote but a small lead when it came to the electoral college. Italy-based Expert.AI, which found that Biden ranked higher on social media in terms of sentiment, put the Democratic candidate slightly ahead of Trump (50.2% to 47.3%). On the other hand, Advanced Symbolics’ Polly system, which was developed by scientists at the University of Ottawa, was wildly off with projections that showed Biden nabbing 372 electoral college votes compared with Trump’s 166, thanks to anticipated wins in Florida, Texas, and Ohio — all states that went to Trump.

For many — like Johnny Okleksinski back in 2016 — the instinctive reaction is to claim these misfires are down to flawed social media data which is simply not reflective of real world populations. In 2018, 74% of respondents agreed and told Pew Research that: “content on social media does not provide an accurate picture of how society feels about important issues.” 

But while it’s certainly true that some of these inaccurate AI forecasts were down to the under-representation of certain groups (e.g. rural communities), an interesting paper published earlier this year by open access journal MDPI suggests that social media analysis can actually be more reflective of real-life views than these results might indicate. 

The authors of Electoral and Public Opinion Forecasts with Social Media Data: A Meta-Analysis acknowledge the debate around the usefulness of social media in understanding public opinion, but at the same time they caution that dismissing social media’s predictive capacity based on its inability to represent some populations actually misses an important dynamic — namely, that politically active users are opinion-formers and influence the preferences of a much wider audience, with social media acting as an “organ of public opinion”: 

…the formation of public opinion does not occur through an interaction of disparate individuals who share equally in the process; instead, through discussions and debates in which citizens usually participate unequally, public opinion is formed. 

In other words, although political discussions on social media tend to be dominated by a small number of loud-mouthed users (typically early adopters, teens, and “better-educated” citizens), their opinions do tend to pre-empt those that develop in broader society. 

Further, in capturing political opinions “out in the wild”; social media analysis is also able to understand the sentiments of silent “lurkers” by examining the relational connections and network attributes of their accounts. Report authors state that, “by looking at social media posts over time, we can examine opinion dynamics, public sentiment and information diffusion within a population.”

In brief: the problem with social media-fueled AI prediction does not appear to lie within the substance of what is available via online platforms. It seems to be in the methodology and/or tools. So, where do predictive AI tools go wrong? And where can researchers mine for the most useful indicators of political intention? 

One of the major areas where social media analysis seems to break down is with language. This intuitively makes sense when we think about how people express themselves online. Problems with poor grammar or sarcasm are doubtless compounded by the difficulties of trying to understand context. Similarly, counting likes, shares and comments on posts and tweets is viewed as a fairly thin and simplistic approach (to use Twitter parlance “retweet ≠ endorsement”).  

More robust, according to report authors, is an analysis that considers “structural features”, e.g. the “likes” recorded to candidate fan pages. Previous research found that the number of friends a candidate has on Facebook and the number of followers they have on Twitter could be used to predict a candidate’s share of the vote during the 2011 New Zealand election. But there is still the problem of which platform to focus on for thw closest accuracy. 

Most AI systems use Twitter to predict public opinion, with some also using Facebook, forums, blogs, YouTube, etc. Yet each of these suffer from “their own set of algorithmic confounds, privacy constraints, and post restrictions.” We don’t currently know whether using multiple sources (vs. one platform) has any advantage, but with newly popular players like Parler on the scene, there’s reason to believe that covering several platforms would yield an accuracy advantage (though few currently use a broad range). 

Finally, the actual political context within which the social platforms operate likely plays into their predictive accuracy. The report in question recalls that the predictive power in a study conducted in semi-authoritarian Singapore was significantly lower than in studies done in established democracies . From this authors infer that issues like media freedom, competitiveness of the election, and idiosyncrasies of electoral systems may lead to over- and under-estimations of voters’ preferences. 

---

As the VentureBeat article spotlights, there are additional problems to counter for AI, like the electoral college system and the way legal challenges, faithless electors (members of the electoral college who don’t vote for the candidate they’d pledged to), or other confounders might affect the outcome of a race. Nevertheless, this does still appear to be an area in which good AI could deliver a more accurate picture of human intention than what currently exists via human endeavor. 

There’s no getting away from the fact that in this new political world, traditional polling has failed time and time again. And it’s not as though voters can sit back and rely on political punditry — “expert on experts” Philip E. Tetlock once wrote that the assertions of political commentators were often no more reliable than “a dart throwing chimp.”

So, as we look for a better bellwether, could AI have found its opportunity to convince a global population of its usefulness? Well, there’s always an election around the corner and time to find out….