Research News

Polling by Probabilities?

IPR economist proposed, studied polling method that predicted Trump win


graph
Probabilistic polling, a method proposed by IPR economist Charles F. Manski, helped predict Trump's win.

In the wake of the 2016 presidential election, one of the major questions that pundits and electors are wondering is, “What happened with the polls?” Most of the major, national polls predicted a Hillary Clinton win. Yet a select few predicted a Donald Trump victory, in particular the USC/L.A. Times Daybreak tracking poll, which started forecasting a Trump win four months out from Election Day. 

Why did this poll get it right? One reason might be that it relied on an alternative method called “probabilistic polling,” which IPR economist Charles F. Manski has been conducting research on since the early 2000s. 

When pollsters call up voters, they usually ask a few standard questions: “Do you expect to vote in the upcoming election?” And, “If the election were held today, how likely is it that you would vote for Candidate X?” Then, they instruct respondents to reply with set verbal responses, such as “very likely, fairly likely, not too likely, or not at all likely.” 

Manski
Charles F. Manski

Pollsters who use the probabilistic polling method, however, ask voters to assign percentage chances to questions. For example, “What is the percent chance (from 0 to 100) that you will vote in the upcoming election?” And, “If you vote in the election, what is the percent chance that you will vote for Candidate X?”

While pollsters and political scientists have generally assumed that voters cannot accurately respond to percent-chance questions about future events, Manski has found that probabilistic polling can make pollsters’ work more accurate. 

“This idea of asking people to express uncertainty in a quantitative way eliminates the need for pollsters to distinguish between likely and unlikely voters, since people tell you that themselves in their answer,” Manski said.  

The success of the model, then, comes down to whether the percentages that voters express are actually tied to their likelihood of voting.

In work with Adeline Delevande of the University of Essex (U.K.), Manski studied the accuracy of voting probabilities in the 2008 presidential election. Their article, published in Public Opinion Quarterly in 2010, reported the first large-scale application of probabilistic polling to a presidential election. They found that probabilistic responses predict actual voting behavior beyond what is possible using verbal responses alone.  

In 2016, the USC/LA Times poll took a lot of flak for diverging from other polls. “It was the only major public survey that consistently showed Donald Trump winning,” wrote David Lauter, the newspaper’s Washington correspondent, on November 9. Ultimately, the newspaper underscored that it was this reliance on probabilistic polling that helped the poll to predict Donald Trumps’ victory.

“Early on, there was a lot of skepticism about whether ordinary people could answer questions like this,” Manski said. “But when you’re asking about things people are thinking about anyway, like an election, then the chances are it will work pretty well.”

Charles F. Manski is the Board of Trustees Professor in Economics and an IPR fellow. 

Photo credit: Pixabay.