With a fortnight to go, Barack Obama looks in big trouble on national polling averages, but retains breathing space on electoral college projections. The vagaries of polling methodologies might have something to do with this.
With less than two weeks to go, it’s high time for another presidential election thread – especially now that I’ve been inspired to put pen to paper by a fascinating article from Peter Kellner of YouGov, a British polling firm which has been sticking its oar into the American campaign.
The broad picture painted by the mountain of opinion polling is that a handy lead to Barack Obama disappeared after the first debate, and that to the extent that he is still favourite it is because he maintains slender leads in key swing states. According to RealClearPolitics, Mitt Romney now has a 0.5% lead on aggregated national polling after trailing by 0.2% two days ago. However, Nate Silver of FiveThirtyEight gives Obama a 70.3% chance of victory by virtue of state polling which shows, among many other things, an adjusted 1.9% lead to Obama in the likely crucible of the election, Ohio.
Until now, my favourite explanation for Obama’s stronger performance on electoral college projections has been that America’s decaying industrial “rust belt” is over-represented in the list of key states, which includes Wisconsin, Michigan and Pennsylvania together with Ohio. Romney’s image as a rapacious capitalist has by all accounts been especially damaging to him in these areas, owing to their long history of mass lay-offs and economic decline. This was illustrated when the General Motors bailout emerged yesterday in the context of what was supposed to be a debate about foreign policy, with Romney again haunted by his assertion from 2008 that the government should, as the New York Times subeditors helpfully paraphrased it, “let Detroit go bankrupt”. However, Kellner points to an intriguing alternative explanation involving polling methodology, with encouraging implications for Obama.
In the United States as in Australia, polling generally involves contacting random samples of respondents, the composition of which differs entirely from one poll to the next. However, the alternative approach, known as panel surveying, is to call back on the same set of respondents to determine how many are changing their minds. As Nate Silver observes, there are good reasons why this method is not generally favoured: the fact of being surveyed on multiple occasions may influence the way respondents behave, and a biased sample will produce consistently biased results, rather than random variation in the direction of errors from one poll to the next. The virtue of the approach is that it provides a more stable footing for evaluating changes over time, which is especially useful in the event of a significant shift such as that which the polls appeared to detect after the first debate.
As Kellner explains, YouGov fortuitously conducted just such a survey on a vast scale both before and after the debate. Whereas the RealClearPolitics aggregate saw a 4.3% lead to Obama on September 29 turn into a 1.3% lead to Romney by October 13, the panel survey found next to no change, with the small number of respondents switching from Obama to Romney matched by an equal share going the other way. What did emerge though was a crucial distinction in response rates from one survey to the next. Whereas the first survey elicited 33,000 responses, YouGov was only able to get 25,000 to complete the survey after the debate. This included 80% of those who indicated support for Romney the first time, against only 74% of the Obama supporters. That meant the raw numbers became immensely more favourable for Romney, and remained so after the data was weighted in the normal fashion according to demographics (by age, gender, region and race).
However, when weighting was further done according to party identification – so that responses from those identifying as Republican, Democrat or independent carried equal weight from one poll to the next – the effect of the differential response rates washed out, along with all but sliver of the swing to Romney. Responses are weighted in this fashion by YouGov as well as Rasmussen, but not by most other American pollsters. The argument against this approach (which, amusingly enough, has most often been heard from liberal critics of Rasmussen, which is renowned for its Republican lean) goes that party identification can change sharply in response to specific events, and that weighting for it negates their impact on voting intention. However, YouGov’s evident failure to find large numbers of individuals who changed their tune after the debate (allowing for the previously noted qualification that panel respondents may be shy about admitting they have changed their minds) suggests that, on this occasion at least, party identification weighting might have produced more meaningful results.
Nate Silver was moved to hypothesise that a lack of such weighting might cause polls to exaggerate bounces which occur in response to focusing events such as party conventions and clear debate victories. This is not to say that the poll shift to Romney isn’t meaningful, as the surge of enthusiasm which made Republicans more forthcoming when pollsters came calling could equally translate into higher turnout, with very real consequences for the outcome. However, Kellner offers a compelling counter-argument: that as the campaign intensifies with the approach of polling day, the enthusiasm gap and its attendant advantage to Romney will diminish. This may well be reflected in Obama’s lead in the swing states, where campaigning is already quite intense enough.