We’ve been looking at house effects of the pollsters in our pursuit of better polling aggregations lately, but rather than looking at each Pollster individually, it might be worth looking at the results of Australian pollsters grouped by methodology.
If we use a LOESS regression to run a line of best fit through just the phone polls (Nielsen, Newspoll and Morgan Phone Poll) and compare it to the same regression using just the non-phone polls (which is face to face surveys with Morgan and an online panel with Essential Report), the difference between the phone polls and the non-phone polls is considerable for both the primary vote and TPP estimates.
What slaps me around the face instantly here is the way that the different results between the two groups are neither consistent in their gap, nor is there any apparent pattern in their gap. Apart from a period in April, the two groups have never really been in agreement, except to say that they are unanimous in the reality of the ALP thumping the Coalition.
Currently the non-phone polls are 5 points more favourable to the ALP on the primary vote, while the phone polls are around 3 points more favourable to the Coalition on the primary vote. The two party preferred vote has the non-phone polls being around 4.5 points more favourable to Labor at the moment. But next month or at any arbitrary period in the future, those gaps are likely to change in their magnitude if history is anything to go by.
5 Comments
I don’t know much about this but I would question grouping the online Essential Report with the face-to-face polls.
It would seem to me that answering questions online (talking to a computer) is about as far removed for answering them face to face with a real person as your can get, with answering to a real person over the phone somewhere in between.
I’d agree with prettyrage above. Answering questions onlines, just like posting a blog entry such as this, forces you to notice your response in a much more calculated way than answering questions either on the phone or face-to-face. In a way it ought to most accurately represent actual voting results, IMO, because it has the most similar profile (privacy, chance to think, written responses) to marking a ballot.
Perhaps this should be a requestathon suggestion. Do Essential online polls, all other things being equal (which they aren’t) have any better predictive power in the final week before the election?
I’m grouping this way not because of any similarity between methodologies (apart from Newspoll and Nielsen both being phone polls of course!), but because of the similarity in results.
That in itself is interesting because as you two said, you cant really get two more different types of questioning methodology. But even though the methodology used to generate responses is vastly different, the results are very similar. In terms of which poll’s numbers are closer to which other poll’s numbers, Essential Report is closer to Morgan Face to Face results than any other pollster’s results (and vice versa). The same relationship exists between Nielsen and Newspoll.
Unfortunately I haven’t got any pre-2007 election Essential Report polls to compare to the 2007 election result, but even if I did it wouldn’t be of much use because Essential, being a relatively new pollster, has undertaken at least one change in their methodology (possibly more) since the election in their pursuit of generating a more accurate poll.
So comparisons between now and what happened with the 2007 election would be a bit of an exercise in apples and oranges.
A good question is why Morgan and Essential produce such similar estimates?
Poss, Any chance it has something to do with the mobiles thing I mentioned in requestathon (although this wouldn’t explain the reversal of the results in early 08)?
The way the gap size changes over time just indicates that the slightly different groups found by the different polls not only have different political opinions, but change those opinions in response to events in a different way.