There’s a new Morgan out today – a face to face poll showing a Labor lead on the primaries of 50 to 36.5, washing out into a TPP of 59/41 the same way. The sample size was 1838 for a hypothetical MoE of 2.3%.
Nothing really new there, but something that is pretty funny about Morgan face to face polling is the relationship between the sample size and the Coalition primary vote they estimate. If we run a scatter plot of the Coalition primary vote against the sample size of each Morgan face to face poll and whack in a quick linear regression line,we get:
Sample size explains just over 12% of the variation in the Coalition primary vote estimate in all Morgan face to face polls taken since January of 2008 (40 of them). For every 500 person increase in the sample size, we can expect a 1 point reduction in the Coalition primary vote and where that relationship is statistically significant at the 5% level (p-value of 0.02, and 0.01 using heteroskedasticity consistent standard errors & covariance )
I have absolutely no idea how or why this has turned out the way it has, or whether it’s just an unfortunate bit of random fallout that has come from them choosing small sample sizes when the Coalition vote has been up, and larger samples during periods when the Coalition vote has been down – but it’s pretty quirky and worth mentioning.
8 Comments
Very, very curious – and a very good pick-up as well, by the way.
How on earth did you notice it?
That is extremely bizarre. Maybe people see the researchers walking around more in bigger polls, and crystalize their hatred of the LNP? I’m grasping at straws though.
TD – I’ve been following what Simon Jackman has called the ‘Data Wars’ going on in US polling at the moment:
http://jackman.stanford.edu/blog/?p=1186
Which I might write a piece on over the weekend – but in Simon’s piece, he said “If/when the bias issue more or less neutralized, Internet will most likely kill RDD in terms of sampling variability due to the huge effective sample sizes” – which got me thinking about the current variability of online panels in Australia (Essential Report, and the Nielsen Online Panel when it publishes)
So I started looking through the data, but using phone polls and Morgans face to face polls as separate comparative yardsticks – and then it just sort of popped out.
The same thing doesn’t happen with any other pollster BTW, and with Morgan face to face, it only happens significantly with the Coalition primary vote – but not Labor’s and not the TPP’s.
Evan,
As far as straws go – that’s a pretty funny one.
The more people you ask, the smaller the percentage admit to supporting the Coalition. Quirky perhaps, but no mystery from my pov!
Could it be that larger sample sizes come from larger population concentrations which tend to lean Labor? By the way Poss, what is the r-squared value for this correlation?
C’mon Poss, you can’t extract any meaning from that line through those points!
The simple explanation might just be methodology. And simple human nature.
A Morgan phone poll is conducted from a centre in Collins St. The interviewer has no control over sampling.
A Morgan Face To Face Poll is conducted in field. The interviewer can influence sampling – to a point.
That is (and sweeping generalisations follow):
Your average Morgan Face To face interviewer has a choice as they stroll down your street. They can choose whether they knock on the door of No 52 or No 54.
They could make a value judgement as to which house looks – to them – more welcoming. Big black dog on the porch? Rusty old Falcon out front? Maybe I’ll go next door.
So, what I am saying here is that a Door To Door interviewer is likely to first approach households that they determine will give them the easiest interviewing experience.
It is my view that this may well lead to more “conservative” households being included in smaller sample size polls.
When the sample size increases, less conservative households come into play – as interviewers run out of “less desirable” households in their designated interviewing region.
Or it could simply be something more whacky.
Sounds like we need to test if there’s a correlation between voting intention and number of rusty Falcons on the front yard / breed of dog owned. Does the ABS have a data series for that?