Out of today’s Fairfax press comes the results of the JWS Research poll (full results here) we’ve all been hearing so much about. With a total sample size of 28,000, (no you didn’t read that wrong, twenty eight thousand!) some of you folks would have either been polled yourself last Saturday and Sunday, or know of someone that was.
UPDATE: The polling itself was actually undertaken by Telereach on behalf of JWS Research.
What makes this poll so interesting (apart from it being so enormous that only the ABS Australian Labour Force Survey that gives us our unemployment rate in Australia is larger) is that it’s not your standard phone poll.
Phone pollsters ordinarily have banks of people manning telephones, actively interviewing respondents. This JWS poll however, is what is called an automated IVR poll – Interactive Voice Response – otherwise known as a “robopoll”.
With IVR polls, there are banks of machines that phone random households and a recorded voice asks questions which the respondent answers via pressing the keypad on the phone.
The big question about IVR polls all over the world is whether they create some form of structural bias in the results they produce, particularly as a result of having a non-response bias that may be hard to adjust for, but also as a result of a few other technical survey issues.
Yet, the thing that really stands out with research into IVR – particularly in the US where it is very common (e.g. Rasmussen polls) – is the absence of evidence of a consistent skew in political IVR polling that can be put down to the methodology itself.
This is the first full blown IVR poll on voting intention I’ve seen in Australia – so it’s worth comparing the results of this poll against results of other phone pollsters. To get a robust sample at the state breakdown level of the traditional phone pollsters, we’ll have to do a two week combined sample (compared to JWS poll which was taken last Saturday and Sunday). We have a two week Newspoll breakdown in The Oz today, two Nielsen polls and two week’s worth of an unpublished phone poll taken over the last two weekends. Combined, these 3 pollsters give us pooled sample of approximately 8200.
If we look at the state and national level results of our ordinary phone pollsters on the ALP two party preferred compared to the results of JWS, we get:
The results are pretty close – with JWS being a little more favourable in WA for Labor and a little less favourable in SA and Qld. But even with those differences, it’s right in the ballpark of what the other phone pollsters are getting with the national result only differing by 0.1% (though the high accuracy of that is probably more to do with luck than anything else by all pollsters)
The other issue here is that two of the three phone pollsters in this pool found a move towards Labor last weekend compared to the result they found the week before (probably all three did, with Newspoll’s move being washed out by rounding to the nearest whole number if one had to make a guess). So our two week’s worth of traditional phone pollster results is slightly lower for Labor than what was happening on Saturday and Sunday when JWS was in the field, simply because Labor was polling lower during the first week of this two week sample.
So JWS appears to be as accurate as the traditional phone pollsters – where, if there was any lean towards one party or the other, it would be more likely to be a slight lean towards the Coalition than to Labor (as the phone pollsters combined had Labor in a better position during last Saturday and Sunday than did JWS – but the sample is much smaller so it could easily be just random variation)
I’m not quite yet ready to say that I welcome our new Robopoll Overlords, but these initial results are striking in their similarity to traditional phone pollsters, and those sample sizes – 28000! – well, that’s enough to make you moist.
Next up – we’ll break out the election simulations.