Proving only that Bob Ellis is a tiresome, overrated bullshit artist.

It’s a conclusion that is difficult not to be drawn to.

What all the stuff about bragging rights for polls really comes down to is that you can’t judge a pollster’s accuracy on one election – you need to look at how the pollster has gone over many elections. Just like you can’t draw a sensible conclusion by polling only one voter!

On aggregating the polls, we would be making the assumption that public opinion was constant over the period and there’s not enough evidence to back that assumption. People can and do change their mind – to pick out that change, like we do with Pollytrend in the sidebar, we need a great many more polls than were conducted in the lead up to and during the campaign. If there were 30 polls in the 6-12 months before the campaign and 15 polls during the campaign, we could do some spiffy things to track a likely trend – but with the small amount of polling we had to work with it’s really beyond our scope unfortunately.

When you say on the polls:

they could have picked the result exactly - being within the margin of error isn’t the same as being exactly right

I’ll tell you something that will change the way you look at polls forever.

Let’s say we have a hypothetical poll, a poll so perfect in every way that it has absolutely no non-sampling error. The only difference between the results this hypothetically perfect poll produces and the true level of public opinion is in the sampling error which our MoE is derived from.

Let’s say that the true level of public opinion was 50% on some question, and that this perfect poll tried to estimate that true level of public opinion using a sample size of 2400 (which gives a margin of error of exactly 2%)

Whether the poll comes up with an estimate of 48,49,50,51,52 or something else is ENTIRELY up to chance.

1 in 20 times it will come up with a result that is outside of the 48-52 range – simply as a result of the mathematics of sampling and even though this hypothetical poll is perfect.

In the 95% of cases where this hypothetically perfect poll gets a result within the 48-52 range, it will be a random result drawn from a normal probability distribution with a mean of 50 and a standard deviation of 1.

That means the poll result will more often than not be closer to 50 than either 48 or 52, but the actual poll result itself is random.

If a pollster gets the result perfect, it’s luck. If three pollsters were measuring an election and the election result ends up within the MoE of all the pollsters final results – the pollster that got closest will claim they were the most accurate, but that’s just marketing spin as all the pollsters were equally right and it was nothing more than pure luck that one ended up being closer than the other.

Bigger sample sizes are always good, but the trade off is increased costs of the polling itself.Polling is actually pretty expensive business, so the larger the samples the higher the costs, which would mean less polls being commissioned over the course of a campaign for any media outfit with a set budget.

Caf,
I’m really hesitant about pooling those last two polls since they’re the only polls for the last week of the campaign, and the period both polls were in the field covers 3 days. To pool them we would be making an assumption that over those 3 days, public opinion was completely static. We could estimate a DEFF for them utilising previous polls to extrapolate the variance differences between Newspoll and Galaxy – but it would still be pretty much a sophisticated guess and the squeeze isnt really worth the juice since the election result would still fall within the 95% CI of the expected sampling error of the pooled poll.

Once the final election results are tallied up, the LNP primary will probably be a little higher than it currently is, the ALP primary a little lower than it currently is and the TPP a little lower for the ALP than it currently is. Both the primaries will be less than 1 point out from the final Newspoll, meaning most of the TPP error comes not from polling estimate issues, but from the way preferences were allocated by the polls to achieve a TPP estimate. In an OPV system, the TPP of the pollsters is pretty damn good!

Scorpio, thanks for that link, what a cracker of an example of what not to say!

He senses a conspiracy.

Proving only that Bob Ellis is a tiresome, overrated bullshit artist.

LOL Poss you might be interested in reading Bob Ellis’s take on polling

http://www.abc.net.au/unleashed/stories/s2519417.htm

He senses a conspiracy.

As Bob Ellis himself says, he gets it right sometimes.

His conspiracy theory might have legs if Newspoll relied on Rupert Murdoch for their income. I don’t think they do. I think they rely for their income mainly on businesses who want research to tell them what sort of advertising to run. One of the ways Newspoll touts for their business is by trying to impress them with the accuracy of their political polling (which I’m guessing is a small fraction of their work, but the only part that gets massive media coverage). Deliberately rigging their political polling would, for this reason, be throwing away their own bread and butter.

(The same would apply to all their pollsters.)

Some of the specific points Ellis makes actually cancel each other out, if you understand how the business works, which reinforces my doubts that he does.