tip off
10

Qld State Polling and Bad Analysis

How two pollsters showing identical results can deliver two completely different accounts of voter behaviour – why timeliness matters if you want to avoid bad analysis

Today we see a Qld State-based Newspoll released by The Australian, accompanied by the usual bad analysis across the media spectrum that’s come to be a bedrock of the Australian political scene.

The Newspoll was taken across the three month period from July to September – where Newspoll adds Qld State-based political questions to their Qld component of their regular fortnightly national sample.  After 3 months, Newspoll has a large enough Qld sample to publish the Qld based results.

In the ordinary course of events at the state level, quarterly results like this tell a pretty normal story – but Qld has been far from ordinary over the last few months, so the Newspoll component from the early part of its sample becomes a bit of an exercise in nostalgia and telling us next to nothing about what has happened *within* the long period of the sample.

To demonstrate how timeliness in polling is important when it comes to understanding what is actually happening among the population you’re measuring, we can compare monthly ReachTEL polls that were taken over the same three month period that the Qld Newspoll was taken.

First up – let’s look at the two party preferred results over the July to September period.

The ReachTEL monthly polling averaged out over the 3 months produces an identical result to the Newspoll that was in the field across that same period.  Yet the polls tell two completely different stories because of the timeliness involved. Newspoll says the LNP has experienced a 3% swing away from it since the election and everything you’ve seen happening in Qld over the last few months has been little more than theatre.

The actual story is how the Newman government’s support has contracted from 68% in early July to 55% today – a massive 13 point drop – and one much more compatible with the events we have all witnessed in Qld.

We also see the same thing with the primary votes. Newspoll had Katter at a fairly ridiculous 1% in this poll (compared to their election result of 11.5%) – suggesting that they may not have added Katter to the list of party names they read out to respondents – yet Newspoll also had the broad “Others” at a massive 12 points.

UPDATE: Newspoll actually had Katter’s Australia Party on the readouts to respondents, making it an even more extraordinary result!

Comparing the ReachTEL and Newspoll results over the period tells the story – especially if we compare like with like and add the Others vote to the Katter Australia Party vote to accommodate Newpsoll’s weird treatment of KAP as 1%’ers. Again the results are nearly identical.

The Newspoll story is a 2 point primary swing away from the LNP compared to the election – a nothing to see here result – compared with the 12 point swing away from the Newman government since July. Same results for the two pollsters over the 3 months, but Newspoll washes out the underlying dynamics.

The approval ratings of Campbell Newman also exhibit the same behaviour. While Newspoll measures “satisfaction”, ReachTEL measures performance in terms very good, good, indifferent, poor and very poor. So we can use ReachTEL’s Total Good and Total Poor to compare with Newspoll’s Satisfied and Dissatisfied to give us a handle on generic approval.

Even with different questions containing different descriptions of generic approval, the results are again almost identical, and again tell two completely different stories. The Newspoll story is that Newman still popular while the ReachTEL story shows him crashing and burning like no other leader of a new government has in recent history. For instance, we can compare Newman’s approval with a bunch of other leaders of new governments to show the extent of the crash (click to expand).

So let this be a lesson in the dangers of timeliness when it comes to polling. Polling aggregated over long periods of time in dynamic political environments runs the risk of telling you a story that is at best meaningless and at worst completely wrong. Here the Newspoll and ReachTEL results averaged over the same period give identical results – but underneath, taking account of timeliness – the polls tell a completely different story. One is an accurate reflection of current Qld voter behaviour and the dynamics involved, the other is just bad analysis.

10

Please login below to comment, OR simply register here :



  • 1
    paddy
    Posted October 1, 2012 at 10:07 am | Permalink

    Thank goodness for a bit of sanity from Australia’s canniest marsupial. I thought those newspoll figures were dodgy, but I now understand why. Bravo again Poss.

  • 2
    Steve777
    Posted October 1, 2012 at 11:04 am | Permalink

    Interesting. Of course the error in interpreting results averaged over three months would have been picked up by a first year Statistics student. Was the bad analysis done by the pollster? If so, it was very amateurish. Or was it done by the Australian? And if so, was the poor analysis the result of incompetence, lack of time / resources or was what is an essentially meaningless result published to support the Australian’s political agenda?

  • 3
    Posted October 1, 2012 at 11:16 am | Permalink

    Steve,
    The bad analysis wasn’t done by Newspoll, but by The Australian – like is usually the case.

  • 4
    Widow Twankey
    Posted October 1, 2012 at 1:21 pm | Permalink

    Splash, leader … all because it suits The Australian’s agenda. Anyone who lives in Qld knows how much the tide has turned against Vandal Newman.
    Thanks Possum for your intelligent analysis of this poll. It shows up the stupidity and ongoing bias of “reporting” in the Oz.

  • 5
    JackLacton
    Posted October 1, 2012 at 4:28 pm | Permalink

    Or it just could be that the election is only recently over and nobody cares right now.

  • 6
    cud chewer
    Posted October 1, 2012 at 5:38 pm | Permalink

    It rather looks like The Australian came up with this story as a response to the Liberal’s losing it Federally and was another attempt to drive sentiment. It also looks like it was planned as a front page lead for some time. The story about Jones gets a small box further down the page. Yeah, right…

  • 7
    GeeWizz
    Posted October 3, 2012 at 1:24 pm | Permalink

    So Campbell is STILL getting 55% of the TPP, despite sacking 15,000 or so public servants.

    He is still more popular than Julia Gillard as leader and has a solid majority in parliament, all this ho-ha seems to be from union hacks unhappy they are getting the sack but most QLD’ers support his action.

    I can see this government lasting another 15 years or so.

  • 8
    chrispy 52
    Posted October 3, 2012 at 10:31 pm | Permalink

    He has actually lost a great deal of support, quite possibly a record in such a short space of time, as shown in one of the graphs above. This blog is a very good, accurate and fair analysis, unlike what we read in the Australian or hear from the Queensland Government.

    Campbell has to turn around a 2012-13 deficit of $10.8 billion, larger than Labor’s deficits of the last several years, and Labor had the global recession, failed privatisation plans, floods and cyclones, and tunnels and bridges to ease traffic congestion. The loss of support will quite possibly get worse as more and more people are affected by the reduced level of services.

    In year or so, we might see Campbell and Seeney out and Springborg and Nicholls in. Then 12-18 months after that, we could even see a new government, given the large proportion of swinging voters these days.

  • 9
    scorpio
    Posted October 15, 2012 at 9:06 am | Permalink

    GeeWizz, what a wangker!

    I can see this government lasting another 15 years or so.

    And how long do you think Can Do will be the head of it. Probably be lucky to last out the year and his wonderful government of incompetents better enjoy this three years.

    It may be a while after that before they get another chance!

  • 10
    Frank Sheehan
    Posted December 3, 2012 at 10:36 pm | Permalink

    Based on the QLD Poll numbers (more importantly where they are coming from) Katter’s Australian Party is going to win 8-12 seat on a national level. None of those seats will be taken from the ALP. In fact to win that number I expect Katter will pref the ALP. If Katter polls 8-12% on a national level that is polling 30-40% Primary in Regional NSW QLD. Remember these numbers have remained steady, Labors have improved. Unlike the Greens, Katter has appeal at the seat level (and that is where it counts). The Greens got 12-14% national for 1 seat, the nationals got how many seats on less than 5-7%. Katter is seen as a joke by the political pundits. The polls fail to take into account his appeal where it falls. You have the position which few talk about. Labor 36-38% Greens 12-14% Katter 8-14% Coalition ? Think he is an idiot now?

Please login below to comment, OR simply register here :



Womens Agenda

loading...

Smart Company

loading...

StartupSmart

loading...

Property Observer

loading...