We all know about our Pollytrend chart – where we build a rolling average of the most recent poll from each pollster weighted by sample size, then run an aggressive locally weighted polynomial regression through the results to give us an adaptive trend line. While Pollytrend is good for picking up medium term shifts in voter sentiment, some of us question it’s accuracy when it comes to it’s actual level at any given point in time. As the two pollsters with the largest sample sizes (Essential and Morgan Face to Face) have a slight relative lean to the ALP in the vote estimates compared to the phone pollsters, it forces the entire Pollytrend line to have a relative lean as well.
Today we’ll solve that concern by introducing a new addition to our Pollytrend – the Phone Trend. The phone trend line is calculated a little differently to our all pollster Pollytrend line.
First and most obvious, it uses only phone pollster data.
Secondly, only the most recent two polls of each phone pollster get entered into the initial calculations. We use the most recent two polls because any given pollster’s “poll before last” still contains valuable information about the state of public opinion today, but the information is just not as valuable as their most recent poll. In order to differentiate between “best” information (the most recent poll) and “second best information” (the poll before last) from any pollster – we simply take a rolling all phone pollster average of the two most recent polls from each phone pollster, but weight all results by time – so the further back in the past a poll was taken, the less weight that particular poll carries when we calculate the weighted average. For example, a poll taken 60 days ago will only have one tenth of the weight of a poll taken today.
Thirdly, every time a phone poll is released we calculate our new weighted average to build ourselves a time series.
Finally, as we do with the all pollster Pollytrend, we then run a locally weighted polynomial regression through the results (which itself gives more weight to the most recent values compared to past values) and we end up with a relatively smooth and highly adaptable trend line that carries the advantage of being able to accommodate the uncertainty of sampling error in the polling results – giving us a phone poll trend line that attempts to “see through the short term noise”.
When we compare the all pollster Pollytrend against our new phone poll trend, this is what we end up with.
We’ll also run a version of it that covers the last 12 months in the sidebar to replace our old Pollytrend chart:
22 Comments
Doesn’t the phone polls favour the coalition, since phone polls miss out on people who don’t have land lines? (This is something that loads of pundits discussed during the US 08 election.)
More and more 18-24 year olds (who typically favour Labor) don’t have land line phones and rely almost exclusively on mobile phones.
Older people (who typically favour the Coalition) are more likely to have land lines and thus tip the phone trend towards the conservative side.
Not that we’ve seen so far. Cell phone only households will probably become a problem in the future, but as far as anyone can tell they’re not a significant problem at the moment.
Counting up all the dots- well some of them I noticed earlier this year that there was a lot more at 58% and every now and then I had a look and saw this flat line was still common- Is there any statistic that it reflects – like the mid point or whatever of the polls?
Poss – here’s how the mobile has extended into my mob of late.
3 kids in 40s married with kids (4 teenage g/kids) – only 1 house has a landline but that will go shortly. Works out at 6 adults +4 = 10 mobiles and 1 landline.
The landline family rarely answer the darn thing so we now ring the mobiles instead. Every house uses wireless broadband. It’s been a great surprise to us that they don’t feel the need for a landline.
This research agrees with you, but says there may be a problem if you want to find out what young age groups are thinking:
From:
http://www.srcentre.com.au/Mobile%20Phone%20Only%20White%20Paper%20v4.pdf
Geoff – the 58% estimate for the ALP two party vote is the number where the phone polls and the non-phone polls have tended to overlap, often hitting 58 at the same time.
The phone polls generally wander around between 55 and 59 while the non phone pollsters generally wander between 57 and 62 – so the 58 (and to a lesser extent 59) is the result that pops up the most… albeit often at different times.
Thanks for that Don, I didnt realise that paper existed (or if I did, had completely forgotten about it).
Sorry Pos, you’ve lost me there. Which exact sets of data go into each of these trend lines? I was under the impression that PollyTrend was phone poll only (didn’t include Morgan’s face to face). Obviously I’ve missed something
The other problem I have is you really need to colour the data points so we know which points belong to which trend line (and which belong to both).
Cud, Pollytrend has always been an All Pollster trend. The new Phone Pollster Trend is just phone pollsters.
The dot points are the actual two party preferred poll results from all pollsters – so all of the dots belong to the All Pollster Trend while only a subset of those dots (the phone pollster results) belong to the new Phone Pollster trend.
I’m a bit hesitant to start coloring the actual poll results for the sidebar chart – but I probably could do a specific chart that colors the phone vs. non phone when I update the Pollytrend every polling cycle.
Ok fair enough. I was partly confused because the most recent Morgan poll doesn’t seem to be included yet.
Sorry Cud – I hadn’t added the latest Morgan result to the chart (even though the trend lines have been calculated with it). I’ve fixed that up.
No – you weren’t really going crazy!
so does one of the polls seem to move first?
The phone polls appear to have a slight leading indicator quality – which would be expected since both non-phone pollsters generally run their polls over a two week period, as opposed to the phone pollsters single weekend period.
I think I’m confused. I was under the impression there was a new poll coming out tomorrow which might give Kevin Rudd another rude awakening. So I’ve been pottering around on a very hot day, making the odd comment or two on Crikey. I come back home to discover another discussion broiling away. And now it’s time to do someting about the house. Damn!
Keep up the good work Poss. Each new graph brings us closer to the truth…
On the issue of mobile only households, I can offer some anecdotal evidence. My partner and live in a mobile only household, and counting on my fingers, I can easily fill both hands with friends I know who are also living without landline. Come to think of it, I’m not sure if I can think of a student household I’ve visited that does have a landline. I have two friends who work for the same telemarketing/research company, and amongst them and their colleagues, it’s de rigeur to whinge about how out of touch their superiors are on this issue. They quite firmly believe that much of their data is skewed because of it.
Nick,
The big question comes down to whether cell phone only households are significantly behaviourly different to identical households (in terms of age, income and education) that have landlines. So far the evidence seems to be a bit mixed. As long as cell phone only households and landline households remain approximately the same in terms of their behaviour (controlling for basic demography) then the “problem” of cell phone only households skewing results will remain in the “mostly hypothetical” column where it sort of sits at the moment.
It’s when a significant difference starts to emerge between the two household types that will start causing pollsters grief. The big issue is more about when that will happen rather than if!
In other words, so long as (for eg) student households with/without landlines behave similarly, it doesn’t matter that students overall are under-represented because that is already being corrected for.
My comment on 18-24 year olds and the mobile phones is that they’ve probably always been a difficult group to capture, so being on mobiles makes no difference.
When I was in that age group (about thirty years ago) I didn’t have a landline until I bought my own home and I know it was the case with many of my friends.
I would simply assume that this has always been a hard group for pollsters to capture.
Anecdotally (and I seem to be polled with remarkable regularity, by different firms) I’ve been dismissed for polling purposes a couple of times because ‘we’re looking for someone 24 or younger, we’ve got our quota in the other age groups’ – which would tend to bear this out.
The state polls see Labor down in every state and the LNP well ahead in three…
they also appear ancient surely there has been a poll in W.A since March…
I wonder how they would look today….
Peter, the only poll taken in WA since the election has been a few Westpolls which I don’t touch with a barge pole
Don, Poss and Zoomster
Does the the research above about landline versus mobiles has a fatal assumption?
As zoomster points out they will reach the predetermined quota/ratio of 18-24 year olds. Therefore, the question is not, for me, about who has a landline and who doesn’t. They’ll get their numbers.
The question is:
Are the 18-24 year olds who are phone polled different to those that are not?
Here is my guess, taking into account zoomsters comment about being asked for 18-24 year olds: the polls will be over represented by 18-24 year olds who live with their parents because they have a landline. For the 18-24 year olds who don’t live with their parents a solid majority will rent. In rentals, amongst the people I know, landlines are increasingly uncommon. Additionally it is also increasingly common that if a ‘landline’ exists that it’s VOIP on naked DSL (basically a freebee with the internet). Do pollsters have VOIP numbers? My wife and I (26) have VOIP and are not listed in the whitepages.
It would be my contention that with phone polls I think it’s likely you will reach a population of 18-24 year olds with demographics different to a random sample of 18-24 year olds.
The next question then is if these demographic differences effect voting… I ‘feel’ that people renting are more likely to vote labor than those living at home. But this is based on an assumption.
For history buffs:
In 1948 there was a presidential election in the US. It was Dewey (republican) versus Truman (Democrat). ALL the polsters predicted a Dewey landslide. In fact, one newpaper printed a front page with a the heading Dewey wins then took a photograph of Dewey holding a copy of the front page. Very embarrassing.
Dewey lost by heaps. When the problem of why the pollsters got it so wrong was analized, it turned out the pollsters relied on telephone polling. Seems Republicans were on average richer than democrats and therefore a phone poll was more likely to reach a Republican