Introducing Pollytrend

Today we bring a new metric to the blog specifically designed to measure change in political opinion. We’re all by now familiar with the Pollytrack series – the weekly polling aggregation of all pollster results weighted by individual sample size. The Pollytrack series is mostly good enough for ordinary periods of political activity, but it has a few problems when it comes to measuring real time political trends.

Firstly, it will fail dismally in election campaigns where its 7 day time period will be swamped by the reality and speed of political events. Secondly, while the mechanics of Pollytrack massively reduces polling noise that may arise from pure sampling error (the margin of error that we see given with each poll), we still expect some level of random sampling noise to infect the results. Depending on which side of the real (yet unknown) level of public political opinion that random error falls, over a couple of polls it can give us slightly erroneous impressions of a trend forming or being in action when one might not actually exist.

The final problem with Pollytrack when it comes to measuring the change in public opinion comes from it being a pooled sample of all the pollsters. If any single pollster has a bit of a bad day and produces a poll that shows a large change in support levels for one or both political parties from one poll to the next – even if no other pollster measures a similar change during the same period, that single result can still throw our Pollytrack out by a point or so – especially with the pollsters like Morgan face to face and Essential Report which use large sample sizes of 1700 respondents plus.

However – we can devise a solution to that problem, and one that still gains the benefits from the reduction in sampling error that a pooled supersample like Pollytrack brings to the table.

Over at the top of The Polls page, we measure Pollytrack not on a weekly basis, but on a real time basis – as a pollster releases a new poll, it replaces the value of that pollster’s previous poll in the Pollytrack series algorithm. If we make a time series out of these real time Pollytrack numbers, it has the fortuitous result of giving us the key benefit of Pollytrack – the large reduction in polling variation – but also expanding the number of observations we have from the 52 that we get when we measure it weekly, to as many observations as there are polls themselves.

If we look at how the real time Pollytrack stood up since the end of January and compare it to the actual polling results of all the pollsters – the ability of Pollytrack to reduce polling variation stands out. Here, we’ll look at the two party preferred margin.

This lets our Pollytrack series move on a daily basis (or any timeframe we choose) – as any number of new polls come in, we can simply spit the new Pollytrack numbers out – so we’ve solved the first problem of Pollytrack not adapting to election campaigns.

The second and third problems can now be solved by running a fairly aggressive locally weighted polynomial regression through the real time Pollytrack numbers.

Effectively, this allows us to add a trend line that not only accommodates a few outliers if they arise (our rogue poll problem), but also one which allows for a small variation in the Pollytrack results (the noise problem) by putting a localised line of best fit through it.

By reducing the actual polling results into a smaller variation pooled result, and then using those smaller variation results to be the basis for a regression trend that accommodates a small bit of noise and the occasional outlier, we end up with a chart that looks like this:

The dots shown there are the actual raw polling results with the red line being our twice cooked Pollytrack trend – our Pollytrend.

I’ll keep a shrunk down version of this chart on the top of the right hand sidebar that will be updated whenever The Polls page is (which is usually the day that a new poll is released).

Remember folks, the trend is what is important here rather than the level. We might not know the true level of public opinion at any given time – currently it’s extremely likely to be somewhere between 56 and 60 on a two party preferred basis to the ALP, but we can be extremely certain that the true level is higher today than it was in mid September and mid November.

In fact, we can be more certain about identifying movement in the real level of public support over time than we can be about identifying what the true level of public support actually is at any given time.

The trend is your friend – and now we have a trend to work with.