The more we look, the more we don't see

Colleagues and scholars from coast to coast, across Bass Strait and all the ships at sea.

Dateline: Australia, Federal Politics, Polling, March 2017.

Have a look at these numbers: 54, 53, 52, 52, 55, 53, 53, 52, 55, 55.

These are the estimates of the ALP's two party preferred 'vote' over the last two months for the last ten polls from Newspoll, Essential and Ipsos.

They are also just data points.

If you were to take a simple average of all ten, it would equal 53.4%

What does that tell us? That the ALP has held a solid lead over the Coalition for this political year.

Analysing each poll and the variation between polls and then ascribing 'explanations' for the changes in polling numbers is a pointless exercise.

We saw this occur when there was a 'shift' in voting intentions between the last two Newspolls. A change from 55/45 to 52/48 was an improvement in the Government's position and must have been as a result of the announcement on the Snowy River Hydro scheme. Now we have Ipsos reporting the ALP vote at 55. What must have happened last week to result in that shift?

Nearly seven years ago I wrote the following piece for the Sydney Morning Herald illustrating why this is a pointless exercise (see below).

I include this here today with some reservation because I remember that legend of the Canberra Press Gallery, Alan Ramsey, (a man not afraid of filling his columns with large pieces of quoted material) lamented that he knew it was time to go when he even started quoted himself!

Nevertheless, I shall proceed. Simply because I wouldn't change what I wrote and the point needs to keep being made.


Why that surge in popularity could be a load of balls

APRIL 3 2010

THE most recent Newspoll of federal voting intentions reported the two-party-preferred vote had changed by 4 percentage points since the previous poll.

The ALP had risen from 52 per cent to 56 per cent and the Coalition had fallen from 48 per cent to 44 per cent. This translates to more than half a million voters having changed their vote in the past fortnight. Is this realistic? What if the next Newspoll shows the two-party vote returning to ALP 52 per cent, Coalition 48 per cent? Would this mean over a half a million voters would have changed again? The short answer is: it's very unlikely.

This may be the one poll in 20, or 5 per cent, that is outside the pollsters' 95 per cent confidence range, termed a rogue poll, and so is unreliable.

However, if we assume this poll is not a rogue then the suggested change in vote may or may not be real. If we assume it's real, then the results indicate a trend towards the ALP with a magnitude of 4 percentage points - a very large and statistically unlikely figure. A trend towards the ALP is possible but the most likely explanation is that the change in voting intentions is as a result of sampling variation and so most probably represents an illusion.

Sampling variation is the unavoidable variation in results that occurs from poll to poll because we are polling a sample of the population and not the whole electorate. It is the most common explanation for changes in polling numbers, yet is the most misrepresented.

Consider the following example: imagine you have a bag of 100 balls, 50 black and 50 white. You randomly select 10 balls from the bag. For your sample to accurately reflect the 50/50 split between the white balls and the black balls, you should have pulled out five white balls and five black, but you would not be at all surprised if you pulled out only three white balls and seven black. You would put it down to a chance variation in the sample. You would not immediately assume that while the bag did originally have 50 white balls and 50 black balls, the make-up of balls in the bag had now changed to 30 white and 70 black to reflect the sample.

Equally, if after replacing the first sample your next sample pulled out seven white balls and only three black, you would not be surprised and would put it down to a chance variation. You would not believe that the make-up of the balls in the bag had changed once more to now be 70 white and 30 black to reflect the new sample and would understand that it is the samples of 10 that are varying, not the original 100.

In other words, you would not conclude that the make-up of the balls in the bag was ''volatile'', first ''swinging'' to the black balls and then back to the white balls. In political terms these sampling variations tend to be seen as a ''poll surge'' followed by a ''poll slump'', but are more likely to be a statistical illusion. In a poll it is possible that reported changes in voting intentions, compared with the last poll, are accurate reflections of shifts in the mood of the population, but it is more probable that it is just sampling variation.

This highlights the problems associated with poll-to-poll analysis and reinforces the need to examine polling data over time. With this Newspoll, the most sensible approach would be to wait for another few polls from at least two pollsters before drawing any definitive conclusions. However, this won't occur, because the political world, participants and observers alike, is obsessed with using polling as a real-time measure of the state of the political parties. Thus every movement, big or small, must mean something. Consumed by the 24-hour news cycle, the political world assumes the real world is equally consumed and therefore poll movements must reflect the voters' response to the last fortnight's political events.

There is a Latin phrase, ''astra non mentiuntur sed astrologi bene mentiuntur de astris'', that translates as ''the stars never lie but the astrologers lie about the stars''.

Just as the astrological world is confident there is meaning in the movement of the stars, the political world is equally confident that there is meaning in the movement of the polls. As for those in the real world, they believe there is meaning only when it says what they want to hear.