Ipsos poll hysteria explained

Fellow citizens,

Last week an opinion poll from Ipsos was published in the Fairfax press.

It seemed to 'show' a collapse in the ALP's vote and a surge in the Coalition's vote. The media ran wild with 'explanations' that 'the boats issue' had sunk the ALP again - just like in 2001 with Tampa - and that the Coalition was 'back in the game'.

 

Apart from the fact that Tampa is one of the biggest myths in our political polling history (the issue of September 11 2001 is totally airbrushed from media discussion about the causes for the result of the 2001 election) the Ipsos poll, heralded as a turning point, was just a statistical illusion as a result of sampling variation. That's all it was.

Prior to the release of the Ipsos poll last week, there was a poll released over the weekend by Galaxy in Queensland (conducted in the same week) which showed that there was an improvement in the ALP's position, not a deterioration.

That's Queensland, the state where boats is supposed to hit the hardest.

As soon as Ipsos was released, the Galaxy result was ignored by the press, because, I suspect, the Ipsos poll showed what the press expected to see.

There was no corroboration by any other pollster to support Ipsos, indeed, there was a total contradictory result from a more reliable pollster. Yet this didn't matter as Ipsos showed boats had hit hard - as the press 'knew' it would.

So the press had not only an unconfirmed report, it had a report that was disputed by a more reliable witness, and yet the press ran with the 'evidence' it wanted to run with.

Newspoll today - showing there is no collapse in the ALP vote and corroborating the Galaxy poll in Queensland - illustrates how unreliable that witness was, and how foolish the press had been.

None of this is new.

I have written about this for over 10 years.

This piece published in the SMH on April 3, 2010 is just one example (see below).

While I am fully aware that the definition of insanity is doing the same thing over and over and expecting different results, I am optimistically hoping that this time all those who comment on polls read the piece and absorb the lesson.

"Why that surge in popularity could be a load of balls"

THE most recent Newspoll of federal voting intentions reported the two-party-preferred vote had changed by 4 percentage points since the previous poll.

The ALP had risen from 52 per cent to 56 per cent and the Coalition had fallen from 48 per cent to 44 per cent. This translates to more than half a million voters having changed their vote in the past fortnight. Is this realistic? What if the next Newspoll shows the two-party vote returning to ALP 52 per cent, Coalition 48 per cent? Would this mean over a half a million voters would have changed again? The short answer is: it's very unlikely.

This may be the one poll in 20, or 5 per cent, that is outside the pollsters' 95 per cent confidence range, termed a rogue poll, and so is unreliable.

However, if we assume this poll is not a rogue then the suggested change in vote may or may not be real. If we assume it's real, then the results indicate a trend towards the ALP with a magnitude of 4 percentage points - a very large and statistically unlikely figure. A trend towards the ALP is possible but the most likely explanation is that the change in voting intentions is as a result of sampling variation and so most probably represents an illusion.

Sampling variation is the unavoidable variation in results that occurs from poll to poll because we are polling a sample of the population and not the whole electorate. It is the most common explanation for changes in polling numbers, yet is the most misrepresented.

Consider the following example: imagine you have a bag of 100 balls, 50 black and 50 white. You randomly select 10 balls from the bag. For your sample to accurately reflect the 50/50 split between the white balls and the black balls, you should have pulled out five white balls and five black, but you would not be at all surprised if you pulled out only three white balls and seven black. You would put it down to a chance variation in the sample. You would not immediately assume that while the bag did originally have 50 white balls and 50 black balls, the make-up of balls in the bag had now changed to 30 white and 70 black to reflect the sample.

Equally, if after replacing the first sample your next sample pulled out seven white balls and only three black, you would not be surprised and would put it down to a chance variation. You would not believe that the make-up of the balls in the bag had changed once more to now be 70 white and 30 black to reflect the new sample and would understand that it is the samples of 10 that are varying, not the original 100.

In other words, you would not conclude that the make-up of the balls in the bag was ''volatile'', first ''swinging'' to the black balls and then back to the white balls. In political terms these sampling variations tend to be seen as a ''poll surge'' followed by a ''poll slump'', but are more likely to be a statistical illusion. In a poll it is possible that reported changes in voting intentions, compared with the last poll, are accurate reflections of shifts in the mood of the population, but it is more probable that it is just sampling variation.

This highlights the problems associated with poll-to-poll analysis and reinforces the need to examine polling data over time. With this Newspoll, the most sensible approach would be to wait for another few polls from at least two pollsters before drawing any definitive conclusions. However, this won't occur, because the political world, participants and observers alike, is obsessed with using polling as a real-time measure of the state of the political parties. Thus every movement, big or small, must mean something. Consumed by the 24-hour news cycle, the political world assumes the real world is equally consumed and therefore poll movements must reflect the voters' response to the last fortnight's political events.

There is a Latin phrase, ''astra non mentiuntur sed astrologi bene mentiuntur de astris'', that translates as ''the stars never lie but the astrologers lie about the stars''.

Just as the astrological world is confident there is meaning in the movement of the stars, the political world is equally confident that there is meaning in the movement of the polls. As for those in the real world, they believe there is meaning only when it says what they want to hear.