ExtraPOLLations

Colleagues and scholars from coast to coast, across Bass Strait and all the ships at sea.

Dateline: Australia, Federal Politics, March 2015.

"Bewitched, Bothered and Bewildered" is a wonderful song written by Richard Rodgers and Lorenz Hart from the 1940 musical "Pal Joey" which you can hear performed superbly by the fabulous Ella Fitzgerald on YouTube here: 

They are also three words that could easily describe the reaction of the political world to the recent poll results.

First, Prime Minister Abbott had one of his "worst weeks" and yet the IPSOS Poll showed an "improvement" in the Government's standing.

Then, Prime Minister Abbott had one of his "best weeks" (no doubt buoyed by the IPSOS Poll at the start of the week) and yet the Newspoll this week showed a "deterioration" in the Government's standing.

How could this be?

At the risk of making the grave mistake that Press Gallery legend Alan Ramsey warned of, "when you start quoting yourself, it's time to give it away" you might find these explanations of the idiosyncrasies of polling of some interest.

Sydney Morning Herald "Why that surge in popularity could be a load of balls" April 3, 2010, which followed a Newspoll that showed the then ALP Government had achieved a "poll surge" increasing its two party preferred vote from 52% to 56% in one fortnight.

Poll of Polls segment, ABC's Insiders program, February 2, 2013.

Both of these items are included below. 

"Why that surge in popularity could be a load of balls." Sydney Morning Herald, April 3, 2010, by Andrew Catsaras.

THE most recent Newspoll of federal voting intentions reported the two-party-preferred vote had changed by 4 percentage points since the previous poll.

The ALP had risen from 52 per cent to 56 per cent and the Coalition had fallen from 48 per cent to 44 per cent. This translates to more than half a million voters having changed their vote in the past fortnight. Is this realistic? What if the next Newspoll shows the two-party vote returning to ALP 52 per cent, Coalition 48 per cent? Would this mean over a half a million voters would have changed again? The short answer is: it's very unlikely.

This may be the one poll in 20, or 5 per cent, that is outside the pollsters' 95 per cent confidence range, termed a rogue poll, and so is unreliable.

However, if we assume this poll is not a rogue then the suggested change in vote may or may not be real. If we assume it's real, then the results indicate a trend towards the ALP with a magnitude of 4 percentage points - a very large and statistically unlikely figure. A trend towards the ALP is possible but the most likely explanation is that the change in voting intentions is as a result of sampling variation and so most probably represents an illusion.

Sampling variation is the unavoidable variation in results that occurs from poll to poll because we are polling a sample of the population and not the whole electorate. It is the most common explanation for changes in polling numbers, yet is the most misrepresented.

Consider the following example: imagine you have a bag of 100 balls, 50 black and 50 white. You randomly select 10 balls from the bag. For your sample to accurately reflect the 50/50 split between the white balls and the black balls, you should have pulled out five white balls and five black, but you would not be at all surprised if you pulled out only three white balls and seven black. You would put it down to a chance variation in the sample. You would not immediately assume that while the bag did originally have 50 white balls and 50 black balls, the make-up of balls in the bag had now changed to 30 white and 70 black to reflect the sample.

Equally, if after replacing the first sample your next sample pulled out seven white balls and only three black, you would not be surprised and would put it down to a chance variation. You would not believe that the make-up of the balls in the bag had changed once more to now be 70 white and 30 black to reflect the new sample and would understand that it is the samples of 10 that are varying, not the original 100.

In other words, you would not conclude that the make-up of the balls in the bag was ''volatile'', first ''swinging'' to the black balls and then back to the white balls. In political terms these sampling variations tend to be seen as a ''poll surge'' followed by a ''poll slump'', but are more likely to be a statistical illusion. In a poll it is possible that reported changes in voting intentions, compared with the last poll, are accurate reflections of shifts in the mood of the population, but it is more probable that it is just sampling variation.

This highlights the problems associated with poll-to-poll analysis and reinforces the need to examine polling data over time. With this Newspoll, the most sensible approach would be to wait for another few polls from at least two pollsters before drawing any definitive conclusions. However, this won't occur, because the political world, participants and observers alike, is obsessed with using polling as a real-time measure of the state of the political parties. Thus every movement, big or small, must mean something. Consumed by the 24-hour news cycle, the political world assumes the real world is equally consumed and therefore poll movements must reflect the voters' response to the last fortnight's political events.

There is a Latin phrase, ''astra non mentiuntur sed astrologi bene mentiuntur de astris'', that translates as ''the stars never lie but the astrologers lie about the stars''.

Just as the astrological world is confident there is meaning in the movement of the stars, the political world is equally confident that there is meaning in the movement of the polls. As for those in the real world, they believe there is meaning only when it says what they want to hear.

Poll of Polls, ABC TV's Insiders, February 3, 2013.

BARRIE CASSIDY: OK well it is an election year, there'll be all sorts of polls week in and week out and all sorts of analysis to go with them. What should we take from all of that?

ANDREW CATSARAS: Well there is a great deal of misunderstanding about polls. An opinion poll is not some Delphic Oracle delivered to us from the gods, it's just an estimate of how people are likely to behave in a hypothetical situation: that being, if an election were held now.

And by aggregating the polls as we do on Poll of Polls, we're trying to get a more complete estimate, from a series of estimates, of how people are likely to behave in a hypothetical situation. And then we track those estimates over time to see if there are any trends emerging.

Now while such a measured approach might not suit the manic nature of the 24-hour news cycle with its insatiable desire for instant gratification and superficial analysis, it is the only sensible way that you can look at polls.

BARRIE CASSIDY: OK, so what about the analysis that you provide though, averaging out the polls every month; how much weight should we put on that process?

ANDREW CATSARAS: Because we're aggregating the polls it is a more comprehensive approach, but even these results are not answers and nor are they predictions. They're simply reflections. And because they're reflecting likely behaviour in a hypothetical situation, those reflections are not clear images of the future but rather blurred images of the past, and should be understood as such.

Now viewed in that context, we can better understand what polls actually are, not what many people would like them to be. And it exposes the extraordinary frenzy that follows the release of each new poll - treating each poll result as if it's an actual election result, and each change from poll to poll as if they're actual swings between elections - for the absurd and highly destructive political ritual that it truly is.

A ritual that regularly weaves the threads of very little into the illusion of very much; one that routinely values opinion over fact; and one that is convinced that random variations in polling results actually mean something, is a ritual, that our political system could well do without.