Wednesday, October 15, 2008

Ranking The Pollsters

With the onslaught of polls this election, I thought it would be interesting to rank the various outfits to see how accurate they were in the final analysis. In fairness, Ipsos did their last polling October 10, Angus Reid and Strategic Counsel October 11, NANOS, Decima and EKOS on October 13. Here are the actual results, and the pollsters final predictions, ranked on accuracy, with the difference in brackets:

CPC 37.6%

Ang Reid 38% (.4)
Ekos 34.8% (2.8)
NANOS 34.2% (3.4)
Decima 34% (3.6)
Ipsos 34% (3.6)
Str Cou 33% (4.6)

Lib 26.2%

EKOS 26.4% (.2)
NANOS 26.7% (.5)
Decima 25% (1.2)
Ang Reid 28% (1.8)
Str Cou 28% (1.8)
Ipsos 29% (2.8)


NDP 18.2%

Ipsos 18% (.2)
Str Cou 18% (.2)
Ang Reid 19% (.8)
Decima 19% (.8)
EKOS 19.4% (1.2)
NANOS 21.4% (3.2)


Greens 6.8%

Ang Reid 6% (.8)
Ipsos 8% (1.2)
NANOS 8.2% (1.4)
Decima 9% (2.2)
EKOS 9.6% (2.8)
Str Cou 11% (4.2)

To gauge the most accurate overall, we simply add up all the total "off" numbers for all the parties. Surprisingly to some, Angus Reid, the dreaded "online" pollster comes out on top, by a wide margin:

Total:

Ang Reid (3.8)
EKOS (7)
Decima (7.8)
Ipsos (7.8)
NANOS (8.5)
Str Cou (10.8)

One, small caveat here, if we take NANOS final night of polling, he is actually the closest. However, it was a rolling poll, one night of polling is only a partial sample, but it does show that NANOS saw the late breaking vote:

NANOS:
CP 37.1% (.5)
Lib 26.7% (.5)
NDP 20.3% (2.1)
Greens 7.1% (.3)

Total- (3.4)

Overall, the pollsters did a decent job this election. It seems the Conservative total was the least accurate, with the exception of Angus Reid. At the very least, these results should end the "online" poll= crap argument that occurs whenever somebody mentions this outfit. Angus Reid was also the most accurate in many recent provincial elections, so let's give the methodology it's due, the results should be taken as seriously as other, more traditional findings.

10 comments:

Gazetteer said...

Thanks for doing this Steve, very useful excercise.

"It seems the Conservative total was the least accurate...."

Gotta wonder if the big US Market bounce on Monday followed the way big TSX on Tuesday morning first thing drove that Con number up.

____
Finally, I think it would be interesting to assess how different the pollsters are at the front end of these things, because here there they are held to much less account.....

.

MilitantLiberal said...

Nanos is still the man but I must give credit to Ipsos whom I always trash, they seemed to be pretty accurate this time. Your point about online polls rings true. I've always thought they were pretty useless but the numbers don't lie.

lance said...

I'll echo gazetteer here Steve, Cheers.

I'll also admit some giggle factor towards Mr. Tribe.

Cheers,
lance

Anonymous said...

Polls are bad for democracy.

seaninsaskatchewan said...

My interest was also peeked by the last Nanos poll that showed the Tory uptick (albeit on a very small sample size). Infact, I remember turning to my wife and saying "oh oh, that might be bad" at the time. I wonder if they decided to publish the rolling 4 days of results in their last poll because of that, as they hadn't in any of the previous daily polls.

Raging Ranter said...

Nanos' final night of polling caught the last-minute swing to the Tories much the same as they caught the last-minute swing to the Liberals in 2004.

Frankly Canadian said...

You truly have the absolute best analogy of polls and how they break down, great work and extremely informative.

Raging Ranter said...

seaninsaskatchewan, Nanos published the individual daily tallies of their last three days polling the night before the 2004 election as well. I can't remember if they did so for 2006 or not, but I know for sure they did in 2004.

Lucas said...

FYI - Angus Reid also published one last poll on the 13th with 37-27-20-7, which would give them a total differential of 3.4, just like the last-minute Nanos poll. BUT... in those last minute polls, Reid did beat Nanos on the Bloc vote by 0.3, so the edge has to go to Reid on this one.

Steve V said...

lucas

Thanks for that, I didn't see the last one on the website when I put this up. You're right.