Guest Blogger John Santos on the Global Poll

Nation, it's funny how small a world this is when you think about it...

I received a Facebook "friend request" a few months ago, from a young man named John Santos. It turns out that his brother had attended a music school at which I had been a teacher. The message that John sent along with his "friend request" indicated that he had long been a reader of The Enlightened Savage, and now that he realized there was a personal connection, he'd be sure to read even more often.

Well, it turns out that John is quite the political animal, and when the Global/NRG poll came out this Thursday night, John made sure to point out on Twitter that, "Not that the overall picture would change, but there are methodological issues with the Global poll."

Now, many would disagree with me on this point, but I firmly believe that methodology MATTERS in these things... a badly-constructed poll, whether by circumstance or by purposeful design, leads one to believe that certain things are true, when in fact they're not - and donors, volunteers, and supporters pay attention to the numbers that these polls trumpet as STONE. COLD. FACT. A poll that's constructed badly can cost a candidate donations. Heck - it can cost them the ELECTION.

So, I asked John to email me about those problems. His response is below.


******

Some methodological commentary on the Global Calgary municipal election poll

With the first poll out by a polling company, I thought I would provide some commentary on how the numbers were reported. NRG is a reputable firm and would have taken proper steps to safeguard the more “scientific” aspects of polling (such as proper sample selection). However, that does not mean that there are not other methodological issues with this poll. With as wide of a gulf as what is observed, I must emphasize that the overall picture is probably accurate (i.e. that Higgins and McIver are the frontrunners), though I am doubtful if the gap is actually as wide as reported.

The numbers as reported by Global and NRG:


Name of mayoral candidate that comes to mind first

Barb Higgins 27%
Ric McIver 20%
Bob Hawkesworth 2%
All other candidates 1%
Unsure/none 39%

N=500; +/-4.4%, 19 times out of 20


Candidate committed to or leaning towards

Barb Higgins 44%
Ric McIver 40%
Bob Hawkesworth 3%
All other candidates 2% (each, or less)

N=500; +/-6.2%, 19 times out of 20


Which mayoral candidate do you intend to vote for?

Barb Higgins 16%
Ric McIver 15%
All other candidates 1% (each, or less)
Undecided 51%

N=500; +/-4.4%, 19 times out of 20


Timing

Higgins announces on Wednesday. A poll is done asking the “name of the mayoral candidate that comes to mind FIRST”. This one should be a no-brainer. There will be a tendency for committed voters to mention their candidate first, though those less committed could be inclined to mention whoever dominated the news in the last little while. I wonder who that person was.Moreover, polls in the summertime are somewhat dubious to begin with. Many people are on vacation, and many of those left in town are more worried about next weekend’s barbecue party than politics. Nenshi has been quite active over the summer, but his base is more interested in policy to begin with. McIver, I’ve heard little from, which is expected because his target group is made up of Average Joes. Unless someone really loves or really hates the other candidates, it is unlikely they would be mentioned. Higgins, because we see her on the news every night, would be a no-brainer to mention.


The shotgun marriage

Does anyone else find it strange that a plurality of Calgarians have already “committed” or are “leaning towards” Higgins despite only being in the race for less than 24 hours? Obviously “committed” and “leaning towards” are separate categories that have been combined, but the devil is in the details. A breakdown of 10%/34% committed/leaning is different from 40%/4% committed/leaning, but we don’t know how the numbers are added up.

Then, there’s the question of the validity of a person’s opinion. No, I’m not saying that some people’s opinions don’t matter; I’m saying that what some people think is an opinion is not actually a real opinion. By most accounts, Higgins’ announcement was less-than-stellar. It was also sparse on details other than she is both socially conscious and financially conservative. In other words, she is like every single politician. I don’t see Liberals running on “I’m gonna rob you through taxes and piss it away” or Conservatives running on “screw poor people; we have money.” Some people take longer to buy a lamp than the time it took 44% of respondents to decide they were leaning towards a candidate.

Finally, we don’t know the question or questions asked. Was it one question: “which candidate are you committed to or leaning towards?” Was it two questions: “is there a candidate you have committed to?” (if no, then) “is there a candidate you’re leaning towards?” Were response categories provided or was it open-ended? That last one is especially important because this question would have come after the knowledge question. It isn’t a quantum leap to say you think you’re leaning towards someone because you thought of them first, especially when the alternatives are not currently on the top of your head. This holds as much validity as Phoebe’s (from Friends) game of “answer these questions as quickly as possible.”


Two birds of a feather

I’m at a loss why a “committed/leaning” question was asked AS WELL AS a specific vote intention question. The only question that is worth reporting coincidentally also happens to be the least exciting. The vote intention question is the most important because it links an idea to an action—“if an election were held today, who would you vote for?” It’s very possible to lean towards someone, but to stay home or even change your mind.

As with the previous question, we don’t know if respondents were prompted with candidate names or if the question was open-ended. Ivrnet’s poll published in Metro prompted a more evenly distributed range of responses.

Still, even if we did accept the vote decision numbers prima facie, what does it show? Take out the undecideds and Higgins pretty much consolidated the “not-McIver” vote.


Technical notes

Of the two action questions, the gap between Higgins and McIver is a statistical tie. (based on the margin of error of each question)

The table doesn’t mention the percentage of the voters supporting another candidate. The known percentages add up to 82%, which leaves 18% decided voters that are split up between the remaining 8 candidates. Since each candidate has, at most 1%, that adds another 10%, which brings the total up to 90%. Where’s the other 10% of voters to make the total an even 100%?


So what’s the real story?

The real story is the 39% of respondents could not name a mayoral candidate and 51% would not know who to vote for if the election were held today. This is despite a recent announcement of a high-profile candidate.

Don’t get me wrong—Higgins has been a great newscaster, but everyone’s waiting to see if she’s has the political chops and the right policies. McIver has some room to grow by knocking down other conservative candidates like Connelly, Lord and Burrows, but his potential is far more limited than Higgins, who will likely be to the left of McIver and will have better access to the progressive vote IF she has the right policies.

Still, the fact remains that this poll comes at a bad time and is poorly reported. I’m sure NRG made good on their end of the deal, so I place blame squarely on the shoulders of Global for releasing a bad poll to get ratings. They even released a teaser on their website and Twitter feed that morning saying “early results are surprising—tune into the news at six for the full story.” I’m sure the odd political hack must have cancelled a date or something similarly important to make sure they were in front of the TV to see the news.

Effectively, what this poll does is set the narrative of the race, and it does so by chopping down the support levels for every other candidate below what is actually true in order to get a more exciting, two-horse race. Thus, while some of the candidates have a lot of potential to grow once the mass public starts to pay attention in late August, it's questionable if they'll get to that point because no donor in their right frame of mind would give substantial sums of money to a candidate at 2%. In fact, given the lack of policy announcements on Higgins' part, it becomes quite easy for her to knock off candidates early on and appropriate the policies she likes. (which is a perfectly valid strategy) What saddens me is that we might not hear what some of those other candidates are trying to campaign for, which is a real loss for democracy, which is already at a low point in municipal politics.



(This has been cross-posted to CalgaryPolitics.com)