Media and Polling: You’re Doing It Wrong

Delphi Logo (small)

 

Forum Research is one of the most recognized names in the world of polling and market research in Canada.  Few firms have the ability to pump out an Interactive Voice Response (IVR) poll with a decent sample size and get an A1 over the fold in less than 24 hours.  When I was at the Ontario Legislature, a number of my former colleagues would wince at even hearing the name Forum Research.  They weren’t the only ones, however; many believe that the frequency and tabloid-like nature of Forum Research’s publications actually hurts Canadian democracy and negatively affects the public’s view on public opinion polling at large.  Torontoist Magazine best illustrated this through their political cartoon found here.

Forum Research should not be seen as the villain in this story, however.  On the contrary, they are merely a product of their environment.  Polling in today’s economy is very difficult. There is a plethora of polling and market research firms all competing to hear from a population that is getting harder to reach every year.  Low response rates and significant decreases in the use of landlines are just a few of the realities that a pollster today has to deal with that their predecessors did not have to.

In order to generate business and distinguish themselves in the industry, some firms conduct a study or poll and simply give the results (or at least the top-line results) to a friendly media outlet at no charge.  Why does this happen?  Media outlets don’t have the money they once did to pay for public opinion polls.  Polling is expensive no matter how you slice it and declining sales and readership over the years have seriously affected how much media outlets are willing or able to pay for this kind of data.  By providing public opinion data for free (or at very minimal cost), polling firms have become marketers first and pollsters second.

Polling firms seem to promote their companies as a brand as opposed to promoting the information they produce.  When large or reputable media outlets publish opinion polls, the firm conducting the poll gets earned media and publicity, thus increasing their visibility and potential for future business.  However, when fewer outlets can afford polling data, pollsters and media outlets ‘race-to-the-bottom’ in an attempt to mutually promote each other’s products and services in the best way they can under the circumstances – and it’s the Canadian public that ends up losing.

I don’t think the public should point fingers at Forum Research for their business model.  It’s in the best interest of all pollsters, regardless of what firm they work for or what political party they are affiliated with, to be ‘correct’ or ‘accurate’ in their representation of what the public thinks.  What I think critics of Forum Research or polling in general should focus on, however, is the manner in which public opinion data is digested and regurgitated by the media.

The evolution in how the media covers politics and public affairs over the years has created a dependency on horse-race polling or the idea that there is always an election no matter the status of the legislature or how far out the election actually is. By focusing on the top line numbers of a poll and not the specifics within it, media outlets do the public an injustice. Intentional or not, this practice creates a less informed electorate even though outlets may actually believe that they are informing them.  This also leads me to believe that media outlets are more in the business of creating the news than reporting the news.

It is possible that the media has a real dependency on polling to sell papers or generate website traffic – and polling firms need to get that A1 over the fold to get noticed.  These two players are both in a bind, and we as Canadians should understand what this relationship means to the discussion and evolution of ideas and policy.

Many journalists do not know how to critically examine public opinion polls and even if they did have that knowledge, editorial staff do not believe that this knowledge is relevant to the narrative that the paper is trying to sell.  Journalists have frequent and tight deadlines, and when a press release crosses their desk with only so much as a basic explanation of what the numbers say or mean, the report of what the public actually thinks begins to suffer.

I’m not painting all journalists with the same brush – on the contrary, there are a number of former journalists in Canada who have gone on to become successful pollsters.  However, it is important when a journalist writes an article and references a poll to inform their readers why they were given this poll, if they had to pay for it, the methodology behind the poll – besides the margin of error, and why they chose to use this poll in their article.

Polling may not be a science like chemistry or physics, but there is a lot of methodological and statistical rigor that goes into polling.  Forum Research is just one of many trying to make a buck in a very tough and competitive environment.  The polling industry today is making a serious effort to ensure that firms and researchers disclose their methods as best as possible and to be transparent with their findings.  Media outlets should follow and move away from superfluous horse-race coverage simply to sell papers and switch their focus to explaining why a certain poll they chose to work with is valid or significant.  When media outlets increase their focus on the mechanics of polling the Canadian public stands a greater chance of being informed.

Adrian Macaulay is President of Delphi Polling & Consulting Inc. and blogs at Within The Margin.
Follow Adrian on twitter: @djsamplingerror

 

Click here for more political news headlines.

 

Share this article

Add your comments: