Why News Coverage of Polls is Even Worse than the Polls Themselves

Recently a major national news site contained the headline ‘polls show race still close in swing states’.  Of course they are close!  If they aren’t close then they wouldn’t be the swing states, would they?

The real bias in the media is a bias toward attention getting stories.  This manifests itself as a bias toward conflict over consensus, toward change over status quo and toward the explainable over the ambiguous.  If a particular media channel wants to reflect their audiences’ demographics it may make a difference as to whether they regard a particular story as alarming or encouraging certainly, but the way that the facts are otherwise analyzed follows a very standard template.  After all, there will be plenty of other polls to write stories about in a given week, which is part of the problem.

Some reporters/editiors appear to understand the concept of margin of error but mostly for the purposes of writing that a contest is ‘too close to call’.  When it comes to comparing between different polls or between one week’s running of a poll and the next, the concept seems to be completely lost.  One of the more probable reasons for a three point swing in a particular poll from one week to the next is margin of error.  But how often do you see a headline say ‘3 point gain in latest poll probably doesn’t mean anything’?

Nor do reporters/editors demonstrate that they understand confidence intervals.  It will occasionally get mentioned at the end of the story, but never assigned any importance.  Most polls calculate their margin of error at a 95% confidence factor, meaning that the results would be duplicated within the margin of error 95% of the time.  What about the other 5% of the time?  Well, it would be more than that.

But including national, statewide and regional polls from all the various agencies how many new polls are released each week now that the election is less than three months away?  More than 20?  Certainly.  So that means that in any given week there will likely be a poll reporting results completely outside of their margin of error.  And because that poll will be the one showing the most dramatic change, it will likely be the one getting the broadest attention.

Permanent link to this article: http://betweenthenumbers.net/2012/08/why-news-coverage-of-polls-is-even-worse-than-the-polls-themselves/

Leave a Reply

Your email address will not be published.