Beware Of Poll Results

The Associated Press (AP) does its readers a disservice by suggesting that its most recent May 2011 survey results are indicative of future election outcomes.   AP makes significantly incorrect claims about likely election results because its poll (i) starts with a more Democrat-friendly profile, and (ii) is not based on likely voters.

The AP poll starts with a population having a different political profile than other nationally recognized pollsters including Gallup and Rasmussen.  One of the questions in poll asks people to identify themselves as Democrat, Republican, or Independent.  This is a common question asked by pollsters polling the general population.  Practically every nationally recognized pollster agrees that historically, more people identify themselves as Democrats than as Republicans.  However, in the AP poll, significantly more people identify themselves as Democrats than Republicans.  Both Gallup and Rasmussen’s historical polls show that Democrats have historically enjoyed only an approximate three to four percent advantage over Republicans.   In practically every month where data was available (since April 2009) , significantly more people identified themselves as Democrat than Republican in the AP polls than in the Rasmussen polls. In the most recent May 2011 AP poll, AP reported that Democrats have a spread of 17 percentage points over Republicans.   Although Rasmussen’s May 2011 data is not yet available, Rasmussen’s poll between January 2011 and April 2011 showed Democrats and Republicans have been approximately even. (Monthly data was not available for Gallup to make similar monthly comparisons.)

No poll that is based on surveying the American general population should be used as an indicator of upcoming election outcomes. Statistically valid random samples of the general population might be good indicators of the average American’s political identification; however, they are not good indicators of election results.  Practically every nationally recognized pollster acknowledges that a statistically valid random sample of likely voters (i.e., registered voters) is the best indicator of upcoming election results.   As is seen in results from each of the three pollsters, Democrats have an edge in the general population surveys. However, Republicans are more likely to participate in elections. Even the AP poll recognizes this. In one of its monthly surveys conducted in October 2010, AP conducted two different polls with the same party identification: one poll of likely voters and one poll of the general population.   More likely voters identify themselves as Republican (48%) than as Democrat (42%); in contrast, more of the general population identify themselves as Democrat (43%) than as Republican (39%). In short, because of the change in which group is larger, likely voters vs. the general population show a significant 10 point swing between the two populations.   See our more in depth recent article on this topic that includes graphs and charts here.

Polls are useful to gather data in a wide variety of situations.; however, their design and interpretation must be performed properly. Our Primer On Creating & Using Surveys provides guidance on what to consider.

Permanent link to this article:

Leave a Reply

Your email address will not be published.