Lying About Statistics
Surveys on National Policy Issues—Asking Bad Questions Gives you Bad Information
AMERICANS ARE EVENLY SPLIT ON “GUN CONTROL.”
| Quinnipiac University. June 24-30, 2014. N=1,446 registered voters nationwide. Margin of error ± 2.6. | ||||||
| “Do you support or oppose stricter gun control laws in the United States?” | ||||||
| Support | Oppose | Unsure | ||||
| % | % | % | ||||
| 6/24-30/14 | 50 | 47 | 3 | |||
| CNN/ORC Poll. Nov. 18-20, 2013. N=843 adults nationwide. Margin of error ± 3.5. | ||||||
| “Do you favor or oppose stricter gun control laws?” | ||||||
| Favor | Oppose | Unsure | ||||
| % | % | % | ||||
| ALL | 49 | 50 | 1 | |||
| Fox News Poll conducted by Anderson Robbins Research (D) and Shaw & Company Research (R). April 20-22, 2013. N=1,009 registered voters nationwide. Margin of error ± 3. | ||||||
| “Which is more important: protecting the constitutional right of citizens to own guns or protecting citizens from gun violence?” Options rotated | ||||||
| Protecting right to own guns |
Protecting citizens |
Unsure | ||||
| 4/20-22/13 | 53 | 42 | 5 | |||
| 1/15-17/13 | 51 | 40 | 9 | |||
So, Americans are evenly split on the issue of more gun control. A clear majority think that it is more important to protect the constitutional right to own guns than to protect citizens from gun violence. That is evidenced in the polls. But, these questions are the wrong questions. See what happens when you ask policy relevant questions rather than generic “philosophical” questions.
| ABC News/Washington Post Poll. May 16-19, 2013. N=1,001 adults nationwide. Margin of error ± 3.5. | ||||||
| “As you may know, the Senate recently rejected a proposal to require background checks on people buying guns at gun shows and online. Do you think the Senate did the right thing or the wrong thing in rejecting expanded background checks?” | ||||||
| Right thing |
Wrong thing |
Unsure | ||||
| 5/16-19/13 | 29 | 67 | 4 | |||
| Quinnipiac University. April 25-29, 2013. N=1,471 registered voters nationwide. Margin of error ± 2.6. | ||||||
| “Would you support or oppose a law requiring background checks on people buying guns at gun shows or online?” | ||||||
| Support | Oppose | Unsure | ||||
| 4/25-29/13 | 83 | 13 | 3 | |||
THE SAME HOLDS FOR PUBLIC OPINION ON THE NRA.
| CBS News/New York Times Poll. April 24-28, 2013. N=965 adults nationwide. Margin of error ± 3. | ||||||
| “Is your opinion of the National Rifle Association favorable, not favorable, undecided, or haven’t you heard enough about the National Rifle Association yet to have an opinion?” | ||||||
| Favorable | Not favorable |
Undecided | Haven’t heard enough |
Don’t know/ No answer |
||
| % | % | % | % | % | ||
| 4/24-28/13 | 34 | 26 | 16 | 21 | 3 | |
| 1/11-15/13 | 38 | 29 | 14 | 17 | 2 | |
Why not ask?
How does knowing that the NRA opposes background checks for people buying guns at gun show or online make you feel about the NRA—much more favorable, more favorable, less favorable, much less favorable, no change
Pew Research Center. May 1-5, 2013. N=1,504 adults nationwide. Margin of error ± 2.9.
| “Do you think the National Rifle Association has too much influence, too little influence or the right amount of influence over gun control laws in this country?” | ||||||
| Too much | Too little | Right amount |
Unsure/ Refused |
|||
| 5/1-5/13 | 39 | 18 | 35 | 9 | ||
Why not ask?
The NRA’s actions played a major role in the passage of concealed handgun legislation in Kansas that allows anyone to carry a concealed handgun without a license or any training. Would you say that means the too much, too little, just the right amount of influence, or are you unsure about the amount of influence the NRA has on legislation?
Sure, you conflate attitudes about the NRA and the Kansas legislation. But that is exactly what you want to do. You want to provide concrete examples of what an organization does, then ask about its influence.
AMERICANS ARE EVENLY SPLIT ON PRO-LIFE/PRO-CHOICE ISSUE
| Gallup Poll. May 2-7, 2013. N=1,535 adults nationwide. Margin of error ± 3. | ||||||
| “With respect to the abortion issue, would you consider yourself to be pro-choice or pro-life?” | ||||||
| Pro-choice | Pro-life | Neither/ Mixed (vol.) |
Don’t know what terms mean (vol.) |
Unsure | ||
| % | % | % | % | % | ||
| 5/2-7/13 | 45 | 48 | 3 | 3 | 2 | |
| 12/27-30/12 | 48 | 44 | 3 | 4 | 1 | |
| CNN/ORC Poll. Aug. 22-23, 2012. N=1,055 adults nationwide. Margin of error ± 3. | ||||||
| “Now I am going to read some specific situations under which an abortion might be considered. For each one, please say whether you think abortion should be legal in that situation, or illegal. . . .” | ||||||
| Legal | Illegal | Depends (vol.) |
Unsure | |||
| % | % | % | % | |||
| “When the woman’s life is endangered” | ||||||
| 8/22-23/12 | 88 | 9 | 1 | 2 | ||
| “When the woman’s physical health is endangered” | ||||||
| 8/22-23/12 | 83 | 12 | 2 | 2 | ||
| “When the pregnancy was caused by rape or incest” | ||||||
| ALL | 83 | 14 | 1 | 3 | ||
LESSONS LEARNED:
The general rule is that the broader the question becomes, the less meaningful you can consider the answer. Pollsters, of course, know this. They often follow up these philosophical or “hot-button” style questions with more nuanced queries. But, commentators, of whatever political persuasion, choose those results that support their position.
Remember the old book, Lying With Statistics? The important part of that title is the second word—With. Maybe a better title would have been—Lying About Statistics. The statistics are rarely wrong. It is what one says about what those statistics mean that is always the question.
Go to http://www.pollingreport.com to see polling results from a wide range of polls on a wide range of issues. The poll data reported here comes from that site.
theleftcornerdotcom 