Reader's Guide to Polls

Já que estamos nisto, e via pollster.com, cheguei a um texto de Jack Rosenthal, publicado no New York Times. Tudo isto já foi escrito e repetido mil vezes, mas vale sempre a pena repetir. Transcrevo algumas passagens:

Precisely False vs. Approximately Right: A Reader’s Guide to Polls
By JACK ROSENTHAL
Published: August 27, 2006

"False Precision
Beware of decimal places. When a polling story presents data down to tenths of a percentage point, what the pollster almost always demonstrates is not precision but pretension."

"Sampling Error
For a typical election sample of 1,000, the error rate is plus or minus three percentage points for each candidate, meaning that a 50-50 race could actually differ by 53 to 47. But the three-point figure applies only to the entire sample. How many of those are likely voters? In the recent Connecticut primary, 40 percent of eligible Democrats voted. Even if a poll identified the likely voters perfectly, there still would be just 400 of them, and the error rate for that number would be plus or minus five points. This caution applies forcefully to conclusions about other subgroups. What could a typical survey tell about, say, college-age women? Out of a random sample of 1,000, a little more than half would be women and only about 70 would be of college age. That’s too small a subsample to support any but the most general findings."

"Questions
How questions are phrased can mean wide shifts, even with wholly neutral words. Men respond poorly, for instance, to questions asking if they are “worried” about something, so careful pollsters will ask if they are “concerned.” (...) The order of questions is another source of potential error. That’s illustrated by questions asked by the Pew Research Center. Andrew Kohut, its president, says: 'If you first ask people what they think about gay marriage, they are opposed. They vent. And if you then ask what they think about civil unions, a majority support that.'"

"Answers
People never wish to look uninformed and will often answer questions despite ignorance of the subject. (...) Respondents also want to appear to be good citizens. When the Times/CBS News Poll asks voters if they voted in the 2004 presidential election, 73 percent say yes. Shortly after the election, however, the Census Bureau reported that only 64 percent of the eligible voters actually voted. Jon Krosnick, an authority on polling and politics at Stanford, uses the term “satisficing” to describe behavior when a pollster calls. If people find the subject compelling, they become engaged. If not, they answer impatiently. Either way, says Kathy Frankovich, director of surveys for CBS News, 'people grab the first thing that comes to mind.'"

"Intensity
How strongly people feel about an issue may be the most important source of poll misunderstanding. In survey after survey, half the respondents favor stronger gun controls — but don’t care nearly as much as the 10 percent who want them relaxed."

"Public opinion is not precise, and in any case it is constantly churning. Measuring it cannot hope to be precise. What readers can hope for, whether in an individual poll, a consensus from several polls or from the polling profession generally, is the truth — approximately right."