Slog: News & Arts

RSS icon Comments on Scientifically Analyzing an Unscientific Poll

1

"I am sure you have had the experience of making up your own mind on a question and then discovering, on the evening news of the same day, that only 23.6 percent of people agree with you. Ought you to be depressed or disconcerted by this alarmingly exact dissection of the collective brain? Only if you believe that a squadron of undertalented by overpaid pseudo-scientists have truly and verifiably arrived at this conclusion. And perhaps--indeed I would argue, in any case--not even then. [...] To the consumer the 'poll'... may seem like a mirror of existing opinion. But to the one who produces it, the poll is a swift photograph of the raw material to be worked upon."
-- Christopher Hitchens, Letters to A Young Contrarian

Posted by Katelyn | September 18, 2007 11:15 AM
2

"Consider this basic argument true for just about any poll you read in the lay press: statistically significant and utterly meaningless."

To be fair, your average Gallup poll is still more meaningful than this poll, as they are more likely to get a reflective sample of the population with their methods, and reweight their responses based on known demographic disparities in the sample. Good polling is possible; it's just hard to do.

Posted by tsm | September 18, 2007 11:20 AM
3

Dude, that was completely worthless. What you assume below is exactly what margin of error is trying to capture.

"let’s start with the reasonable assumption a slog poll is a decent sample of the slog-poll-responding population."

And then you compute the SD, which leaves no doubt that you're completely asea. Get thee to a stats class stat!

Posted by Bison | September 18, 2007 1:24 PM
4

"let’s start with the reasonable assumption a slog poll is a decent sample of the slog-poll-responding population."

A poll is supposed to be representative of a larger population. In a scientific poll, you sample in such a manner that you can extrapolate the results. The conclusion "this poll is representative of the people we polled" is useless.

Posted by Gabriel | September 18, 2007 2:02 PM
5

What #3 and #4 wrote -- beat me to it.

Posted by twee | September 18, 2007 2:44 PM
6

@4 - I think you could extrapolate it beyond Slog to include the unemployed and those that slack-off at work on the internet.

@3 - Thank you. Assuming you got a decent sampling is not reasonable.
Assuming the people who lost interest after the first poll were overwhelmingly "Gore voters" is actually much more reasonable.

There is no doubt that the August and September poll results are skewed by the fact that fewer people were interested in filling out a poll for a non-existent candidate twice. Therefore, the results were both statistically insignificant and utterly meaningless.

But, I still enjoy talking about it anyway, so nyaa! And, actually, there is a doubt, even though I say there is no doubt, so there is no doubt that poll results can be interpreted in a way that suits the interpreter/spin-doctor. Does that make sense? Good.

Posted by Mahtli69 | September 18, 2007 2:59 PM
7

An easier way is ask yourself:

1. Did the poll have clear questions with RANDOMLY selected population of at least 525 people? If phone, did they randomly dial or use phone books (miss cell phone users).

2. Even if random large sample, 19 out of 20 polls will be way off. Maybe by a bit - but maybe by a whole heck of a lot.

Posted by Will in Seattle | September 18, 2007 3:19 PM
8

This repeats #7's point, but it bears repeating.

"Let’s start with the reasonable assumption a slog poll is a decent sample of the slog-poll-responding population."

Sorry, but from a statistical point of view, this the Slog poll is NOT a decent sample of anything: it's not a RANDOM sample. Slog readers CHOOSE to participate in the poll, or not. All these online polls suffer from participation (self-selection) bias -- those that want to make their point vote (usually multiple times), those that don't have an axe to grind don't.

Margin of error is based on the strong assumptions of a truly random sample. While this makes sense for a Gallup poll where participants are selected (if imperfectly) randomly, it is totally meaningless for the Slog poll and all self-selected online polls.

Posted by David | September 19, 2007 9:58 AM
9

I don't want to be critical or pile on the complaints. But it might be helpful to ask whether or not one would estimate standard error, not standard deviation, with the formula one over the square root of n. I'm not sure what role standard error would play in any of this discussion -- I thought Eli wanted to know what the margin of error was.

That is, what is the random error? And the reason that the question itself is invalid is that a non-random poll can not have a random error. It almost certainly has systemic error, but there is no way to guess which way that systemic error skews the results.

Posted by elenchos | September 19, 2007 10:32 AM
10

Margarine of air or not, I think these polls are set up to take one's own pulse and thereby conclude that one's opinion shapes further opinion. Sew knot.

For all you Al-Gore-Rhythms: he's won an Oscar, now an Emmy. Why should he run for president when he has the clear oppportunity to join a very select group of nine who've won the Big Four? next a Grammy for "Al Gore Speaks" then a Tony for "Internezzo: the Musical". Gore then could become the fourth elected president to win a Nobel Peace Prize. Google aweigh, kids.

Posted by KY. COL. of TRUTH | September 19, 2007 12:16 PM

Comments Closed

In order to combat spam, we are no longer accepting comments on this post (or any post more than 45 days old).