Slog

News & Arts

The Stranger Suggests

Critics' Best Bets
Music Arts & Food


Line Out

Music & the City
at Night

Tuesday, October 27, 2009

McGinn Campaign Questions Washington Poll

Posted by on Tue, Oct 27, 2009 at 10:32 AM

The campaign of Mike McGinn just gave me permission to post an off-the-record e-mail that they blasted out to reporters this morning, questioning a Washington Poll that has McGinn trailing Mallahan by eight points (and also shows McGinn's change of position on the deep-bore tunnel causing him to lose votes).

From the e-mail:

The Washington Poll violates the cardinal rule of polling and the media should be very very skeptical:

They do not report the sample size of their poll.

And it's actually worse than that. They list a statewide sample size of 724 on the poll. But that is very misleading. They do not report the sample size for the poll in the mayor's race.

The population of washington state is 6.5 million. The population of Seattle is 582,000. Seattle represents about 9% of the state population.

Thus, if the statewide sample is 724, a proportional sample in Seattle would be a sample size of 65. Yes, that's right...65!

The pollster makes allusions to "oversampling" King County and the City of Seattle, but NEVER DISCLOSES what the extent of the oversample is. This is highly irregular in the scientific world of polling.

Low sample size also probably explains why they were so far off in their pre-primary polling:

Nickels 23%
Mallahan 11%
McGinn 8%

It's true, the new poll—which was conducted over 11 days—reached 724 registered voters throughout the state, and while the pollsters say they "oversampled" for Seattle, they don't provide the actual size of their Seattle sample.

It's not clear that the size of the sample was 65 voters, as the McGinn campaign speculates. That would indeed be a ridiculously small sample, if true. But the point is: we don't know the actual size of the Seattle sample.

I've sent an e-mail to the University of Washington pollsters asking for the sample size.

UPDATE:

Matt Barreto, one of the pollsters, tells me via e-mail that the Seattle sample size in this poll was 400.

The Seattle oversample was an n of 400; and so was the King County oversample; The McGinn people are down in pretty much all polls, and I suspect they are just upset. As you probably know, our polls have pretty much predicted the outcome of every contest since we starting this effort in 2006. We have a team of public opinion scholars who are experts in polling and have PhDs, so we'll let the data speak for themselves.

The McGinn campaign, for its part, tells me that their internal tracking poll—which uses a Seattle sample of 1,000—has McGinn in a statistical dead heat with Mallahan.

Impossible to verify, of course, since it's an internal poll. But it's worth noting that the Seattle sample size the McGinn campaign is claiming to use is bigger than the sample size the Washington Poll used for the entire state.

 

Comments (16) RSS

Oldest First Unregistered On Registered On Add a comment
gloomy gus 1
Thanks for the update from the pollster. If McGinn's gonna draw that quick he better improve his aim.
Posted by gloomy gus on October 27, 2009 at 10:43 AM
Baconcat 2
Oversamples are not sample sizes. I suspect the actual sample in this case would be about 133, assuming Seattle has about 1/3rd the voting population of King County.
Posted by Baconcat on October 27, 2009 at 10:48 AM
3
How can they say that their polls have predicted the outcome of every race? If that were true, Nickels would still be in the race.

Also, perhaps they should learn how to spell Hutchison's name correctly.
Posted by Gidge on October 27, 2009 at 10:53 AM
4
McGinn -- if it's that good, um, release it brudda!
Posted by Lamplight/bushel basket etc. on October 27, 2009 at 10:59 AM
Will in Seattle 5
"reached" - translation: a lot of young people with cell phones didn't answer.
Posted by Will in Seattle http://www.facebook.com/WillSeattle on October 27, 2009 at 11:17 AM
6
A poll over 2 weeks? Yeah, explains how they predicted McGinn's 8% finish in the primary.
Posted by kurisu on October 27, 2009 at 11:20 AM
7
RE: "The McGinn campaign, for its part, tells me that their internal tracking poll—which uses a Seattle sample of 1,000—has McGinn in a statistical dead heat with Mallahan."

What this likely means is that their numbers show Mallahan is ahead but by an amount that's within the margin of error -- thus the "statistical dead heat"; i.e. not a REAL dead heat.

Or put another way, if McGinn was ahead in their poll, they would announce that, even if the margin was tiny.
Posted by Citizen R on October 27, 2009 at 11:39 AM
Original Andrew 8
Seems like when I took stats in college, we learned that a truly representative sample has to have at least a certain minimum number of participants (1051?) in order to produce a reliable, accurate result.

Anything less than that can be ignored because the results will be skewed.
Posted by Original Andrew on October 27, 2009 at 11:42 AM
9
Oversample means "in addition" to the regular sample... so the actual sample was the 65 or so used in the statewide survey plus 400 used as the oversample for a total of 465 or so.

Sample size is really overrated, though. Sample size helps with precision ("margin of error"), but what makes polling good is accuracy. Accuracy is determined by the sampling method. Both the Washington Poll and Survey USA did an awful job with accuracy in primary election polling. It doesn't matter if you poll 50 people or 5000 people; if your sampling method is bad you'll still get bad results.

Their method was taking phone numbers from registered voters and recording the people who answered and answered their questions. I'm sure it makes quite a difference that younger, more urban voters probably don't answer their phone as much from random numbers, use more cell phones, change their phone numbers more often, and don't take unsolicited phone calls. How much of a difference? Who knows. Considering how inaccurate Washington Poll and Survey USA were with the primary election, probably a huge difference.

In short, a lot of polling is dumb and you'd be just as accurate in many cases just taking a wild guess.
Posted by miss stats on October 27, 2009 at 11:43 AM
Will in Seattle 10
so, is this n a sex, age diverse pop? is it excluding people in Evans School (trend Mallahan) and The Daily (trend libertarian/neocon)?

.... didn't think so.

Come back when you learn basic statistical science.

At best you have a skewed t set.
Posted by Will in Seattle http://www.facebook.com/WillSeattle on October 27, 2009 at 11:46 AM
Original Andrew 11
@ 9,

That's exactly what I was thinking. People self-selected to opt-out of the poll by not answering the questions, thus invalidating the results (except for the political pissing contest component, of course).
Posted by Original Andrew on October 27, 2009 at 11:47 AM
Will in Seattle 12
Now a t size of 12 can be used to identify a basic question to follow up with a full set sample of 512 if you want. You don't need controls, but you do have to norm to voting patterns, and using polls-based numbers (where you physically went to polls) won't exactly correspond to mail-based balloting, but not much you can do about that.

Age and sex breakdowns, neighborhood (LD) pop sizes on a pop graph would answer the drift factor, but I'm guessing you're overrepresenting certain pop groups so it will not match actual general election voter turnouts. You also have to count on an overvote by the gay population (R-71).
Posted by Will in Seattle http://www.facebook.com/WillSeattle on October 27, 2009 at 11:50 AM
w7ngman 13
#8 there is no minimum sample size, per say. Consider a population size of 1 that only needs a sample size of 1 to have no margin of error. The number you're thinking of was probably just an appropriate sample size for some sample population. As #9 pointed out, the way you find your sample and the way you ask the questions can be a lot more important.
Posted by w7ngman http://userscripts.org/users/89370 on October 27, 2009 at 11:58 AM
Will in Seattle 14
(in the other thread I point out their PDF shows a sample size of 100 voters in Seattle, with no age/sex/econ data or LD distribution to match census, so we can't even answer a basic question, as the sample size lacks statistical power)
Posted by Will in Seattle http://www.facebook.com/WillSeattle on October 27, 2009 at 11:59 AM
passionate_jus 15
This poll does sound like bullshit. Even 400 is still a pretty small sample size and 11 days is an incredibly long polling period. Plus, their pre primary poll did turn out to be bull shit.

Did they call cell phones or just land lines?

I wouldn't put much into polls anyway.

With an election this close and with so many undecideds, whoever gets their supporters to vote wins.
Posted by passionate_jus on October 28, 2009 at 12:51 AM
16
Let's get something clear about the UW preprimary mayoral poll. This poll had Nickels leading with plurality in the low 20s%, with the remainder scattered among the other contestants. That seems like a reasonable reflection of where the race was at a couple of weeks out from election day. Most Seattle voters by then had decided that they didn't want to reelect Nickels, but they hadn't decided who they would choose among the largely unknown field. In the end, most of these voteres eventually gravitated to Malahan and McGinn.

Remember, folks, polls are just snapshot of where the electorate is at at the time it's taken, not a prediction of what the ultimate vote will be. The UW polsters seem to be a pretty capable bunch, and they have an excellent track record.
Posted by Mavis on October 28, 2009 at 11:38 PM

Add a comment

Advertisement
 

All contents © Index Newspapers, LLC
1535 11th Ave (Third Floor), Seattle, WA 98122
Contact Info | Privacy Policy | Terms of Use | Takedown Policy