March 3, 2006

More On That Fox News Poll

Hatched by Dafydd

This joint is turning into Poll Central!

I need to say another word about that Fox News poll I blogged about yesterday; and the next post is a really big one, examining the bizarre Zogby poll in detail, question by question....

Sachi pointed this out to me: if you go to RealClearPolitics poll coverage and click on the Fox News Poll up top (or just click on the link in this sentence), you'll get the complete writeup of the poll.

Scroll to the last page of the pdf, where some of the demography is listed. Bottom section. What do we see?

For the poll before the current one, taken February 7-8, party affiliation of respondents:

Democrats: 40% (advantage +4 to the Democrats)
Republicans: 36%
Independent/
other/decline
to state: 24%

Presidential job approval: 44% approve, 47% disapprove

For the current poll, taken February 28 through March 1st, party affiliation of respondents:

Democrats: 43% (advantage +10 to the Democrats
Republicans: 33%
Independent/
other/decline
to state: 24%

Presidential job approval: 39% approve, 54% disapprove

Again! They did it again -- the Fox News pool of respondents went from a gap of four percent favoring the Democrats to a gap of ten percent favoring the democrats; is it really any wonder that the gap in the president's job approval also shot up?

Some, of course, must be attributable to an actual drop in the president's approval rating (mostly among Republicans). But mathematically, the lion's share of the change simply reflects the bad luck of the draw in respondents, since this poll does not weight for party affiliation. My quick-and-dirty guess is that Bush's rating has actually dropped 2-3 points -- but the rest is simply an artifact of the pool of respondents.

This reminds me of the Mystery of the Misadded Restaurant Checks. For three years, I carefully added up all my restaurant checks (this was before everything was computerized). Whenever there was a misaddition, I noted it down; in three years, I had about twenty-five or thirty checks that were misadded.

Only one out of the twenty-five was misadded in my favor.

Now, as a math guy, I know that according to the laws of probability, if these were purely random errors, they should be as likely to go in my favor as the restraurant's; it's like flipping a coin twenty-five times and getting tails only once... that is, about 1 out of 1.3 million.

I concluded that the misadding of restaurant checks was not a random function. By the same reasoning, it seems that virtually every poll taken lately overrepresents Democrats... which tells me that the selection of respondents in political polling is likewise not a random function. There is a reason why so many more Democrats show up in these polls than Republicans.

That reason needn't be deliberate fraud; it could be a factor that nobody has considered. Maybe Democrats are more willing to sit still for the interview, I can't say. But it is a point to consider.

Hatched by Dafydd on this day, March 3, 2006, at the time of 3:30 AM

Trackback Pings

TrackBack URL for this hissing: http://biglizards.net/mt3.36/earendiltrack.cgi/541

Listed below are links to weblogs that reference More On That Fox News Poll:

» A Second Look At That "Second Look" At Same-Sex Marriage from Big Lizards
Pew -- and everybody to the left of John McCain -- is waving the lavender shirt over the new Pew Research poll purportedly showing that opposition to "gay marriage" (they mean same-sex marriage) is crumbling. Interestingly, the real agenda was... [Read More]

Tracked on March 24, 2006 2:48 AM

Comments

The following hissed in response by: BigLeeH

You said: "That reason needn't be deliberate fraud; it could be a factor that nobody has considered. Maybe Democrats are more willing to sit still for the interview, I can't say. But it is a point to consider."

It could easily be true that those who hate the president are more willing to sit through a long survey, waiting for the questions that let them vent their spleen. But it is more likely to be sampling error which can be persistently tendentious without being intentional. In some parts of the country the party affiliation of people who are able to answer their phone when it rings has a strong corelation to time of day, time of year, and what's on TV when the survey operator calls.

The above hissed in response by: BigLeeH [TypeKey Profile Page] at March 3, 2006 6:59 AM

The following hissed in response by: Mr. Michael

Another reason could be that the pool of (self identified) Democrats and Republicans is a shifting pool... When the Republicans are doing well, more Americans will identify themselves as Republicans. When the Republicans are being slammed in the media and are behind in the PR SPIN machine, then more Americans will identify themselves as Democrats.

Therefore, a more useful pair of identifiers would be to ask respondendts would be to ask "In the last 5 years, have you identified yourself as a Republican or as a Democrat (or other)? In the last year, have you changed your opinion on that?"

I would LOVE to see a pair of polls done, one a 'push poll' against the current administration, and one a 'push poll' FOR the current administration. Use all of the tools available, question construction, question order, limited response options, etc. Perhaps add a THIRD poll that was constructed by professionals to see how they would do it to determine what they advance to their clients as 'valid data'. Do the polling on the same days, with the same pool sizes, with the same call methods. I think it would be instructive in the extreme.

What would it take to commission such a pair of polls? Do you think it would be fun to set such a thing up by the Blogs, paid for by their readers? Maybe bring in CQ or Hugh to increase the size of the donation base, and lessen my financial share.

Because I WOULD donate to such a poll... even to make it a tracking poll over time. What do you say, Dafyyd?

The above hissed in response by: Mr. Michael [TypeKey Profile Page] at March 3, 2006 10:17 AM

The following hissed in response by: Dafydd ab Hugh

Mr. Michael:

I don't know how much it would cost, but I'm pretty sure it would be a lot -- all polls cost a lot -- and you wouldn't get many people to donate. Those who might be interested in funding a push-poll certainly won't pay for one for the other side!

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at March 3, 2006 2:36 PM

The following hissed in response by: Dafydd ab Hugh

I thought of another possibility for why so many of these polls oversample Democrats.

It's entirely possible that Democratic registration outnumbers Republican registration nationwide. I can imagine that many young college or high-school students, registering for the first time, reflexively check Democrat for their political party.

This is probably the dodge that would be trotted out by the pollsters to explain the disparity. But this is a canard, of course... because the opinions of people who don't vote are irrelevant on the issue of presidential job approval.

Thus, what matters is not the total registration but the registration of those who actually vote -- which in the most recent several elections has been pretty much dead even. In other words, all polls on political questions should be restricted to likely voters, not registered voters... and then the Democrat/Republican responses should be cross-tabbed and weighted to match the average of the ratios of the last two or three elections.

If we had several elections in which Democrats turned out significantly more than Republicans, so that, say, the actual voting pool turned out to be 48% Democrat, 38% Republican, and 14% independent or other or decline to state, then Democratic responses would deserve to dominate subsequent polling. But until then, they do not; it gives a false picture of the effects of this "opinion."

If there is a special case -- a huge groundwell of new registrations, coupled with a surge in grass-roots activisim (things that can signal an upcoming, 1994-like seismic shift in the vote) -- such circumstances can be noted, and two different sets of numbers released: one with the normal weighting, one with a more speculative weighting (both clearly labeled).

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at March 3, 2006 3:12 PM

The following hissed in response by: YouGottaBeKidding

One thing that people from the South, who are not familiar with the history, may not be aware of is that a lot of older voters are registered Democrats, even if they wouldn't vote for a Democrat if you paid them to! My father and my significant other are two fine examples.

I don't remember what I registered as when I registered to vote (some thirty-few years ago), but it was probably Democrat -- because if you wanted to vote in a primary, you had to be a Democrat. There was no Republican primary because there pretty much weren't any Republicans.

LA has open primaries now so it doesn't matter what your party affiliation is, you can vote. I registered as an independent (aka "none of the above"), mainly to avoid phone calls and mail from either party....

Therefore, in places like LA (that's the state, not the city), the fact that someone is a registered Democrat doesn't mean that s/he actually votes Democrat, and that helps explain why southern states that appear to be more Democrat than Republican based on actual voter registration, vote red in national elections.

The above hissed in response by: YouGottaBeKidding [TypeKey Profile Page] at March 4, 2006 10:24 AM

The following hissed in response by: YouGottaBeKidding

Oops! I meant "One thing that people NOT from the South..."

The above hissed in response by: YouGottaBeKidding [TypeKey Profile Page] at March 4, 2006 3:28 PM

Post a comment

Thanks for hissing in, . Now you can slither in with a comment, o wise. (sign out)

(If you haven't hissed a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Hang loose; don't shed your skin!)


Remember me unto the end of days?


© 2005-2009 by Dafydd ab Hugh - All Rights Reserved