October 28, 2008

My "Two Elections" Thesis in a Nuthouse

Hatched by Dafydd

Nothing could more perfectly illustrate my point -- that we have two completely different elections, depending on which pollster you ask -- than a pair of polls released today:

  • First, we have the Gallup tracking poll with its traditional test for likely voters, in which Barack H. Obama's lead over John S. McCain has shrunk to 2%... well within the margin of error (not even counting general biases in favor of Democrats, particularly with most of the poll conducted over the weekend).
  • And on the same day, covering nearly the same period, we have the Pew poll... which finds Obama's lead over McCain ticking up to fifteen points!

The poll by Pew Research would lead to Obama winning somewhere north of 400 electoral votes... essentially winning every single toss-up state, plus every state that is currently shown as leaning towards McCain (pale red) on the Real Clear Politics electoral map; that would give Obama 411 electoral votes to McCain's 127.

But the traditional Gallup poll would almost certainly result in McCain winning all of the toss-up states, plus several of the states currently shown as leaning towards Obama (pale blue) -- in particular, the Bush states of Virginia, Ohio, and Colorado, plus the conservative district of Maine; this would give McCain a 275 to 263 electoral victory over Obama. (If we headed into the election with Obama and McCain in a photo finish, McCain would probably add New Mexico and possibly Pennsylvania to his stack for a convincing 301 to 237 win.)

So one respected poll tells us it's going to be a watershed landslide for Obama, with McCain's haul being reduced to a small core in the middle of the country -- while another respected poll tell us that McCain is going to win by 12 electoral votes. Reconcile that, brother!

It is of course theoretically possible that the actual spread on election day will be right in between those two, with Obama winning the popular tally by 8.5% (and the election, of course). But my instinct tells me that it's more likely that one of these two scenarios is prophetic, while the other is flat-out wrong, based upon completely erroneous turnout predictions.

The only question is -- which is which? We'll have to hold our collective breath for one more sennight to find out, as Paul Harvey would say, the rest of the story.

Note: In another mesmerizing blogpost, WLS -- now calling himself WLS Shipwrecked (God knows as your dog knows) -- goes into more detail about his own thesis: Obama probably must win the popular tally by more than 5% in order to win the electoral-college vote, the only one that counts. I have used WLS's analysis in this post, in the paragraph discussing what an Obama "win" in the popular tally of a scant 2% would look like in the electoral college.

Hatched by Dafydd on this day, October 28, 2008, at the time of 6:48 PM

Trackback Pings

TrackBack URL for this hissing: http://biglizards.net/mt3.36/earendiltrack.cgi/3309

Comments

The following hissed in response by: Dick E

Dafydd-

But my instinct tells me that it's more likely that one of these two scenarios is prophetic, while the other is flat-out wrong, based upon completely erroneous turnout predictions.

We had a discussion on this topic a few days ago. (OK, I vented my spleen; you stood by bemusedly.)

I mentioned some methodological issues with polling procedures. Our friend (McCain style friend, at least for me) Iowahawk seems to agree with me -- see his post Balls and Urns.

I won’t blather on again about how, even if we believed public opinion polls were accurate, they don’t realy provide useful information to the public. (Unless, of course, the information is first “adjusted”, then filtered through the brilliant insight of some oracle -- maybe a reptile.)

But how can a process that is so obviously flawed procedurally and therefore innately inaccurate yield meaningful information?

I don’t know how you could quantify the margin of error issues IH discusses. But if you could, the precision interval would probably skyrocket to maybe ± 15 or 20%

I can see it now: “In the local dogcatcher race, the latest Smogby poll has Junius P. Dingphud leading Duard Farquhar by 2.3 percentage points. The poll has a margin of error of plus or minus 18 points.”

That would probably be more accurate than the figures now published. But precision that high renders the data essentially meaningless.

The above hissed in response by: Dick E [TypeKey Profile Page] at October 29, 2008 1:40 AM

The following hissed in response by: David M

The Thunder Run has linked to this post in the - Web Reconnaissance for 10/29/2008 A short recon of what’s out there that might draw your attention, updated throughout the day...so check back often.

The above hissed in response by: David M [TypeKey Profile Page] at October 29, 2008 8:24 AM

The following hissed in response by: Dafydd ab Hugh

Dick E:

Yes, I know about all those biases that Iowahawk mentions that can mess up the sampling. But the fact remains, and cannot be denied, that certain polls are consistently close to the actual vote.

Given that observation, clearly those confounding factors usually do not render those particular polls (Gallup, Rasmussen, Zogby telephone, Battleground, IBD) as useless as you and Iowahawk imagine. They do in fact give useful information... and if they didn't, campaigns would not consistently use them (and their own polling), husbanding scarce campaign money -- well, "scarce" except for Barack Obama's campaign -- for adverts and GOTV.

Still, though, other polls are consistently wrong, from election to election (the Star Tribune's poll, for example, or the Los Angeles Times' polling); and it's reasonable to ignore those. We must discriminate between polls that seem to get the balancing act right, and those which tumble off the wire darn near every time (and always to their left).

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at October 29, 2008 12:31 PM

The following hissed in response by: Xpressions

The polls will tell all on election night. All the other national polls and bias polls on in the media all favor the liberal illuminati. But light will be shined on the right-wing in 6 days!

The above hissed in response by: Xpressions [TypeKey Profile Page] at October 29, 2008 1:27 PM

The following hissed in response by: Dick E

Dafydd-

David M:

We share a first initial, but I don’t have a blog. ;-)

But the fact remains, and cannot be denied, that certain polls are consistently close to the actual vote.

According to your October 22 post, Zogby had Obama at +10, tied with one other poll for the biggest lead, and Battleground at +2 was just 1 point above the smallest Obama lead. You called these results “all over the place.” So which of the select 5 polls are we to believe?

Political campaigns have to use SOMETHING to decide how to allocate resources. Even if one has very little faith in the accuracy of polls, what other tools are available?

As to actually paying for polling, I guess I would have a bit more faith in a survey that was constructed specifically for my purposes and for which I was comfortable with all the assumptions, methodologies, adjustments, etc.

The above hissed in response by: Dick E [TypeKey Profile Page] at October 29, 2008 10:52 PM

The following hissed in response by: Dafydd ab Hugh

Dick E:

We share a first initial, but I don’t have a blog. ;-)

Oops, sorry; I looked up to see whom to respond to, but I didn't look high enough. I'll correct it.

According to your October 22 post, Zogby had Obama at +10, tied with one other poll for the biggest lead, and Battleground at +2 was just 1 point above the smallest Obama lead. You called these results "all over the place." So which of the select 5 polls are we to believe?

The ones I mentioned. That doesn't mean every release from every one of those select polls is accurate; a 95% confidence interval means that 5% of the time, that poll errs by more than the ±... even if conducted perfectly.

But the Zogby telephone tracking poll is still a worthwhile instrument (as opposed to their worthless online "interactive" version), as are the others I cited.

By contrast, I have never taken the Pew Research or Field polls seriously and only use them when there is nothing else available, and even then, with caveats. Same with the CBS poll, the CNN poll, and many others.

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at October 29, 2008 11:49 PM

The following hissed in response by: Dick E

Dafydd-

I see.

So this just happens to be the 5% of the time when 40% of the preferred polls are outliers.

(I'm no mathemetician, but wouldn't it actually be 5% X 5% or .25%?)

The above hissed in response by: Dick E [TypeKey Profile Page] at October 30, 2008 12:09 AM

The following hissed in response by: Dafydd ab Hugh

Dick E:

So this just happens to be the 5% of the time when 40% of the preferred polls are outliers.

No; it happened to have matched polls that used a different metric to define "likely voter." That doesn't make it worthless.

Nor are the other set of polls "outliers;" that's not what the word means. An outlier is a single poll that, likely because of an unrepresentative sample, has a result that is totally at odds with the rest of the polling.

The situation we're in now is that we have two competing sets of turnout assumptions, leading to two different tracks of polling: one where Obama leads by 14%-15%, the other where he leads by 2%-4%. Most probably, one of the two is correct, and the other is simply wrong -- it's assuming a turnout model that will "turn out" to be completely off the reality.

Every so often, even a good poll gets a bad sample set; if it's a tracking poll, that produces a slow rise to a suspicious peak, followed by a slow fall (or vice versa). Statistics tells us that even aside from any bias, that "bad sample set" happens about one out of every 20 times a poll is taken.

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at October 30, 2008 3:49 AM

The following hissed in response by: Dick E

Dafydd-

Then how do I decide whether to believe the polls where Obama leads by 14-15% or those where he leads by 2-4%?

Most probably, one of the two is correct, and the other is simply wrong…

If you can’t tell me TODAY which is more accurate, then what good is the data?

It matters little why the difference exists -- whether it’s turnout assumptions, unrepresentative samples or the phase of the moon -- except to the polling community itself. I suppose they might be able to look back after the election and decide which methodology was better, but polls are supposed to be useful (if they are at all) when they are published, not after the post-mortem analysis.

…it [the polling results] happened to have matched polls that used a different metric to define "likely voter." That doesn't make it worthless.

OK, so what is the value of these polls? (Again, I recognize that political campaigns need some kind of tool, however inadequate, to allocate resources. I am referring to the value to the rest of us.)

The above hissed in response by: Dick E [TypeKey Profile Page] at October 30, 2008 6:50 PM

The following hissed in response by: Dick E

Dafydd-

Please allow me to retract the last comment above. Knowledge has value in and of itself.

Sorry.

I still don’t like polls, though. I guess they have value in that they are news. But what other category of news is created just for its news value?

The above hissed in response by: Dick E [TypeKey Profile Page] at October 30, 2008 8:41 PM

The following hissed in response by: Dafydd ab Hugh

Dick E:

Then how do I decide whether to believe the polls where Obama leads by 14-15% or those where he leads by 2-4%?

Blind faith? We won't know until we know which was the correct set of turnout assumptions. But we can remember for next time; it all becomes part of that pollster's permanent record.

Every time a pollster makes an egregious mistake, anyone who remembers (or uses Google to look back) will be more skeptical of that pollster's results in the future. The reason I reject Pew and Field and the CBS and WaPo polls is not merely that they're associated with organizations that are left wing; the main reason is that they have been wrong so often in the past, and so substantially, that I no longer trust them.

I expect to still be sucking air in 2010, 2012, 2014, and 2016... and I want to know who had a very good handle on a unique turnout question, and contrariwise, who was a git.

Besides, as an upcoming post here will demonstrate, there are signs we can all look for during the electoral reporting that will tells us fairly quickly which model was accurate.

Dafydd

The above hissed in response by: Dafydd ab Hugh [TypeKey Profile Page] at October 30, 2008 9:14 PM

The following hissed in response by: Dick E

Dafydd-

The reason I reject Pew and Field and the CBS and WaPo polls is not merely that they're associated with organizations that are left wing; the main reason is that they have been wrong so often in the past, and so substantially, that I no longer trust them.

Fine, but what are the rest of us supposed to do? I mean the 99.999%+ who don’t assiduously track polls after the fact for accuracy. We don’t even know which poll(s) to place our blind faith in.

For that matter, how do you do it? The only poll that an election really verifies is an exit poll. Every prior poll is not only subject to the errors previously discussed, but the passage of time changes voters’ perceptions and opinions.

How, for example, does an election tell you how accurate a poll taken in June was? Just because the same poll taken in late October was close to the actual election results doesn’t necessarily mean it was accurate in prior iterations.

Then, how many reasonably accurate prior polls (as compared to actual election results) does it take before you decide a particular poll is trustworthy? I would think it should at least 2. (Maybe the pollsters got lucky once or twice. Did they change their assumptions or methodology since last time? How do you know?) And if we’re talking about presidential elections, wouldn’t the test cases also have to be national elections? How many polls have been around that long?

Finally, if public opinion polling were a scientifically or mathematically valid construct (as is statistical sampling when properly applied), then NONE OF THIS DISCUSSION WOULD BE NECESSARY. (Not that it’s really necessary anyway, but I think you know what I mean.)

The above hissed in response by: Dick E [TypeKey Profile Page] at October 31, 2008 9:54 PM

Post a comment

Thanks for hissing in, . Now you can slither in with a comment, o wise. (sign out)

(If you haven't hissed a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Hang loose; don't shed your skin!)


Remember me unto the end of days?


© 2005-2009 by Dafydd ab Hugh - All Rights Reserved