Skip to main content

(From the diaries - I was pleased to see how many posters here recognized an instant classic when they saw one - DemFromCT)

We will all be avidly watching the exit polls on Tuesday night.  Some of us will simply be avid to know what they can tell us about who won.  Others will be avid to parse them for evidence of skulduggery.  This is an attempt to sort out fact from fiction, and help all of us understand what is going on.

First of all: there will be more than one exit poll exercise on Tuesday, and some of the smaller independent exit polls will be specifically designed to shed light on the integrity (or otherwise) of the vote-counting process.  But the big one will be the Edison-Mitofsky poll for the NEP (National Election Pool], so this diary is about that.

Purpose of the NEP exit poll

The NEP exit polls are designed primarily to answer these questions:

http://www.exit-poll.net/...

*  WHO voted for each candidate
*  WHY voters in your area made critical choices
*  WHERE geographical differences on candidates and issues were a factor

They are also designed to allow members to "call" state results (Senate and Governor in 2006), once it is unambiguously clear (with 99.5%) confidence who is ahead.  

Note that "verifying the integrity of the election" isn't one of the goals of the survey, and this is important, because, whether we wish it were otherwise or not, the exit pollsters assume that the vote count is correct.  

The poll questions are addressed by means of a substantial questionnaire completed by what is designed to be a representative sample of voters.  By cross-tabulating characteristics of the voters (age, race, sex, etc) with their answers to questions regarding their vote, and their reasons for their vote, a picture emerges as to who, where, voted for whom and what, and why.  This is extremely interesting information.

Getting a representative sample: sampling error

But the accuracy of the information depends on how truly representative the sample is.  And, unfortunately, it is remarkably difficult to get a truly representative sample of anything, let alone people.  Ideally, one would put every voter into an enormous barrel, shake the barrel, and get a designated toddler to pull out a large number of voters at random, and administer the questionnaire under duress, and with the aid of a truth drug.  If this was done, the only "error" in the poll would be "sampling error" - the variability you'd get if you repeated the exercise several times with several toddlers.  And from this variability, the "Margin of Error" (MoE) would be computed.  The MoE is the range of results you'd expect to get, say, 95%, or 99.5% of the time if your sample each time was completely random (the MoE will be wider for greater degrees of "confidence; e.g. the MoE for 99.5% confidence will be greater than the MoE for 95% confidence).

But, clearly, pollsters can't select that way.  They can't even randomly sample every voter, as that would involve being, potentially, everywhere.  So what they do is select a sample of precincts in such a way that each voter in a state has an equal chance of being selected, and then try to interview a similar sized sample of voters from each of the sampled precincts.  A small precinct will have a lower chance of selection than a large precinct, but IF your small precinct is selected, you yourself will have a higher chance of being selected than you would if you voted in a large precinct, so the net result is that everyone has an equal chance of being selected.  However, this form of sampling ("cluster sampling") means that there is less variability in the data than there would be if the sampling was truly random. Exit polls are, in effect, a large number of small polls, nested in states.  Each precinct in the sample is a mini-poll, with a small sample size (and a large MoE), and only a relatively small number (in the tens, not hundreds) of precincts are sampled in each state.  In order to give an estimate for the entire state, however (or for the nation, in the National precinct sample), the voters are considered as though randomly sampled (to give more statistical power), and the MoE is increased to compensate for the reduced observed variability.

Getting a representative sample: non-sampling error

Unfortunately, we can't stop there.  Not all "error" in polls is "sampling error" even after we've allowed for "cluster sampling".  There are many sources of "non-sampling" error, and these include factors that may systematically bias the polls.  And to make it worse, unlike the sampling error that is expressed in the "MoE", this error can't be quantified in advance.  Some sources of "non-sampling" error will tend to cancel out across precincts.  For example "measurement error" - simple mistakes - or "coverage error" - a group of voters missed because the interviewer was taking a break - may cause error that favors one candidate or party in one precinct, but are just as likely to result in an error in the opposite direction in another.  So while each precinct might have a "biased" result, the precinct "biases" should cancel out - sum to zero - over all.  However, other sources of non-sampling error may not do so.  If, for example, in a particular state, voters for one party tend to vote at crowded times, where voters are more likely to be "missed" (selected, but not interviewed because the interviewer is busy), then that party's voters may be systematically under-represented in the poll, across precincts.  This will result in a discrepancy between poll and count that may well be statistically "significant".

Non-response bias

More serious still, is "non-response bias".  Participation in the poll is entirely voluntary; selected voters are free to refuse if they do not want to participate.  A problem therefore arises if there is any tendency for one group of voters to be less willing to participate than the other group (and we can never rule this out: there is no reason to think that people's attitude to political choices is unrelated to their attitude to pollsters, particularly pollsters sponsored by the news media).  If this happens,  then the pollsters are, in effect sampling from "a different population" from the total population of voters - they are sampling from that subset of voters who are willing to participate in the polls.  And that subset may be more Democratic - or more Republican - than the total population.  

There are measures the pollsters can - and do - take to compensate for non-response bias.  Interviewers are asked to note the age, race and sex (by visual estimate) of all those who are selected for participation but who do not take part.  Data analysts can then compare the age, race and sex ratios in the non-respondents to those in the respondent sample, and if a particular demographic group appears under-represented, they know that some form of "non-response bias" has occurred - and can re-weight the data to compensate.  Indeed it is because of these kinds of data that we know a great deal about non-response bias.  Unfortunately, only non-response bias by visible characteristics can be observed.  The pollsters cannot know whether Republican or Democratic voters are over or under represented in their samples.   And "non-response bias" may be subtle - if those who are reluctant to participate manage to avoid actually being selected - then we will tend to get what is called "selection bias".

Selection bias

Interviewers are given an "interviewing interval" that is designed to net a consistent sample size across precincts, for example 100 voters.  So large precincts will be given a longer interviewing interval than small precincts.  For example, interviewers may be asked to interview every 10th voter in a large precinct, while those interviewing in a small precinct may be asked to interview every second, or even every, voter.  However, when the interviewing interval is large, and especially when a polling place is crowded, then strict "Nth voter" protocol may be more difficult to adhere to, and it may be easier for interviewers to find themselves unconsciously selecting the Nth +1, or Nth-1 voter if the Nth looks likely to refuse.  Indeed, it may also simply make it easier for unwilling voters to evade the selection process.   In the 2004 poll data, bias was significantly greater where N was large, and/or when the interviewer had to stand more than 25 feet from the polling place.  Note that where selection bias occurs, response rate may actually be enhanced - if you tend to select voters who are more likely to agree to participate, then your completion rate will go up.  Unfortunately, so may your bias.  

In short, therefore; exit polls are surveys, and they are subject to both sampling and non-sampling error, and the sources of non-sampling error include sources of bias.  In addition, and increasingly, the pollsters need to use telephone samples for absentee and early voters, and for all these samples - absentee, early, voters exiting the polls on election day - they need to make guesstimates about the likely turnout, based on past results, which may or may not be extrapolated correctly to the current election.

Compensating for non-sampling error

For these reasons, the pollsters have a number of data sources that they use to corroborate (or not) the results they get from the actual polls.  One of these sources is pre-election polls - if the exit poll responses diverge greatly from pre-election polls, then they have reason to regard their exit poll responses as potentially biased.  The pollsters became aware of such a divergence election day 2004 (as we know, from a leak by Wonkette), before a single result was available.  Another indicator is the vote-returns themselves.  Again, if the incoming vote returns indicate a systematic divergence from the exit poll response, the pollsters have reason to suspect bias in their sample.  For this reason, they dynamically reweight their estimates of the final result as first the precinct, and then the county tabulations are reported, and only when they are sufficiently confident that their estimate is correct (statistically confident that is) do they recommend "calling" a state for one candidate or another.  They also dynamically reweight their cross-tabulations from the same sources, to correct for what they assume is biased voter representation in their data. on edit: cross-tabulations are not reweighted dynamically; the first reweighting generally occurs two to three hours after state close of poll, and further reweighting is done thereafter as necessary.

So: if you want to know what the pollster's estimate of the results are, independently of reweighting by vote-returns, what you want are the cross-tabulations at close of poll in each state.  These were provided by CNN in 2004 (not "leaked" as widely regarded - simply posted as an early stab at the numbers) and there is no reason that I know of, despite rumors to the contrary, why close-of-poll cross-tabulations won't be posted again on Tuesday.  They may well change as the night wears on; this won't be because anyone is trying to "cover-up" a "leak" but simply because, rightly or wrongly, the official result is assumed to be correct, and any discrepancy between poll and count due to error in the poll.  

And now to dispel a few myths:

"'Uncorrected' poll results won't be released in 2006"

Joe Lenski, in an interview with Andrew Kohut from Pew, said that "We're going to put in place systems in which no one, even at the networks, can view any of this data before 5 p.m. on Election Day."  This should reduce the chance of information from incomplete samples being disseminated. An incomplete sample is highly likely to be biased, because voters do not arrive randomly.  If you miss a late Republican rush, or late Democratic rush, you will get the wrong picture.  Whether the networks will post close-of-poll cross-tabulations remains to be seen, but I have seen nothing to indicate that they won't.

"There is no evidence that Republicans are less likely to respond to exit polls"

There is plenty of evidence for pro-Democratic bias in exit polls.  The best kind of evidence is experimental evidence, where the experimenter actually controls a variable that is randomly allocated.  The random allocation ensures that it will be "orthogonal" to any other factor that might affect the phenomenon you are interested, in this case, discrepancy between poll and count.  In two experiments that I know of, methodological factors were manipulated in order to try to increase response rates (one involved giving free folders; another involved experimenting with shorter questionnaires).  In both cases, the one condition did result in different response rates, but also, surprisingly, to increased apparent Democratic bias.  As the difference in bias cannot have been due to fraud (there would be no way any fraudsters could have known which precincts had free folders) then we know that the manipulated condition must have been causal - that methodological factors differentially affected the participation rate of Democrats versus Republicans.  

"There is no evidence that Republicans were less likely to respond to pollsters in 2004"

There are strong correlations in the exit poll data between methodological factors (such as interviewing rate) and the magnitude of the precinct-level discrepancy.  Where factors were present that would have made it easier for unwilling voters to avoid being polled (or for eager voters to volunteer) the greater was the observed discrepancy.  

"'rBr' ['reluctant Bush responders'] is disproved by the fact that response rate in 2004 was higher in Republican states/precincts"

Because selection bias may result in high response rates, this "proof" is somewhat flawed from the outset.  However, more importantly, it is when Democratic response rates differ from Republican response rates that bias will occur, whether the response rates are 15% and 20% or 60% and 80%.   Overall response rates can tell us very little about bias.

"The 2004 exit polls indicate that many millions of votes were stolen"

There is absolutely no correlation between the magnitude of the precinct-level discrepancies in 2004 and change in Bush's vote share relative to 2000 (what UK commentators call "swing").  If a single factor, e.g. fraud, was responsible both for both the discrepancy, and for inflating Bush's vote, then you would expect the two to be positively correlated.  In fact, the correlation is slightly, but insignificantly, negative.  If fraud was responsible for the discrepancy in 2004 then either it was absolutely uniform (which is not the case normally made) or it was carefully targeted in precincts (not states) where Bush was expected do badly.  Either way, the data does not support the inference that the discrepancy was due to fraud; heroic assumptions need to be made to make it even consistent with fraud on a very large, nationwide scale.

"Exit polls are uncannily accurate"

The precinct level data has shown a consistent Democratic bias over the last 5 presidential elections, and the causes of the bias have been well-researched.  In 1992, the discrepancy was almost as large as in 2004.  The reputation for "uncanny accuracy" probably, ironically, derives from pollsters' extreme caution about calling states unless they are very sure they are right - and they make sure they are sure by incorporating vote-returns into the estimates in all but the most slam-dunk of races.  In the UK, where, for all our faults, we conduct pretty transparent elections, the exit polls are regarded as a bit of a joke (a cruel joke in 1992).  Peter Snow, the BBC poll presenter on election night has as his catch phrase "it's all a bit of fun".

"Exit polls are used to monitor election integrity around the world"

Not as far as I know. In Ukraine, there was direct evidence of blatant fraud (acid in ballot boxes; candidate poisoned with disfiguring, potentially lethal poison).  Sure, fraud will tend to play havoc with exit polls, but given that exit polls can play havoc with themselves, they can never be a primary instrument for monitoring election integrity.  Indeed, here are some cautions:

http://www.cartercenter.org/...

http://www.cartercenter.org/...

http://www.cartercenter.org/...

http://jimmycarter.org/...

Take home message:

The early exit polls will give you a reasonable idea of who is winning on election night, but there is no point in expecting the results to be within any calculated "Margin of Error", as MoE calculations assume random sampling error only, and do not reflect non-sampling error, which polls inevitably contain.  Therefore, even if the final results diverge "significantly" from the early poll, it won't necessarily mean there is fraud.  Exit polls have too many potential sources of bias for bias ever to be ruled out. If you want to find fraud, don't try and find it in the NEP exit polls.  Independent polls designed for the purpose may tell you more, but they are unlikely to be much more immune from bias than the NEP exit polls, and may be more vulnerable.

The Edison-Mitofsky FAQ is here:

http://www.exit-poll.net/...

Originally posted to Daily Kos on Sat Nov 04, 2006 at 11:48 AM PST.

EMAIL TO A FRIEND X
Your Email has been sent.
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
(The diary will be removed from the site and returned to your drafts for further editing.)
(The diary will be removed.)
Are you sure you want to save these changes to the published diary?

Comment Preferences

  •  I should also add (200+ / 0-)
    Recommended by:
    pontificator, Leslie in CA, DemFromCT, xerico, JekyllnHyde, Donna Z, DavidW in SF, ROGNM, Sean Robertson, cassandra m, DemUnity, Jackson L Haveck, teacherken, Tony, CalifSherry, Adam B, lipris, Bundy, Alan in Phoenix, Dump Terry McAuliffe, thebes, Rolfyboy6, Pandora, GregP, FaithAndReason, decafdyke, Mullibok, JimPortlandOR, jiacinto, vancookie, Dems2004, Tom Ball, leberquesgue, AAbshier, Blue Shark, martianchronic, rhubarb, Jackson, DCCyclone, Ugluks Flea, John Campanelli, NCrefugee, recentdemocrat, ilona, julatten, pollyusa, malc19ken, bumblebums, HL Mungo, exNYinTX, forkush, Jerome a Paris, sardonyx, kissfan, BartBoris, monkeybiz, rwsab, smintheus, worriedmom, Helena Handbag, Athenian, mentaldebris, Wee Mama, baba durag, The 1n Only Leoni, Welshman, srkp23, shipyardian, Pithy Cherub, stevej, SLJ, shanikka, javelina, dmsdbo, masoregonian, Torta, DiMe, MadEye, michael1104, matt2525, Terre, bewert, David Boyle, dmsilev, rocketito, ktakki, oldjohnbrown, Eddie in ME, RallyPoint, Chamonix, kathyfpr2, Nancy in LA, brainwave, cometman, HeedTheMessenger, hoolia, Hawksana, Catte Nappe, snakelass, hazzcon, renaissance grrrl, Eddie Haskell, 4jkb4ia, DMiller, JayBat, KateCrashes, Elwood Dowd, dnta, Marcus L Salyer, jcrit, d to the f, bablhous, Grahamdubya, weirdmusic657, azrefugee, Marc in KS, thereisnospoon, Spam Spam Bacon Spam, AUBoy2007, gradinski chai, Fabian, bloomer 101, red clay dem, Elise, HudsonValleyMark, ignorant bystander, PBen, ejmw, Cmyst, juliesie, Brooke In Seattle, devadatta, reflectionsv37, NeuvoLiberal, AnnArborBlue, Morrigan, lofti, jmonch, SheriffBart, hilltopper, Warren Terrer, fivefouranonymous, Marcus Junius Brutus, rofodem, Steve Singiser, sbdenmon, dsteffen, Jizzol, doe, Asinus Asinum Fricat, CCSDem, Denny in Seattle, Da na na na na na na na Batman, pico, evanaj, PatsBard, midvalley, Black Knight, The Angry Democrat, Malachite, Starseer, Truza, MTmofo, scubaix, NBBooks, nilocjin, OneCrankyDom, plf515, mhw, BarbaraB, vox humana, TeddySanFran, va dare, louise the dog, anais, frenchman, Abraham Running For Congress When I Turn 25, wildNwonderful, WWWD, Susan Something, possum, Mr Met, gloriana, BruceMcF, kath25, St Louis Woman, mefpdx, dconrad, sowinso, World Patriot, Demi Moaned, Big Tent Democrat, wakemeup7nov06, Themistokles, Wreck Smurfy, Pinecone, gateKrasher, mall teacher, Red no more, TheGardener

    my name is Elizabeth Liddle, and, as a result of diaries posted on DKos a while back, was contracted to reanalyse the exit poll data for Warren Mitofsky.

    Flame away, or alternatively tips freely, as I seem to have lost my TU status....

    •  to be honest I just skimmed the above..but (0+ / 0-)

      Could you please briefly (or not) comment on the oft repeated claim that exit polls have been bang on until Bush jr, are accurate in the rest of the free world, and are used to indicate fraud.

      Thanks in advance.

      •  see my post below with Cook's views (3+ / 0-)
        Recommended by:
        brainwave, AnonymousArmy, Fabian

        on that topic.

        "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

        by Greg Dworkin on Sat Nov 04, 2006 at 11:02:55 AM PST

        [ Parent ]

      •  Well, according to the data in the (16+ / 0-)

        reportissued by Edison-Mitofsky in January 2005, the mean state "within precinct error" (WPE), which is a fairly good measure of the discrepancy at precinct level (where it matters, as it is where fraud would show up) has had a statistically significant "redshift" (ie. count "redder" than poll) in every presidential election since 1988.  In 1992 it was particularly marked, but of course Clinton was so far ahead, I don't remember it being remarked on (although I seem to remember the "landslide" language being modified as the night wore on).  The thing about 2004 was that not only was the discrepancy large but the margin was narrow, so the leader position in the poll was actually reversed in the vote.

        As for the rest of the world: certainly in the UK they are not especially accurate, and we can rely on our vote count.  In 1992, the polls had a hung parliament (the pre-election polls had had a Labour victory), but the result was a small but perfectly workable majority of 23 seats for John Major's Tories.  I can certainly recall elections in other countries where the exit polls diverged from the count, and where the count was in question.  But in Italy in April, where the losing Berlusconi accused Prodi's team of fraud, the exit polls were actually in Prodi's favour.

        How accurate exit polls can be partly depends on the electoral system.  Some just poll bellwether, or swing, seats.  It partly depends on whether you are trying to predict the popular vote or the number of seats, and in any first past the post system, these two are not coterminous.

        Back to your question: there seems to be no correlation between poll accuracy and Bush candidacy.  Yes, Bush jnr actually lost 2000, but the exit poll error in 2000 was quite small. It was huge in 1992, but Bush snr lost anyway.  What is clear is that what has been "bang on" has simply been the projections - which is not surprising, as the vote-returns are incorporated in the projections.  There is no evidence that the actual polling data has been "bang on" and substantial evidence that it has not.

        •  Germany (0+ / 0-)

          One of the countries being cited for the accuracy of their exit polls is Germany. This is in part because unlike in the U.S., there are a bunch of independent exit polls being conducted in every national and state-wide election (I don't really know what the point of this is, but it sure feeds a lot of pollsters). Yet, if I remember correctly, all of the exit polls saw the conservatives winning the federal elections in 2002, and yet as the night wore on, Gerhard Schoeder's Social Democrats instead managed to eak out a narrow victory.

          Damn George Bush! Damn everyone that won't damn George Bush! Damn every one that won't put lights in his window and sit up all night damning George Bush!

          by brainwave on Sat Nov 04, 2006 at 12:04:54 PM PST

          [ Parent ]

          •  Yes (2+ / 0-)
            Recommended by:
            brainwave, HillaryGuy

            I remember that well. After the closing of the polls, the CDU/CSU saw the exit poll results, and declared victory. But, as the evening went on, it turned out that the Labour/Green coalition had just managed to retain their majority. It was fun to see the smirk slowly wiped off the the conservative faces :-)

            Here in the Netherlands, I also remember several instances where the exit polls did not match the eventual results, but my memory is more vague on those.

            I do also remember the 1992 UK debacle.. that was frustrating.

            I do not have my own blog.

            by Frank on Sat Nov 04, 2006 at 12:36:53 PM PST

            [ Parent ]

            •  It was painful (1992, UK) (0+ / 0-)

              I went to bed a little after midnight, still quite confident that Neil Kinnock would be reporting for work at 10 Downing ST within a matter of hours. When I got up, I was heartbroken to find out the polls had gone south in the wee hours of the morning.

              Damn George Bush! Damn everyone that won't damn George Bush! Damn every one that won't put lights in his window and sit up all night damning George Bush!

              by brainwave on Sat Nov 04, 2006 at 12:42:39 PM PST

              [ Parent ]

              •  Don't remind me.... (2+ / 0-)
                Recommended by:
                brainwave, AnonymousArmy

                Actually I stayed up all night, getting slowly drunk.

                November 2004 was a horrible deja vu.  And perhaps why I was never very convinced by the argument that the exit polls must have been right.

                •  It could have been the COUNT that was wrong (1+ / 0-)
                  Recommended by:
                  bdevil89
                  Your discussion presumes that the official count was accurate.

                  No good scientist would presume something that major.

                  Your analysis of 2004 must take into account 2 possibiliies:
                  either the exit polls were wrong OR the official count was corrupted.

                  You must examine both possibilities in order to have a solid analysis

                  Thus this sentence from your diary presumes that the "incoming vote" was accurate:
                  Again, if the incoming vote returns indicate a systematic divergence from the exit poll response, the pollsters have reason to suspect bias in their sample.

                  What if it was not?  

                  The fatal flaw in the [Evaluation of Edison-Mitofsky www.exit-poll.net/election-night/] is that they refused to even acknowledge the possiblity that the official count was corrupted - anywhere.

                  •  Well, in the UK (1+ / 0-)
                    Recommended by:
                    Demi Moaned

                    we can be fairly confident of our counting process because it is completely transparent.  So there was no suggestion from anyone in the UK in 1992 that the polls were right and the count wrong.

                    But to take your second point: indeed no good scientist who was investigating fraud as a possible explanation for the exit poll discrepancy would assume the count was correct.  Obviously.  It would be a nonsense.  And I didn't.  I did, as you say "examine both possibilities".

                    As for the pollsters - it is perfectly reasonable for them to suspect bias in their sample.  They know it happens.  But, contrary to your assertion, Edison-Mitofsky did NOT refuse "to even acknowledge the possibility that the official count was corrupted - anywhere".  That's why, if you recall, they compared the discrepancy in precincts that used different kinds of voting technology.  Later, Mitofsky himself contracted me to run further analyses, and again, I did not assume the count was correct.  I simply looked for correlates of the discrepancy, whether those correlates were methodological factors or factors likely to be associated with fraud.

                    And I'd like you to take back your assertion that my diary was anything to do with Edison-Mitofsky, please.  You had no evidence, it isn't true.  You did, in fact, merely "presume" it.

                    •  Satisfied that you mention you worked for Mitofsky (0+ / 0-)
                      That was the point I was trying to make, that you are or have been allied with Mitofsky, who was a premier pollster and I am sorry that he disavowed his 2004 results.

                      He did refuse to acknowledge the possibility that a corrupted count was a cause of the discrepancy, instead blaming it on the awkward and unproven reluctant Bush respondent theory, which is a theory not borne out by his own data.

                      He did also document the much higher accuracy in the paper-ballot districts, though there were extremely few of these alas.  I am only sorry that this finding was not explored further, at least by anyone that I know of.

                      •  Did you actually read my diary? (2+ / 0-)
                        Recommended by:
                        AnonymousArmy, Demi Moaned

                        Mitofsky did not "refuse to acknowledge the possiblity that a corrupted count was a cause of the discrepancy" - why do you think he tested the hypothesis?  And why do you think he contracted me to do more?

                        And the "theory" that Bush voters participated at a lower rate than Kerry voters was indeed borne out by his own data.  It was borne out by his own pre-election data on election day, and it was borne out by his (and my) analyses of the precinct level data.  It's in his report. Asserting otherwise is simply to mis-state facts.

                        As for the paper-ballot finding, he did explore it further, and subjected it to analysis by size-of-place which was important, as almost all paper ballot precincts were in rural districts.  But then I went even further, and found that when similar sized places were compared, there was no significant difference in discrepancy between paper ballot precincts and precincts with other technologies.  Interestingly, when I looked at large urban precincts serving communities of more than 50,000, where there were no paper ballot precincts, I did find a significant difference between technologies.  The discrepancy was significantly greater in precincts using older technology (lever; punchcards) than in precincts using digital technology (DREs; optical scanners).  So although it was significant, it doesn't actually support the inference of electronic vote-switching.

                        And BTW - if you were trying to make the point that I was "allied with Mitofsky" earlier, you made it singularly badly.  You implied that Edison-Mitofsky had "input" to my diary.  Well, Mitofsky is dead, and while I exchanged the emails with Joe Lenski after Mitofsky's death, and I have met him, my contract was not with his firm, but a private contract with Mitofsky.  And in any case - I explained my relationship with Mitofsky and the data in my first post.  

                        •  Controlling for technologies (0+ / 0-)

                          I'm very interested in ur analysis of divergence controlling for technologies.  Did ur analysis also control for supervisory factors.  I.e. if one was to postulate that one side (rebublicans in this case) could use computerized voting machines to switch votes, then it might only be reasonable to do in precints, counties or states where they controlled the vote counting aparatus.  (Now the question of whether they would need control of only the precinct, only the state or both would depend on what their actual method of vote switching was and would need to be tested separately...)
                          I remember scanning the results from Florida compared to exit polls and comparing divergence based on technology and seeing (to the untrained eye) what appeared to be correlation.  However, i can imagine that if you threw in results across the country, these discrepencies might be drowned out.
                          So my question is, was an analysis done that also took into account the party in control of the various vote counting methods combined with the technologies involved.  If so, what did u find?  if not, why not?

                          •  No, that data was not available (1+ / 0-)
                            Recommended by:
                            Dan D

                            but there would have been problems even if it had been. One thing that I think is not clear to a lot of people (understandably) is that the number of precincts polled in each state is very small.  So drilling down into within-state factors is just not likley to yield any statistically meaningful finding.  There is, on average, less than one precinct per county per state.

                            That's why I argue that actual vote return data is potentially much more informative - you just have to find a different baseline, one of which might be divergence from previous elections; another is divergence from party registration (I too had a look at that in Florida, and though at first there was something interesting, but in the end it was inconclusive).

                            So that's why my approach was to do analyses that made sense across the whole sample.  My approach was twofold: to find methodological correlates of discrepancy (and these were clear, and together actually accounted for all the net redshift) and secondly to determine whether the magnitude of the discrepancy was correlated with change in Bush's vote share, and it wasn't -not even slightly.

                            This last finding alone makes it very unlikely that fraud was a major contribution to the discrepancy; coupled with the finding that methodological factors were a major contribution, it seems clear that it is highly unlikely that any fraud effects would be found at lower statistical power, which any within-state analysis is bound to have.  Election Science Institute looked at precinct level discrepancies in Ohio, and while they found what I found (i.e. failed to find any correlation between change in Bush's vote-share and discrepancy) the statistical power was so low, that the confidence limits would have been fairly wide.  But by the same token, the statistical power of any analysis with only 49 data points is going to be extremely small.  

                            So the short answer is: you need a lot more polled precincts in any given state to do really meaningful analysis that might point to fraud.  And of course, all my other caveats about polls would still apply.

                      •  re the rBr data (2+ / 0-)
                        Recommended by:
                        AnonymousArmy, Demi Moaned

                        Well, this week, the Fox News/Opinion Dynamics poll asked a question that demonstrates Democrats' greater enthusiasm for exit polls:

                           Every election, the television networks conduct exit polls of people as they leave their polling places on Election Day. If you were asked to participate, how likely is it you would be willing to spend 10 minutes filling out a questionnaire? Scale: Very likely, somewhat likely, not very likely, not likely at all.

                        More Democrats (72%) than Republicans (66%) said they were likely to fill out an exit poll questionnaire. The gap was far bigger -- and highly statistically significant -- among those who felt strongly (typically a better predictor of actual behavior):  44% of Democrats said they would be "very likely" to participate in an exit poll compared to only 35% of Republicans.

                        link

                        "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

                        by Greg Dworkin on Sat Nov 04, 2006 at 06:22:59 PM PST

                        [ Parent ]

      •  And in case the comments become unwieldy (1+ / 0-)
        Recommended by:
        AnonymousArmy

        You'll find DemFromCT's comment here.

      •  For the alternate view, read the book by Freeman (1+ / 0-)
        Recommended by:
        bdevil89
        WAS THE 2004 PRESIDENTIAL ELECTION STOLEN? by Steven F. Freeman and Joel Bleifuss.

        This book is based on a close analysis of the Evaulation of Edison-Mitofsky, which was the report that Edison Mitofsky prepared in Nov and Dec 2004 and released to the media on Inauguration Eve in Jan 2005 (at a moment when the media would have soooo much time to deal with it).  It was 77 pages long and complex, and the media ran their reports on it the very next day, cribbing straight from the summary.

        Edison-Mitofsky had never done such a report before, it was their attempt to defend their polls - and get hired to poll again - when there was such a pronounced difference between the exit poll results and the official count.

        All is not as Febble says, this diary was written with the input and approval of Edison-Mitofsky and is their defense of their methods.

        They do not want you to question their exit polls . . .

        •  evidence? (3+ / 0-)

          I doubt that "Edison-Mitofsky" is even aware of the existence of this diary, much less has given it "input and approval."

          And who is it, anyway, who doesn't want the exit polls questioned? I mean, are you serious?

          If there's anything in the Freeman book that you found persuasive, maybe you can let us know so we can respond to it.

        •  Factually wrong (2+ / 0-)
          Recommended by:
          AnonymousArmy, Demi Moaned

          This diary was NOT "written with the input and approval of Edison-Mitofsky" and the fact that you can state it as a fact says a lot about the way you regard "facts".

          Facts are things that have evidence to support them.  You have absolutely no evidence for your assertion, and yet you make it.

          And by all means read the Freeman and Bleifuss book.  And if you would like to discuss the arguments Freeman makes here, I'd be delighted to explain to you why I think they are flawed.  You can tell me why you think they are not.  You never know, you might convince me.

          But you certainly won't convince me be stating as true things that I happen to know are false.

    •  Wow... (6+ / 0-)

      ...someone as well-reasoned and thorough as you without TU status?  What is dKos coming to?  

      But really - thanks for the diary.  I appreciate your cautionary message.  Even though this suggestion totally violates dKos policy about repetitive diaries, this might be due for a re-posting in the election aftermath, before the "fraud" meme becomes too prevalent (on either side).  Some rules were made to be broken, eh?  

      "You know I don't spend a lot of time thinking about myself, about why I do things." - GW, June 4, 2003

      by weirdmusic657 on Sat Nov 04, 2006 at 11:14:12 AM PST

      [ Parent ]

    •  Where have u been? (3+ / 0-)
      Recommended by:
      highacidity, Terre, Elise

          Kidnapped by Blair?   :D

      •  Trying to unseat him.... (10+ / 0-)

        but he's a hard guy to shift.

        •  What about early voting? (2+ / 0-)
          Recommended by:
          FaithAndReason, David Boyle

          In our county, in 2004, over 1/3 of all eligible voters voted early.  I would presume that this could introduce all sorts of bias to exit poll data, right?

          (Absentee ballots being likely to have essentially the same effect.)

          •  in 2004, there were phone polls in 13 states (1+ / 0-)
            Recommended by:
            David Boyle

            to pick up early and absentee voters -- but the samples were not very large, and it is hard to get the "mix" right.

          •  What it does is change the connection between ... (1+ / 0-)
            Recommended by:
            David Boyle

            ... what the exit polling measures and the election result. Any exit poll on election day will only allow projection of the vote on the day, and if a large number of absentee / early votes have been cast, then any discrepency between characteristics of absentee/early voters and election day voters will mean a discrepency between exit polling prediction and overall result.

            If early/absentee voting results are counted and released on the evening of the 7th, they will be able to correct for those effects at that time. If they are absentee ballots that are not all counted that day, a tight race may be an extended cliffhanger.

            Personally, I would not be surprised if in states where early voting is heavy, there is also some system either in place or being worked through to engage in exit polling through the early voting period.

            OH15: IN: Kilroy for Congress. OUT:Deborah Pryce

            by BruceMcF on Sat Nov 04, 2006 at 12:47:34 PM PST

            [ Parent ]

          •  Unless you can document it to the contrary (1+ / 0-)
            Recommended by:
            David Boyle
            Republicans and Democrats would vote in equal proportions both during early voting and on Election Day and at all times throughout Election Day.

            I for one don't buy the argument that Republicans and Democrats vote at different times of day, and I've never seen any evidence that this is so.

            I'd be glad to consider such evidence though if anyone has it.

            •  This depends on local conditions ... (0+ / 0-)

              ... and who the voters are.

              Indeed, its the maximum entropy estimate to assume that Republicans and Democrats vote in the same proportions through the day, if you have no evidence to the contrary.

              However, this diary is about the collection of evidence, and the features assumed into the design of your survey are the features that the survey cannot give information about.

              So, for example, the period around the end of first shift and the start of second shift is a relatively busy period in some precincts, but whether those voters normally tend to vote Democratic or Republican is not going to be uniform.

              And of course, changes in the rules can make old patterns obsolete. There has been a long-standing trend in Ohio for absentee ballots to trend more Republican than votes cast on election day, but since in Ohio you no longer need to give a reason that you will be absent on election day, it would be foolhardy to predict that the previous trend will continue.

              OH15: IN: Kilroy for Congress. OUT:Deborah Pryce

              by BruceMcF on Sat Nov 04, 2006 at 05:37:25 PM PST

              [ Parent ]

    •  you surely have your TU back (3+ / 0-)
      Recommended by:
      Rolfyboy6, decafdyke, Elise

      absolutely great diary.  you have a ton of grateful fellow Kossacks today.

      many thanks.

      George Allen: racist bully; you know what to do

      by jmonch on Sat Nov 04, 2006 at 12:03:25 PM PST

      [ Parent ]

    •  There is no reason to flame you (0+ / 0-)

      Analysis that is this strong is what needs to be recommended more. I can't speak to everything you say, but I understood it enough to know that whether one agrees or disagrees this is to be recommended- not put down.

    •  Haven't "seen" you in eons Febble (1+ / 0-)
      Recommended by:
      highacidity

      I always looked forward to your explanations of polling data, especially after 2004.

      Welcome Back! and thanks for this primer.  :o)

    •  Febble! Welcome back! (1+ / 0-)
      Recommended by:
      highacidity

      I've missed your clear-headed analysis. Nice to have it back right when it's needed. :-)

    •  Beautifully written diary on an important topic! (0+ / 0-)

      It's not often that I read a long diary here where the content justifies the length.

      And your points were all clearly made and well-structured overall.

      Thanks.

  •  Thanks, very informative diary. (5+ / 0-)

    Thanks for the American dream, To vulgarize and to falsify until the bare lies shine through. -william s. burroughs-

    by TheGardener on Sat Nov 04, 2006 at 10:54:57 AM PST

  •  couldn't possibly be more useful n/t (6+ / 0-)

    Chuck Schumer is not the enemy

    by AnnArborBlue on Sat Nov 04, 2006 at 10:56:25 AM PST

    •  This belongs in every campaign manual n/t (0+ / 0-)
    •  Need election verification exit poll instructions (2+ / 0-)
      Recommended by:
      patginsd, Flinch
      Election verification exit polls are different from media exit polls.

      This diary is about media exit polls.

      Here are instructions on how an election verification exit poll should be done.  They are prepared by a longtime exit pollster, Professor Kenneth F. Warren of St, Louis University.

      There are important differences, including how voters are sampled (election verification:  as many as possible, not every nth)

      •  "As many as possible" (0+ / 0-)

        is a very poor principle, but I don't think it's what Warren is suggesting.  I think he suggests counting 10 seconds rather than every Nth.  I think there are good reasons for that being a bad idea too, but it's certainly better than "as many as possible" which is a recipe for a volunteer sample that is heavily weighted by enthusiasts.  It's my big objection to "parallel elections" (which work on the "as many as possible" principle).

        •  You are muddling the issue (0+ / 0-)
          An election verification exit poll is NOT a parallel election.

          I don't think parallel elections are worth the time, I think they are a terrible idea.

          Pollsters approach voters and ask them to fill out an anonymous questionnaire.  They do not wait for pollsters to volunteer to be polled.

          Counting 10 seconds is pretty much getting as many as possible, it allows the pollster to stay organized and on top of the process while operating random selection.

          What Warren advocates is interviewing as many voters as is reasonable. as he says "Random selection is achieved by following a protocol that minimizes selection bias and does not break down." He says the nth voter model is good in principle but breaks down too easily in practice, when voters refuse or come out in clumps.

          •  Me? Muddling? (0+ / 0-)

            I didn't say it was.  But I did say that "as many as possible" was a bad principle for an exit poll, and that it was the same principle as for a parallel election, with the same downside.

            Counting 10 seconds is a bad idea because it means that your coverage will be better during slack periods than busy periods.  Fluctuating coverage is a risk factor for bias, because if one group of voters tends to vote at a busy time, that group will be undersampled.

            And I know what Warren says.  And I agree that the Nth voter model breaks down in practice.  It was part of the point of my diary.  I disagree that every 10 seconds is better.  I think it is highly likely to be worse.  I'd recommend every Nth, where the N is not greater than 5, and there are enough interviewers to ensure that truly every Nth is approached.

            •  Point of election verification (0+ / 0-)
              is to poll as many voters as possible.  That is the methodology Warren is using.
              •  Well, it's a bad principle (0+ / 0-)

                and won't produce a random sample.  Fortunately it's not the methodology Ken Warren is advocating.  Nonetheless I think his 10 second protocol is no more workable than Nth voter, and more likely to be prone to bias.  If he wants a large sample, he could go for every 2nd or third voter. Or every voter if he must, but he'd need a big team of interviewers.

                •  I disagree, busier times accounted for (0+ / 0-)
                  I think while neither is perfect, the 10 second interval is less likely to break down.

                  The Election Integrity group's election verification poll is going to track time periods when there are more voters, they are using a time-stamp, so they can adjust for this and examine whether results differ during heavy time periods.

                  That is a LOT more than then NEP poll has done or ever will do, as their data is withheld from the public, and can only be analyzed by them or by people they pay to do so.

                  •  Sure, the Election Integrity group's (0+ / 0-)

                    poll will do more for election integrity than the NEP because that is what it is designed to do.

                    But I'd love to know how they are going to monitor their results with a time stamp.  Can you explain?  

  •  2 comments by Charlie Cook on exit polls (11+ / 0-)

    when he stopped by last week, and note his views on using the exit polls to call races.

    one

    two

    Excellent diary from someone extremely educated on the topic. The "other' exit poll done by the new non-network consortium? Well, let's see. It's a difficult task, as laid out here. let's wish them well.

    "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

    by Greg Dworkin on Sat Nov 04, 2006 at 11:02:27 AM PST

    •  Charlie is one of the coolest guys (1+ / 0-)
      Recommended by:
      Elise

      in Washington.

    •  Hey Dem (6+ / 0-)

      Maybe slip this on the front page in a quiet moment? What do you think?

      •  I was reformatting in my head (5+ / 0-)

        from the time it was posted.  ;-)

        But, if it get recommended, it will actually last longer.

        "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

        by Greg Dworkin on Sat Nov 04, 2006 at 11:38:23 AM PST

        [ Parent ]

      •  I think so too (6+ / 0-)

        a solid antidote to conspiracy theorizing. And that's not to say people should accept black-box voting.

        •  Might as (2+ / 0-)

          well call it the Daily Kool-Aid.

          I am a physician, and when a person comes in with a problem, I ask and look for all the evidence  (symptoms, history, physical) as well as the context of the problem.  I then formulate what could be the possible problems based on the evidence and the likelihood (epidemiology, evidenced based medicine).

          The correlating analogy is that a voting technology that has been counting our votes for the past 6 years or so (i.e. central tabulators, now more local e-voting machines) is done by software that is proprietary (i.e. privately owned, or private property).  We must place trust in a machine owned by corporations and pretty much certified by the manufacturers themselves.  WE have no way to verify the vote except for trusting a black box that it is registering and counting our votes correctly.  Again, where is the basis of the trust coming from?  From the corporations of course.  

          Now the exit polls (not necessarily opinion polls) have had a history of being a sign of whether the official vote count is being counted correctly or not in many parts of the world.  Of course, they have been very off in the US in recent years thanks in part to?  (rBr, False recall, etc.)  Now is it coincidence that the exit polls have been so far off since the Bush administration has come into office, perhaps, or is there more evidence to point that this administration is up to committing more serious and unscrupulous actions in order to maintatin power.

          I could list the numerous scandals, disastrous actions, and outright lies this administration has stated, but I don't want to depress people.  That being said, are they above rigging elections? is that a line they wouldn't cross, or have they  shown time and time again, that there is no bar too low to stoop under in order to stay in power.  And more, importantly is there an avenue to allow that malfeasance to occur?

          Let's see:
          Exit polls widely discrepant from computerized vote count      Check

          Numerous and unprecedented historical precedents and odds that Bush overcame to win the election, baffling many pundits and poli-scientists:  Check

          Vote counting done in secret by partisan corporate software and without a hard paper ballot to verify:  Check

          Massive Retail vote supression done by a Repub party that touts democracy as essential for peace in the world (esp. in battleground states):  Check

          Amazing Repub GOTV (again, unprecedented, but verified by the computerized vote count) that won the election for Bush in spite of all the negatives against Bush:     Check

          Mitkofsky and Edison not releasing the raw data for statisticians and congressmen to analyze and give validity to the election bc it's private property:  Check

          Numerous anecdotes of vote count manipulation by partisan election boards (e.g. Warren County in Ohio), fraudulent recounts, Overvotes (more votes than registered voters), Undervotes (thousands of votes for a local judge, but none for the President), Vote flipping on touch screens all biased toward the Repub candidate, Co-campaign chairs in charge of state elections, etc. :    Check, in fact, double check.

          There's more evidence, but as you can see the evidence gathered (inlc. exit polls) point to some rather grave and disturbing conclusions, or as we say in medicine, a differential or impression.

          Conspiracy theory?  I think not, you would really have to blind, not to see the outright fraud in light of the context (corrupt administration, computerized vote counting) and all the evidence gathered that has been perpertrated on the American people.  This is not tin-foil hat territory, this is reality folks,  Wake up.  This administration is not above doing anything to stay in power, to dismiss or trivialize such things, is a disservice to this country and the world.

          The antidote we need is handmarked paper ballots, counted by hand, in front of all parties involved, and non-partisan independent exit polling for all major and close races,  a huge dose of a wake up call, and a return of the vote counting and the government to the American people.

          That being said, Vote and GOTV, but be prepared to march to DC on the 8th to rectify the fraud that will happen (I hope not, but based on the history and the evidence gathered from this administration it's clear what has been and will happen again on Election Day).

          •  I think Republicans would rig the election if.... (1+ / 0-)
            Recommended by:
            highacidity

            they could. And Republicans think Democrats would rig the election if they could. That's the basic reason why we need to switch to the most verifiable and secure form of casting and counting votes.

            To me, that means paper ballots.

          •  hey, doc (1+ / 0-)
            Recommended by:
            matt2525

            nothing worse than an expert out of their field. That's known as Argumentum Ad Verecundiam, and is all too common. It's one of the reasons the exit poll stuff was questioned early on because many writers and internet posters were out of their field. So now that we have a level playing field, let's talk.

            That being said, you won't find many here arguing that the GOP wouldn't want to steal an election. but check RonK's point here, so we're back on terra firma. It's not so easy to do, actually.

            "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

            by Greg Dworkin on Sat Nov 04, 2006 at 02:10:59 PM PST

            [ Parent ]

            •  Hey Dem (2+ / 0-)
              Recommended by:
              We hold these truths, Flinch

              You can inflate votes in heavily Republican districts, and flip votes in counties that are more evenly divided.  How are you going to know one way or another?  Check out ars tecnhcia's report on the actual levels at which the vote count can be manipulated.  Secret vote counting, proprietary software owned by corporations with partisan ties to the Republican party

              It's happenend before in previous elections (2000, 2002, 2004).  And looking at the evidence and the context, I can safely conclude that this administration has and will continue to manipulate the vote count for their favor.  It doesn't take an expert in polling and voting to recognize fraud.  It takes logic, common sense, and some deductive reasoning.

              This administration does not care about the support of the majority of people.  Has there been an administration more upfront in its contempt for public opinion in US history, more brazen in its law breaking?  If you don't have the right to vote and have the vote be counted fairly, then you will not make any progress of any kind on any front.  All of our freedoms are predicated on the freedom and the integrity of the vote.  

              The reason why this administration is so profoundly dangerous is because it could not possibly push  an agenda like theirs through normal democratic means.  The only possible way a regime like this could suceed at the polls is to subvert the process, they can't win in the normal way.  They r not a majoritarian movement, despite what Rove and the corporate media may tell you.  

              Again, it doesn't take an expert to realize this, my analogy to medicine is the reasoning and logic used to come to a conclusion or a differential of high probability.  This administration has shown that it will do anything it pleases, damn the people and their voting.  

              Is there evidence of this?  yes, vote counts that are inflated or way off, exit polls that are discrepant, numerous anecdotes of vote supression and vote flipping on the machines, a technology that allows fraud to occur without a huge massive conspiracy, a motive to maintain power, and history of lying, manipulating, and distorting.

              It doesn't take a rocket scientist, doctor, or a god to recognize a spade for a spade (or in this case, voter fraud).  Look, just be ready to march on Nov 8th to take back our democracy and at least show to the rest of the world that we care enough to do something about the fraud that has been going down in this country.  I hope there is no fraud, and the will of the people is counted fairly, and the proper candidate wins regardless of party, but based on the evidence, history of the previous election and this administration, I doubt it.

              Always be prepared.

              •  I agree with that (0+ / 0-)

                Always be prepared.

                "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

                by Greg Dworkin on Sat Nov 04, 2006 at 03:02:30 PM PST

                [ Parent ]

              •  PS good article (0+ / 0-)

                link

                Don't confuse malfeasance with incompetence.

                Democracy's dirty little secret: Vote counting is a messy business. In nearly every election, votes that shouldn't be counted are, and votes that should be counted aren't. In the 2004 election, at least 850,000 ballots cast somewhere in the nation were never counted, the federal Election Assistance Commission estimates.

                Some voters inadvertently invalidated their absentee ballots by failing to sign them. Other absentee ballots were signed but weren't counted because scanners failed to detect a signature.

                But voters who showed up at a polling place didn't necessarily have better luck: Some lacked proper identification, or were directed to the wrong precinct. Others discovered that they weren't registered, that their names had been purged from voter rolls, or that someone had already voted in their name. Most of these frustrated folks were allowed to cast provisional ballots, but many of those ballots were not counted.

                When a race isn't tight enough for contested ballots to matter, or when the results are not within what election experts call "the margin of litigation," election officials and the political parties' official observers generally accept the imperfections and move on. That's what happens in most races. It's quite a different matter when the vote count for an important office is very close.

                "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

                by Greg Dworkin on Sat Nov 04, 2006 at 03:08:22 PM PST

                [ Parent ]

          •  your epidemiology is ragged (0+ / 0-)

            "Now is it coincidence that the exit polls have been so far off since the Bush administration has come into office, perhaps...."

            Well, the first step is to ask, is it fact? Certainly it is true that the 2004 exit poll was far off, but we know that earlier U.S. exit polls have been off. So, that's one problem up front.

            Setting that aside, one might ask, if we are considering the exit polls as diagnostic, where do they indicate that the largest problems or shenanigans or crimes in 2004 were? In Vermont, Delaware, New York, New Hampshire, and Mississippi. As a denizen of New York, I'm starting to feel quite sulky that two years later, all the folks who purport to have great confidence in exit polls have shown so very little interest in exposing the massive fraud here. It's not good enough to say that the exit polls and the other evidence 'all point to fraud.' They should, minimally, point to fraud in the same places, shouldn't they?

            How many "poli-scientists" have professed bafflement at "historical precedents and odds that Bush overcame"? Not many that I can find. Bush was ahead in most of the polls, and the predictive models indicated that the economy was good enough for Bush to win easily.

            If you would care to winnow and to marshal the evidence systematically, that would be useful.

            •  The discrepancy was large in NY 2004 (0+ / 0-)
              Exit polls had Bush at 34.4% and Kerry at 64.1%

              Official count was Bush at 40.1% and Kerry at 58.4%

              But the difference would not have affected the outcome in NY which was never really in doubt, so it was largely overlooked.

              •  I think you've missed his point. (0+ / 0-)

                That was the point he was making.  If the people (like Freeman) who are so sure that the exit poll discrepancies must indicate fraud, why is no-one trying to figure out how massive fraud was implemented on New York levers?

                Do you think New York levers were hacked?  And how?

                •  There's a good book on NY elections (0+ / 0-)
                  and the history of fraud here.  

                  It's by Ron Hayduk, a professor at Borough of Manhattan Community College, CUNY, published by Northern Illinois Univ. Press, GATEKEEPERS TO THE FRANCHISE: SHAPING ELECTION ADMINISTRATION IN NY.  Unfortunately it was published in 2005, too soon to really cover the 2004 election in depth, although he does discuss some of the differences between counties in different areas in NYC in 2004.

                  Again, Democratic undercount as revealed by the exit polls does not need to be accounted for solely by fraud.  It is a fact that more Democratic votes are discarded, as undercounts, overcounts and provisional ballots.

                  But who knows what happened in NY, who has investigated?  the point is that no one really has.

  •  Recommended (7+ / 0-)
    Great diary!

    And for those of you who aren't acquainted with Febble's (Elizabeth's) work on the 2004 election, then you might want to read this summary by the Mystery Pollster:

    http://www.mysterypollster.com/...

  •  Excellent! (4+ / 0-)
    Recommended by:
    javelina, Fabian, Elise, Big Tent Democrat

    Great analysis!

    I'd like to ask your opinion...if exit polls aren't really suited for ensuring the integrity of elections, are there other options?  I think we can all agree that a VVPT is the best protection, but is there anything else that could be done?

    •  Ooh, where to start.... (18+ / 0-)

      Well, you certainly need a VVPT, preferably a VVPB, i.e. a system where the paper has the full status of a ballot.

      Then you need some kind of proper audit.  There are various approaches to an audit, but the essential ingredients are:

      • that it must be random
      • that what is audited must be unpredictable
      • that there must be secure custody of the paper ballots between election and audit.

      The next issue is what to audit.  I'm in favor of auditing either whole precincts or whole machines (rather than sampling votes from precincts) - in other words of treating the audit as a quality control procedure (which it is).  If a manual hand count doesn't come within whatever the specified accuracy is supposed to be of the machine, then the machine/precinct should fail the audit.  Moreover, even if the errors are small, if they are significantly more in one direction than another, that in itself suggests that the machines are corrupt in some way.

      So then we have to address the question: what should a failed audit trigger?  I would suggest a full hand recount of the relevant race, but being a Brit, I don't find that an unthinkable thought.  But that's a really important issue.

      Apart from audits, I'd really like to see much more transparency simply in the way vote totals are broken out, especially now that early day voting and absentee voting are much more common. It should have to be clear exactly what vote totals have come from where.  These are the kinds of data that would be of enormous help to data analysts trying to detect anomalies.

      My favorite audit protocol so far is this one:

      http://www.votetrustusa.org/...

      But in lieu of decent audits, I think that the actual vote returns are potentially the most importantly, from past and current elections.  Anomalous patterns of "swing" (change from previous elections) seem to me to be a potentially informative.  If magnitude of "swing" correlates with a particular voting methodology, or with suspicious administrations, then that might be a clue.

      There are also fancy things you can do using Benford's law - but that is strictly for the experts!

      One practical thing, though: where possible and available, volunteers should collect precinct counts, and match them against county tabulations.  It seems to me that this should be routine, but it seems extraordinarily difficult to find this data tabulated anywhere.

      •  swinging in the USofA (2+ / 0-)
        Recommended by:
        Rolfyboy6, Elise

        A large problem with basing judgments even in part upon unexpected 'swings' since previous elections is that in the US, more than in the UK, people move around (on average about every 5 to 7 years, IIRC). If you try to make an argument from a 'swing', you're likely to see it shot down emphatically. In purely political terms, if you want to get a hearing, I think it's better to find other kinds of evidence.

        •  Point taken (3+ / 0-)
          Recommended by:
          Rolfyboy6, smintheus, ek hornbeck

          but the real point is that if you use actual vote returns you have way more data.  The thing that I found, looking at the exit poll data, is that at state level there is very little statistical power to pinpoint anything, because to pinpoint fraud you need to look at precinct level, not voter level.  Only 49 precincts in the Ohio dataset - it's not enough to tell you anything much about the predictors of "redshift" OR "swing".

          •  Miofsky-Edison doesn't release precinct data (0+ / 0-)
            This is a huge problem for people who really want to analyze the exit polls and figure out whether they do indicate actual fraud.

            This is also the huge problem with media exit polls.  They are commissioned by the media - the consortium is ABC, NBC, CBS, CNN, AP and FOX - to be used by the media in their commentary (I think they care more about the "voter values" and demographic analysis than the outcome).  And the basic data is owned by the media and not released publicly.

            We need independent election verification exit polls, which are designed somewhat differently.  We need completely transparent polls where the data is actually released.

            •  Well, that is fair enough (0+ / 0-)

              but my own view is that exit polls are an incredibly expensive way of telling you not very much, even if you have access to the precinct level data. That's why I think analysing vote returns  gives you a much better bang for your buck. You need a different benchmark, but there are many you can use.

              But even with the vast resources of the NEP, there are only 30 - 50 precincts sampled in each state.  That is a tiny sample size, and it means the within-state analyses have very little statistical power to tell you what went wrong.  

              Anway, for what it's worth - I found myself in the extraordinary position of being responsible for analysing that detail, and I did figure it out - and it didn't.  That's partly why I posted this diary.

              •  Further question (0+ / 0-)
                what do you mean by

                I did figure "it" out and "it" didn't - what is "it"?

                And yes, a true election verification exit poll would have to be designed differently, with more precincts sampled within individual states.   One way to do it would be to concentrate on battleground states and/or states with high levels of complaints and machine malfunctions in prior elections, and forego states where the result is a foregone conclusion.

                •  Yes, the best kind of exit poll (0+ / 0-)

                  to try to detect fraud would be one that had a clear a priori hypothesis.  I actually recommended that to one of the independent groups.

                  You wrote:

                  This is a huge problem for people who really want to analyze the exit polls and figure out whether they do indicate actual fraud.

                  That was the "it" I figured.  I analyzed the exit polls to "figure out whether they do indicate actual fraud" and they didn't.

                  My main findings were:

                  • Redshift was strongly correlated with methodological factors likely to be associated with departures from non-random sampling (e.g. long interviewing interval).
                  • A fairly small number of methodological factors together accounted for all the net redshift.
                  • There was absolutely no hint of any correlation between the magnitude of the discrepancy and change in Bush's vote share.
                  • When similar sized precincts were compared, discrepancies were similar in precincts serving smaller communities regardless of technology used.  In precincts serving larger communities, the discrepancy was greater in precincts in which older (non-digital) technology was used.
                  •  Specifically what factors (1+ / 0-)
                    Recommended by:
                    StupidAsshole
                    what were the methodological factors that together accounted for all the red shift?

                    As for the lack of correlation between the magnitude of the discrepancy and the change in Bush's vote share:

                    It is very interesting that the states in which the polls predicted a Kerry win but the official count when to Bush were Colorado, Florida, Iowa, New Mexico, Nevada and Ohio.

                    Victory in Ohio, Nevada and New Mexico- or just in Ohio -  would have given Kerry the presidency.

                    •  not so very interesting, actually (0+ / 0-)

                      If there is bias favoring Kerry across a wide range of states, it stands to reason that the states where Kerry leads in the poll but Bush wins in the count will be battleground states.

                      That doesn't react to the lack of correlation between red shift and swing: it simply changes the subject.

      •  Making unnatural data look natural is hard (0+ / 0-)
        Incredibly much harder than most people imagine. Fudged data tends to be chock full of red flags ... and that's if you have ideal control over its production.

        This is just one of several formidable barriers to successful large-scale election tampering.

        If you want to tip an election, it usually has to be very close. Most elections are mismatches.

        You either have to tip a lot of votes in a few places (big red flags, getting easier to spot in the info age), or tip a few votes in a lot of places.

        Unless "a lot of places" means "everywhere", there will be patterns that stand out against background, in addition to the foundational risks.

        Fudging data a whole lot of places means executing a one-time caper (no practice opportunities) in a changing environment of diverse players and protocols, geographically and communicationally distributed.

        This ordinarily means you'd need a big operations center, with a large number of trusted confederates, and you still risk being exposed on account of something broken or incorrectly documented in the target environment, just to win something you have a 50-50 a priori chance of winning anyway.

        None Dare Call It Stupid!

        by RonK Seattle on Sat Nov 04, 2006 at 01:37:03 PM PST

        [ Parent ]

        •  There was a red flag in 2004 (1+ / 0-)
          Recommended by:
          StupidAsshole
          The red flag was that the discrepancy between the exit poll results and the official count was almost always in favor of Bush.    

          That is, Bush almost always did better in the official count than in the exit poll - this was so in 10 of the 11 battleground states,

          There were only 6 states - Kentucky, ND, Oklahoma, SD, West Virginia, Montana - where Bush did worse in the official count than in the exit poll.  In 3 of those states the difference was under 2% so within an acceptable margin of error.  The highest # of electoral votes in these 6 states was 8 (Kentucky).  And these were all states where Bush won anyway.  

          For a statistician, there were red flags all over the place then.  Error should be random - there should have been roughly the same number of states where Bush did better in the polls than the official count as states where Bush did better in the official count than the polls.  50/50 instead of the statistically improbably 6/44.

          Also - the discrepancy doesn't need to be caused by intentional fraud.  It is a fact that more Democratic votes than Republican votes are provisional ballots and undercounts/overcounts.  So Democratic bias in the exit polls is the same thing as Democratic undercount (or Democratic "spoiled ballots" as Greg Palast calls them).
          And Democratic undercount is documented, see Palast's work for starters.

  •  Febble?!? Where have you been? (11+ / 0-)

    Oh, sure, I suppose that making money analzying things for smarty pants folks is more fun than throwing intellectual pearls before rabble-rousing ne'er-do-well swine.  Regardless, you've been missed.

    I'm on my way out the door for GOTV, but I'll be back to read.

    I've chosen Mike McGavick as my special friend!

    by CJB on Sat Nov 04, 2006 at 11:08:27 AM PST

  •  Give 'em hell Liz... (1+ / 0-)
    Recommended by:
    Rolfyboy6

    ...and have a speed-dial ready to Keith Olbermann, the only national journalist with the fortitude to call it as it is.

    "When the going gets Weird...The Weird turn Pro". -- Dr. Hunter S. Thompson

    by Blue Shark on Sat Nov 04, 2006 at 11:08:54 AM PST

  •  I know of at least three reversal of projections (1+ / 0-)
    Recommended by:
    javelina

    derived largely from exit polls that occurred prior to Florida in 2000: Maryland in 1980 (actually was called for Reagan but was reversed to Carter), Georgia in 1992 (called for Clinton immediately when the polls closed but later retracted (even though Clinton ended up winning by a fraction of a percent, a much too small margin to be called immediately upon the closing of the polls), and the NH Senate race in 1996 (can't remember the candidates but it was called for the dem and went repub).

    No, they are not infallible, far from it. And in fact even in 2004 they were within the margin of error albeit close to the edge.

  •  Thanks febble (2+ / 0-)
    Recommended by:
    Torta, Elise

    You are a treasure.

    Now you need to explain why PRE-elecyion polls are not uncannily accurate.

    Keep up the great work.

  •  Febble! So glad to see you back! (4+ / 0-)
    Recommended by:
    lipris, David Boyle, demkat620, Elise

    I was just recently wondering what had become of you...

    Thanks for the preemptive information delivery.  :)

  •  GREAT diary...unfortunately, it won't shut up... (4+ / 0-)
    Recommended by:
    Frank, TrueBlueDem, dnta, Elise

    ...the idiot tinfoil-hat wearing conspiracy theorists on this site who insist "Ohio was stolen," blah, blah, blah.

    Exit polls are great, and I think it's good that since 2000 the networks are holding back on using them to call the close ones, as they did in 2000 and before much to their own detriment.  It's frustrating for me because I want to know who won, but a wrong call is emotionally devastating and much worse than having to wait until later into the evening.

    We're going to take the House on Tuesday, and have even odds at taking the Senate.  Let's get it done!

    In a time of war, is that really the time to be asking whether we should be at war?...When it is over we should ask whether we should leave. -- Stephen Colbert

    by DCCyclone on Sat Nov 04, 2006 at 11:53:14 AM PST

    •  but some of my best friends wear tin foil (3+ / 0-)
      Recommended by:
      DemFromCT, Rolfyboy6, supersoling

      Lots of bad stuff happened in Ohio -- not necessarily all the stuff people talk about. Personally, I have no gripe with people who are actually trying to find out what happened in Ohio. But I have big gripes with the people who say that the exit polls prove it (or I could rattle off a little list of other Ohio arguments that give me heartburn).

      I have no problem with conspiracy theorists per se -- only with the people who, if you disagree with them, start howling that you are repressing them and/or that you must have been paid to say it.

      •  The thing people don't realize about Ohio... (1+ / 0-)
        Recommended by:
        DemFromCT

        ...is that Dems achieved all our voting targets across the state for Kerry.  We got all the votes and more that we thought were more than enough to win.  That's why these conspiracy theories about voter suppression are so badly overstated, if not flat-out wrong, and notions that Ohio was stolen by suppression are compeltely stupid and ignorant.

        Before 2004, normal turnout in a presidential election since the voting age was lowered to 18 was between 50 and 55 percent of the eligible voting-age population--keep in mind that's 50-55% of all eligible adults, not 50-55% of registered voters.

        But 2004 saw a shocking 60% turnout nationwide.

        We were counting nationwide on the notion that higher turnout would mean Kerry wins, and lower turnout favors Bush; our metric was that 55% turnout or better would ensure a Kerry victory.

        Well, guess what?...Ohio had a whopping 68 PERCENT turnout of the eligible voting-age population.

        We lost Ohio because Republicans simply found an amazing number of new voters.

        In a time of war, is that really the time to be asking whether we should be at war?...When it is over we should ask whether we should leave. -- Stephen Colbert

        by DCCyclone on Sat Nov 04, 2006 at 12:22:46 PM PST

        [ Parent ]

        •  at the same time... (2+ / 0-)
          Recommended by:
          DemFromCT, StupidAsshole

          there are good studies that indicate that the misallocation of machines in Franklin County led to five-figure vote suppression. Fairly low five-figures to be sure -- and why the machines were so badly misallocated remains unclear.

          Your point is a good one: when Kerry got 59 million votes, clearly there were limits to the extent of voter suppression.

        •  You lost Ohio because (2+ / 0-)
          Recommended by:
          BigBite, supersoling
          you didn't protect well enough against the massive vote suppression going on there.

          Do you know how many college students and inner-city residents were not allowed, by arbitrarily enforced ridiculous rules, to register?

          Do you know how many Democratic voters left without voting because of long lines?

          Do you know how many Democratic voters were made to vote on provisional ballots which were then not counted?

          Thank God there are groups who do know these things and who are trying to stop this vote suppression from happening again.

      •  i don't say exit polls PROVE it (1+ / 0-)
        Recommended by:
        StupidAsshole
        and neither does Freeman.

        We do say that the exit poll findings raise serious concern and cry out for further investigation, which has never been properly done.

        Though we are coming close in Ohio - or maybe closer is the word - with the historic court decision recently to preserve the ballots there from 2004 to allow further analysis.

        •  but he doesn't seriously analyze the exit polls (0+ / 0-)

          And frankly, I don't think his willingness to reiterate superficial arguments has done very much to promote further investigation. It's too bad. He should have talked with more people who disagreed with him.

    •  The tin foil people are over the top (2+ / 0-)
      Recommended by:
      matt2525, StupidAsshole

      But, they serve a vital function regarding the flaws in our system. It really doesn't matter whether there conclusion is right to me. Their concerns over our system of voting with it being so lacking in a system of checks and balance is spot on. That part of their conversation at least (and the voter suppression conversation) shouldn't be ignored. It's a balancing act for those of us who aren't tin foil types, but its one that needs to be had.

  •  "Vote Count Protection Project" Doing Exits (6+ / 0-)

    Seem to take a different approach than the group you have contracted with.

    Major Independent Exit Poll Projects to Coordinate

    With increased consciousness of the dangers of electronic vote-counting technology, there is increasing grassroots pressure for independent election verification. In elections around the world, public-domain exit polls are central to such efforts. Unfortunately, US media consortium exit pollsters have announced that they will no longer release any data even to their media clients until they can "correct" their numbers so as to make them conform to the official count. In other words, what they report is no longer exit poll data at all, but rather a meaningless affirmation of official numbers.

    To obtain an honest assessment of how people voted and whether or not machines are recording votes accurately, groups around the nation have developed plans to conduct exit polls. We are pleased to announce that several groups at the forefront of the national election integrity movement groups pushing for independent exit polls are will be coordinating in this effort under the banner of the Vote Count Protection Project.

    The combined exit poll will implemented by Kenneth Warren of The Warren Poll, Steven F. Freeman of the University of Pennsylvania, and Jonathan Simon of the Election Defense Alliance. At the same time, election data forensics will be implemented by Stephanie F. Singer of Campaign Scientific and Bruce O'Dell of the Election Defense Alliance.

    All data and findings will be available for independent analyses.

    Are they approaching the exits differently than Edison-Mitofsky and is there a possibility for results that differ from Edison-Mitofsky?

    Good Government. Traffic Lights Aren't All That Weird. Vote Democratic!

    by HL Mungo on Sat Nov 04, 2006 at 11:54:28 AM PST

    •  Well, I've been in contact with them (8+ / 0-)

      and Jonathan Simon has certainly taken seriously the issues that are likely to increase the probability of bias in polls.  

      My understanding is that this venture is conceived of as a pilot in this election, so it will be interesting to see what they find.  

      Bruce O'Dell is a great guy - someone I have enormous respect for.  I was delighted to see he was on board.

      But to address your question: yes, they are approaching the exits very differently, because their proposed poll has a quite different purpose.  Whether it can serve the purpose they plan for it, I am less sure.  The first thing you need to know about polling is the huge number of things that can go wrong, and the Edison-Mitofsky team have a heck of a lot of experience!  Mitofsky was good because of his caution.  I think I'd like to see more caution.

      •  by the way (0+ / 0-)

        I may have missed you saying it, but the exit polls are for Senate and Governor, not House. NEP will not be doing House exit polls.

        "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

        by Greg Dworkin on Sat Nov 04, 2006 at 12:23:31 PM PST

        [ Parent ]

  •  Thanks for this excellent overview (0+ / 0-)

    It's great to have it laid out so clearly, and by someone who really knows their stuff. Highly recommended.

    He that would make his own liberty secure must guard even his enemy from oppression. -- Thomas Paine

    by Leslie in CA on Sat Nov 04, 2006 at 11:56:17 AM PST

  •  Wow (1+ / 0-)
    Recommended by:
    Elise

    That was incredible!  You did a great job explaining things and helping me understand.  I wish I had your myth buster section right after 2004 to dispell some of the notions going around here since then.

    Good job!

    "No government has the right to tell its citizens whom to love. The only queer people are those who don't love anybody." - Rita Mae Brown (-4.75, -7.13)

    by AUBoy2007 on Sat Nov 04, 2006 at 11:56:41 AM PST

  •  Two Questions (2+ / 0-)
    Recommended by:
    Michigan Paul, dfarrah

    If exit polls aren't used to determine the validity of elections then why have them?  Really, I think this is an important question - there should be a rationale for anything you do.  If it is simply to get information on why somebody voted a certain way after the vote has aleady occurred then all you are getting is ad hoc information on an event that can in no way be altered and will never come about again.  It seems less than a useless excercise and I am wondering why so many are willing to invest in it from a scientific perspective.  It is essentially arational.

    The second question I have is, if exit polls cannot determine the validity of an election then what tool can possibly determine the validity of an election.  The first thing you are supposed to do in developing any process is some method to determine if that process did or did not work.  Are you saying that is an impossibility with elections?  

    •  part of the first answer (1+ / 0-)
      Recommended by:
      Rolfyboy6

      Exit polls were designed primarily to give journalists a tool to describe who did what.  How did whites, blacks, hispanics, men, women, union memmbers, consevatives, housewives, whatever, how they voted and ask why they voted the way you do.

      Cook

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Nov 04, 2006 at 12:14:01 PM PST

      [ Parent ]

      •  maybe a bit more (1+ / 0-)
        Recommended by:
        Rolfyboy6

        The 'scientific' folks aren't paying for the exit polls -- the networks and print-journalism subscribers are. That said, political scientists are delighted that we've had access to the data, because some of us are at least as interested as the journalists are in understanding why people vote the way they do.

        How to audit an election? Perhaps paper ballots, with a secure chain of custody, at least some of which are counted by hand as a check on optical scanners. Exit polls could be helpful at the margins, but at best they are an audit once removed.

      •  The poster addressed the question (0+ / 0-)

        to Febble and not Cook.  

        And the concern is with Mitofsky's [and other professional pollster's] polling and it's weaknesses/strengths, not just run-of-the-mill journalists taking a pulse here and there.

        Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

        by dfarrah on Sat Nov 04, 2006 at 12:23:34 PM PST

        [ Parent ]

    •  Funnily enough (4+ / 0-)

      I actually had originally ended my diary with a FAQ, and your question was one of them!

      Well, the answer to your first (which you actually provide yourself, of course!) is actually given in the E-M page linked to from the top of my diary.  It is a quite incredibly informative survey.  The questionnaires are exhaustive (and exhausting, which might be why response rate is low) and tell you a huge amount about what groups of people voted for whom, where and why.  And while it doesn't enable politicians to re-run the election, it certainly tells them what the country is thinking in a way that is scarcely matched by any other poll.  It is a HUGE poll.  And it strikes me that that kind of information is part of what makes a democracy a democracy - it's information about the way people think, what their priorities are, what they fear, what they hope for, who they see as responding to those hopes and fears.

      As for your second question - I think the answer is probably really good random audit protocols.  Which means, as an absolutely essential prerequisite, voter-verified paper records that are capable of being manually recounted, and have the status of a ballot.  That's why I think that HR 550 is such a good first move, and why I'd like to see it supported (and why I'm rather appalled at the resistance it has encountered in some parts of the election integrity movement.

      But the bottom line is that to verify an election, you need both transparency and some kind of independent check on the count.  Unfortunately the NEP polls are not designed to be that check, and even if they were, probably would be a poor tool for the job.

    •  That's easy (1+ / 0-)
      Recommended by:
      DemFromCT

      It gives the talking heads something to count while they wait for results. It gives them something to speculate about.

      I worked for NBC News Elections as a pollster the year that Reagan was elected. That was the first year that I know of that a network actually predicted the results before the polls closed and predicted them based on exit polls. Bad business all around and set the table for the crow we had to eat in 2000.

      I think that polls in general can show trends, as mentioned in the post above.

      Exit polls can help determine a clue to the validity of an election ONLY WHEN COMBINED WITH OTHER FACTORS. I don't think that the results, by themsleves, are proof of anything.

    •  Media polls vs verification polls (1+ / 0-)
      Recommended by:
      StupidAsshole
      Wilbur, the answer to your questions is that there are two different types of polls.

      Media exit polls, ie the NEP poll that Febble is talking about, is designed for the media, commissioned by the media to be used by them in their commentary, like when the talk about who women voted for, the values that drove people to the polls, etc.   They also own the data, which they do not fully release.

      Election verification polls we don't really have in this country, and they are designed differently and ideally they are transparent, that is, anyone can see the actual precinct-level data and they explain their methodology as well.

  •  Great writing, Febble (5+ / 0-)

    Explaining math and/or science to a lay audience in plain English without sacrificing accuracy is awfully difficult. You did it.

    "We grow too soon old and too late smart."--Pennsylvania Dutch saying.

    by Dump Terry McAuliffe on Sat Nov 04, 2006 at 12:00:30 PM PST

  •  Excellent work here! (2+ / 0-)
    Recommended by:
    Rolfyboy6, jmonch

    High quality!

    Glad to see it front paged or I wouldn't have seen it since I'm calling through moveon at the moment.

    Thanks for the thorough explanation...this is something that has confused me for a while and I mostly attributed my confusion to my inability to understand math and statistics. Apparently all I needed was a good explanation!

  •  Do not buy this (9+ / 0-)

    DemfromConn has been pushing the idea that exit polls are no help in detecting election fraud for a couple of years now. The fact is, in an election as ripe for fraud as this, exit polls are one of the few tools we have for detecting it. The attempt to obfuscate this point (for whatever reason) has been one of the biggest time wasters in the entire fraudster debate in which people become lost in minutae and miss the forest for the trees.

    It is common sense: If there is widespread cheating going on which is otherwise undetectable because it's done by hacking vote machines, vote cards or tabulators, then the raw data of exit polling is one of the best indicators of this acitivity.

    In elections that are less subject to fraud, exit polls play less off a role in monitoring integrity. But in an election like this where the machines are absolutely deficient and supply zero security from hacking, and where one party in particular is suspected of stealing three elections, we need every tool we cann get.

    Borat On Daily Show: "In Kazakhstan We Have Midterm Elections Just Like You, The Next One Is In 30 Years"...

    by moon in the house of moe on Sat Nov 04, 2006 at 12:01:05 PM PST

    •  I didn't actually say that exit polls are no help (6+ / 0-)

      I think they may be a canary in a coal mine - but the problem is that the canary is pretty sick anyway, and if it falls off its perch, you won't really know whether the cause is gas or psittacosis.

      So it wouldn't be my first line of investigation - just a hint that something might be wrong.  I thought it was worth investigating, which is why I wrote the paper I did, in which I called for a reanalysis - and ended up with the job.  But in fact, what I found was evidence against the case that the discrepancy was due to fraud. If I had found a positive correlation, it would have been quite suggestive evidence for fraud.

      But I don't think exit polls are a terribly useful tool, even if they are designed to monitor the integrity of the elections, as the NEP ones aren't.  However, if the NEP poll is all you've got, sure, it's worth raising a red flag.  As long as you are aware that there are many many things other than fraud that can cause a discrepant exit poll result, and that these things are extremely common.

      •  Sorry I still don't buy it (4+ / 0-)

        In an election where the voting machines are hackable and unverifiable, exit polls by comparison, are highly reliable. It's all relative. The red flag in the corrupt Ukraine election came from exit polls. We are no better.

        Borat On Daily Show: "In Kazakhstan We Have Midterm Elections Just Like You, The Next One Is In 30 Years"...

        by moon in the house of moe on Sat Nov 04, 2006 at 12:23:12 PM PST

        [ Parent ]

        •  Exit polls are only (3+ / 0-)
          Recommended by:
          NCrefugee, Febble, cometman

          absolutely reliable to show large flips, well outside the MoE.  If you had an exit poll that read 51-49 Kerry, and the actual counted result was 51-49 Bush, that's only a two point flip.  Even exit polls have MoE's, and that's well within even the most accurate poll.  In addition to your MoE, if you take a bad sample, or make bad assumptions, Now if you had a counted result of 51-49 Bush, and the exit poll said 60-40 Kerry, there's your red flag.  But all it would tell you is some shit went down.  It won't tell you what the shit smelled like and how much there was.

          And that's useful in countries like Ukraine where it's common for ballots to magically appear in a box, or for ballot boxes to fall off the back of a truck as it's driving over a bridge.  Flipping a few thousand votes in a computer is not detectable by exit polls.  Suppressing the vote is not detectable.  Until all the machines start flipping 5-10 points consistently the same way, we won't see a huge problem in the exit polls.  Once you're within the margin of error of the poll, you need better controls, including verifiable voting, and strict controls in the equipment, if you want a more precise count.

          (-7.25, -5.85) "Talk amongst yourselves. The Christian Right: neither Christian nor right. Discuss." --Linda Richman

          by Slartibartfast on Sat Nov 04, 2006 at 12:37:52 PM PST

          [ Parent ]

          •  And that's why I don't trust those machines... (3+ / 0-)
            Recommended by:
            bdevil89, Xeno of Elia, StupidAsshole

            Pre-election polls showed very tight races, and it doesn't require too much of a vote flip to change the result without being detected.

            I think exit polls are a lot like economics in that it is impossible for them to take into account human nature.  People don't always tell the truth.

            I'm sure election fraud has taken place ever since the concept of elections was invented. Democrats do it,Republicans do it, Communists, fascists, Tories, Whigs, federalists, you name it.  However in the past it could only be done on a small scale and  a lot of fraud may have just been cancelled out by both sides doing it.  With these machines, there is at least the hypothetical possibilty that it could be done on a much larger scale, especially when the companies who produce the machines are affiliated with one side or the other.

            Political parties come and go, but human nature doesn't change.  People do nasty things all the time if they think it will give them an advantage.  In Olympus, Greece you can still see statues of the victors of the games, as well as ones for athletes who got caught cheating and had their names etched in stone for perpetuity so the rest of us wouldn't forget what they did.

            Whenever the stakes are high, people will cheat in order to win, and the rest of us must remain vigilant to make sure the cheaters are caught.  

            The meek shall inherit nothing. -F.Zappa

            by cometman on Sat Nov 04, 2006 at 12:59:49 PM PST

            [ Parent ]

            •  that's why we need better recount laws (1+ / 0-)
              Recommended by:
              StupidAsshole
              Or as some call them, REAL COUNT laws.

              In every state it should be the case that if an election result is within 1 or 2 percentage points that triggers an immediate recount.  A handcount of paper ballots.  Publicly observed.

              Actually, we should have public hand counts of paperballots from the get-go.  As they do in the UK and elsewhere.

          •  I love this place (0+ / 0-)

            "But all it would tell you is some shit went down.  It won't tell you what the shit smelled like and how much there was."

            I love this place. :)

            Re-elect Jennifer Granholm. Keep Michigan Blue!

            by dconrad on Sun Nov 05, 2006 at 12:02:58 PM PST

            [ Parent ]

          •  Managed to miss your post (0+ / 0-)

            excellent.

            And I love your handle.

        •  Well, let's unpack this.... (7+ / 0-)

          how does one unreliable way of counting votes make another method more reliable?  It doesn't.  It is appalling that your election system is so non-transparent that it can even be considered that exit polls would be more reliable.  What an indictment.

          If the American people can't trust the electoral system to give them a fair result, then your democracy is broken.  Period.

          But it still doesn't make exit polls a good estimator as to who won.  There is information you can glean from them, which is why I tried to glean some.  But the answer turned out to be: bias in the poll.  And moreover: no relationship between disrepancy in the poll and boost to Bush.  Which means that it is pretty unlikely that the exit poll discrepancy was due to fraud.  Which probably means that the vote count was, if hacked, not hacked on the scale implied by the exit poll.  i.e. not millions.

          But I think, myself, from other evidence, that Kerry won NM (not that the exit poll was particularly out in NM) and that he lost key votes in Ohio by means that wouldn't have shown up in an exit poll anyway (inadequate supply of voting machines to Democratic precincts in Franklin County).

          And the red flag in Ukraine didn't come from the exit polls.  It came from direct evidence in fraud, including the poisoning of one candidate.  And there were two polls, as I understand it which didn't even agree with each other.  Ukraine is one hell of a red herring.  Sure, a corrupt election will have nonsensical exit polls.  But it doesn't mean that discrepant exit polls must indicate a corrupt election. I think US 2004 was corrupt.  But I don't think that because of the exit polls, and in fact, I think the exit polls indicate that the corruption was unlikely to have taken the form of multi-million vote theft.  If it had, I'd like to know why there is absolutely no correlation between exit poll discrepancy and change in Bush's vote-share.

        •  I think you're confusing (1+ / 0-)
          Recommended by:
          StupidAsshole

          potential with actual.

          As a computer geek since the days of the trs-80, I am well aware that computers should never be the sole tool for anything as important (and contentious) as elections...

          But as a poll and count (and recount) observer in a Blue area, I still suspect old fashion voter suppression and the medias equivocating fact and spin, to be as dangerous to democracy as hackers.

          No one has the smoking gun yet though many have hyped potential guns into cannons, so I am willing to read discussions from those who are more familiar with exit polls than I.

          The biggest threat to America is not communism, it's moving America toward a fascist theocracy... -- Frank Zappa

          by NCrefugee on Sat Nov 04, 2006 at 12:46:42 PM PST

          [ Parent ]

      •  Can we get your study (1+ / 0-)
        Recommended by:
        StupidAsshole

        and the raw data you used?  This is all so interesting.

        Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

        by dfarrah on Sat Nov 04, 2006 at 12:44:52 PM PST

        [ Parent ]

        •  Well, no.... (1+ / 0-)
          Recommended by:
          Eddie Haskell

          sorry about that.  I did it under contract for Mitofsky, and so I'm bound by the same constraints regarding release of confidential data as any other employee.  

          I'm aware this is an issue.

          Some of my findings, however, are in the public domain, including the lack of correlation between "redshift" and "swing" to Bush.

          Here's the link to the finding again, which was written up by my colleague, Mark Lindeman, with whom I worked on the measure of discrepancy I used.

          http://inside.bard.edu/...

          •  well phooey, (0+ / 0-)

            [as if I have time to dig thru it all anyway]

            Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

            by dfarrah on Sat Nov 04, 2006 at 01:04:27 PM PST

            [ Parent ]

            •  Read the Freeman book too (1+ / 0-)
              Recommended by:
              StupidAsshole
              It's reviewed as well, along with 9 other titles, in tomorrow's New York Times book review by Michael Kinsley.
              •  In which Kinsley says (0+ / 0-)
                that the worst problem is not election fraud or faulty machines but . . . intellectual dishonesty.

                By which i think he means politicians who say things that they don't fully believe in.   (Not outright lies which are easily exposed, but promises and platforms they don't intend to keep).

                Go figure.  I'm sure there will be much discussion of this one.

                •  he also says (0+ / 0-)

                  "...the authors [Freeman and Bleifuss] offer no particular reason to believe the random exit polls and disbelieve the actual vote.

                  "The authors deny that their case rests on exit polls alone. 'Far from relying only on the exit-poll data, we read widely and had countless soul-searching conversations with each other and with our colleagues,' they write. Actually, the whole stolen-election-2004 indictment has that echo-chamber sound of people having soul-searching conversations with each other."

                  Ouch. I don't expect to be seeing that in your PR materials.

                  Frankly, Kinsley has a good point, however rudely he states it. Unfortunately, it's one of those points that is very hard to hear.

                  •  Wait for letter to the editor (0+ / 0-)
                    from Freeman in response.

                    Kinsley's use of the word "random" is unclear and unscientific, for starters.

                    Still, we are glad that he felt the book was worthy of notice and worthy of being included in the discussion.  There are a number of other books he might have cited but did not.

                    The echo-chamber comment is Kinsley's own opinion, with which I and many others do not agree.  

    •  But... (0+ / 0-)

      the point is, is that if exit polls are not as accurate as people seem to think, then they're not a good tool at all.

      Also, as a few others have pointed out, up to a 1/3 of voters don't vote on Election day.  So that's got to impact exit polls also.

      "No government has the right to tell its citizens whom to love. The only queer people are those who don't love anybody." - Rita Mae Brown (-4.75, -7.13)

      by AUBoy2007 on Sat Nov 04, 2006 at 12:17:58 PM PST

      [ Parent ]

      •  They are only a tool (0+ / 0-)

        for predicting elections by journalists. Journalists aren't in the room with the vote counters so the only way they have of keeping us up to date on what might be happening is to conduct exit polls. They aren't doing it to keep tabs on the honesty of the election.

      •  they are OK for many purposes (0+ / 0-)

        They may not be accurate to 2%, but if you want to know something about how, say, married women voted, the exit polls will give you a decent idea.

        Since some "people seem to think" that they are practically unfallible, I can't agree that if they don't live up to that standard, "they're not a good tool at all," if you see what I mean. But they're not a great tool for election verification, although they could be useful in some circumstances.

    •  You make some good points.... (2+ / 0-)
      Recommended by:
      Michigan Paul, Rolfyboy6

      but I don't think the diarist is trying to say our election results over the last few years should be taken as a valid outcome. She is just trying to show how exit polls work and possible flaws in them. I don't think she is trying to claim that fraud did or did not occur.  

      And until someone can explain to me why a republican led congress would allow voting machines made by republican backed companies to be used in national elections with no paper trail, I'm going to assume they are up to something.  Especially when I see all the voter supression tactics used by Ken Blackwell, Katherine Harris, etc...

      The meek shall inherit nothing. -F.Zappa

      by cometman on Sat Nov 04, 2006 at 12:18:37 PM PST

      [ Parent ]

    •  minutiae? (0+ / 0-)

      Certain folks have been spewing bunk about the 2004 exits for the last two years. It's a bit late for complaints about forests and trees.

      If you have managed not to be in the number of the bunk-spewers, my hat is off to you.

    •  This is an excellent setup, though (4+ / 0-)

      For fraud to be explained away with seeming rationality. On November 8th, I expect that this diary will be resurrected and used as "proof" that there is no fraud in US elections, so as to silence any who suspect it. It will be deja vu all over again.

      There is no such thing, at this stage of the world's history in America, as an independent press. - John Swinton, 1890

      by Xeno of Elia on Sat Nov 04, 2006 at 12:23:04 PM PST

      [ Parent ]

      •  could we please try rationality? (0+ / 0-)

        Exit polls cannot be used to prove that there is no fraud in US elections. Which isn't to say that no one will try it. But we might as well try to get the facts straight.

      •  Well, if it is, I shall be the first to object. (0+ / 0-)

        "Proof" is for math and alcohol.  Not science.  

      •  You can't prove a negative (1+ / 0-)
        Recommended by:
        AskQuestions

        You can't use an exit poll to prove "no fraud."  There may be a presumption against fraud based on exit polls, but that presumption is rebuttable.

        At the same time, exit polls, if taken properly based on the proper assumptions, when they show a result well outside the margin of error, that poll is strong evidence of fraud.  Here there is a presumption for fraud, also rebuttable.

        Just like we are to take the pre-election polls with a grain of salt due to the MoE and sampling errors, we must do the same with exit polls.  When we see one that's way out of whack with the others, we question it's validity.  If there is one that's way out of whack, and it happens to be the actual election, we question it's validity also.  But if the election is in the same range as most of the others, we need more proof.  Not to say, it didn't happen, but if the election results are within the MoE, you just need more proof than if they were outside.

        My point is we shouldn't be happy with having results merely within the MoE, but exit polls alone will not get us there.  I appreciate all the work of those who question privatization of voting, and that's where the energy should be focused.  We can't improve the art/science/damn lies of statistics, so we need something better.

        (-7.25, -5.85) "Talk amongst yourselves. The Christian Right: neither Christian nor right. Discuss." --Linda Richman

        by Slartibartfast on Sat Nov 04, 2006 at 12:53:02 PM PST

        [ Parent ]

  •  You say that (2+ / 0-)
    Recommended by:
    StupidAsshole, supersoling

    two experiments that I know of, methodological factors were manipulated in order to try to increase response rates (one involved giving free folders; another involved experimenting with shorter questionnaires).

    Was there evidence that a methodological factor/s was manipulated--or even in play--for the 2004 elections?  A methodological factor/s that would have had a nationwide effect? [I do recall Mitofsky mentioning some items, but I don't recall].

    I don't know who to believe about exit polls.  Above thread commenters note exceptions for races; however, these are exceptions and not typical.    

    I am overgeneralizing, but it seems just about anything will cause or create a dem bias. If polls are so biased to the dem direction, then why aren't there more exceptions [where the election is not a blow-out for either side] in results versus polls or retracted calls?

    And the big elephant in the room is that now you work for Mitofsky and have a stake in his conclusions regarding 2004.

    If polling is so iffy, then why even bother with it? Why even study its practice or promote it as a business, election, or consulting tool?

    So, would you recommend that dems just automatically shave off, say, 5 to 8 points off of the polls because of bias in favor of dems?

         

    Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

    by dfarrah on Sat Nov 04, 2006 at 12:02:10 PM PST

  •  I see people with torches approaching... :) (15+ / 0-)

    Science is only supposed to dispel right-wing myths. :)

    If you're not on the crazy wild-eyed zealot bandwagon, then you're with the terrorists. :)

    by cskendrick on Sat Nov 04, 2006 at 12:02:29 PM PST

  •  Pollsters run into all kinds of sampling issues (3+ / 0-)
    Recommended by:
    Rolfyboy6, Elise, frenchman

    "You've got your mother in a whirl,
    because she's not sure if you're a boy or a girl..."

    "It's just like the 60's, only with less hope." -Justin Bond in the film "Shortbus" (-6.38/ -4.21)

    by wonkydonkey on Sat Nov 04, 2006 at 12:05:31 PM PST

  •  a question (1+ / 0-)
    Recommended by:
    NCrefugee

    In some areas in the US, extending even to whole states, up to a third of all voters are now voting early. This category includes absentee ballots, but the number of people who simply go to their election clerk's office to vote, weeks prior to Election Day, now can exceed the number of traditional absentees (for whom the process, usually via "snail mail" can be cumbersome).

    It seems conceivable that the number of pre-Election Day polling stations might eventually be expanded - not to the level of precincts, obviously, but such that we can imagine elections in which Election Day would only be the last of many days on which voters could vote, and the day on which the usual large number of stations would be open.

    We could easily reach a point at which over half the voting population votes early. And in a development which was unexpected to me (and which still seems exceedingly odd), we seem to be able to get not only raw counts and party affiliations for early voters, but actual running tallies in a very few cases at this point.

    Don't you think this trend has far-reaching implications for the way in which elections are conducted, and for the way in which pre- and post-election sampling is conducted and interpreted?

    -10.00,-10.00. Beat that, motherfuckers.

    by frenchman on Sat Nov 04, 2006 at 12:12:23 PM PST

  •  Random Sampling is a myth (1+ / 0-)
    Recommended by:
    Rolfyboy6

    I find a good deal to argue with in Febble's analysis (although it is well done).  I'm not sure how much I can get to so I just wanted to take some issues one at a time.  The first and most important is that you cannot trust exit polls because it does not involve random sampling.  But the idea of random sampling is one of the great myths of the social sciences - it basically doesn't exist.  It is far too costly and difficult to do, even for very well funded studies.  I sit on a panel where I have the chance to read hundreds of study designs from all different social sciences.  I have never actually seen random sampling as a design for recruitment of subjects.  Recruitment is almost always done using some variation of snowball, which is where you recruit the first few samples as randomly as possible and then use them to find further samples.  As a matter of fact the design for exit polls I would hazard to guess is more random than more social science experiments.  It is important to understand that if you use this as a critique for saying you cannot trust exit polls to determine the possibilities of an outcome then you can throw much of social science research out the window with a similar critique.

    •  Assuming argumento that you are right (0+ / 0-)

      How then should one evaluate the polls? If cost is a factor, and random sampling is difficult to do, how much weight should they be given if at all when looking at the race on Tues?

      •  You have to give weight (0+ / 0-)

        You have to give weight because basically that is all you have.  Let's say I want to determin how many people voted for dog catcher.  I question the people coming out of the booth, six say A and 2 say B.  I then generalize to the larger population and determine whether my sample is within a standard deviation.  What I have is not random (it is always the case that those who respond to anything are those who are pre-disposed to respond), but it is the best available information and needs to be understood and accepted that way.  It might have flaws, but it is the best information we have at the time and we need to use it to draw conclusions.  Much or research is saying that you cannot let the perfect be the enemy of the good, or even the decent.

        •  I think her argument (2+ / 0-)
          Recommended by:
          Febble, AllisonInSeattle

          and, maybe I am wrong, is that the information is good or descent, but to those people expecting perfect (as in the tin foil types who say this is conclusive proof of stolen votes), that is not true. That it's not conclusive. All this i nformation is merely suggestive, and you need to realize it as such. But. that's just my opinion. I said at the time- there is a 3rd way to view this. You (and it maybe somehow, she) are both saying similar things if in different ways. You point is its descent to good so it shouldn't be discounted, and she flips that by saying its only descent to good so one shouldn't over weigh it. It's really the flip of the same coin which is saying weigh the evidence, not come to definite conclusions- is that not right?

          •  and then there are next steps (1+ / 0-)
            Recommended by:
            Febble

            I'm not sure where to start on this, I've been through it so often. But one step is: if the magnitude of fraud varied from state to state, then we would expect both the exit poll "red shift" discrepancies and the deviations from pre-election polls to be larger in states with more fraud. But actually they aren't statistically related.

            (No stats required: one can look at, for instance, New York, where there was large "red shift," but Kerry if anything did better in the official returns than in the pre-election polls. More generally, one can look at the states with the biggest red shifts, and the states with the biggest deviations from pre-election polls, and note that they generally aren't the same states.)

            So, one doesn't just say 'the exit polls aren't very reliable' and throw up one's arms. One looks around to cross-reference the results with other data sources. Only, the people who interpret the exit polls as proof of fraud tend to be very selective in their cross-referencing.

            •  I think a lot of times people are (2+ / 0-)
              Recommended by:
              StupidAsshole, HudsonValleyMark

              arguing apples and oranges. to me at least, I was amazed by people, for example, trying to call everyone who had a problem with paperless voting machines crazy. To me, one doesn't need a conspiracy to find something wrong with a system that is less accountable than say going to the local ATM machine. I don't know about voter fraud, but I do know mistakes can and do happen. Maybe what is called voter fraud should be called voting system integrity.

              THe other confused argument are those who want to say anyone who says there is voter supprression are engaging in the same arguments as voter fraudsters who always yell Diebold no matter what the conversation. There is no doubt there are attemps by the GOP to engage in suprresion- look at the guy recently imprisoned for it in NH, or Blackwell's manipulation in OH, or any number of other examples. The only people conflating the two are those who want to say that all of it is tin foil theory.

              Honestly, both sides to me are full of a bunch of idiots arguing passed each other. I said that back in 2004. there are some legitimate issues here which need to be addressed, and the best way to address it, to me at least, is by analyzing the situation without regard to trying to convince oneself there is fraud, or convince oneself there is no way that there is anything wrong with the system.

    •  Well, I take your point, but I think you have it (0+ / 0-)

      slightly back to front.  Being a cognitive psychologist myself, I rely almost entirely on volunteer samples.  This makes generalizing results a bit dubious.  However, for the kinds of studies I am doing, I am not attempting to quantify the presence of some characteristic in the population.  I am attempting to compare members of one group (say people who suffer from a disorder) with members of another (people who don't).  The fact that my samples are not random means I need to be cautious when extrapolating my findings to the population, but if they have strong internal validity, it gives us important information to test on wider samples.

      However, with exit polls, the entire study purpose depends on a random sample.  We are not comparing a volunteer group of Democrats with a volunteer group of Republicans on some characteristic.  We are trying to estimate the population of each in a specific population (the population of voters in a precinct).  So a volunteer sample has no validity whatsoever.

      So the next question is: how respresentative is the sample likely to be?  I have recently been working on a dataset that was collected in an attempt to ascertain the prevalence of certain childhood mental disorders conditions in the UK.  Again, the validity of that study entirely depends on the extent to which the sample was randomly drawn (and it was, again, a clustered sample).  It's pretty good, but not perfect, as there was some non-response, and so we cannot rule out non-response bias.  This constrains the accuracy of our estimates.

      So where randomness matters - randomness matters.  And it matters par excellence in exit polls, when used as estimates of the vote count.  It matters less for the cross-tabulations, as a few percentage points here or there won't make a lot of difference to broad findings such as "women with children tended to vote for X" or "evangelicals tended to vote for Y".

      But if there is reason to suppose the sample is not random, then there is no basis for infering that any discrepancy between the poll proportions and the vote proportions must have been due to fraud. It is only too likely that it will be due to non-random sample.

  •  Non-response bias cannot be measured (2+ / 0-)
    Recommended by:
    Rolfyboy6, FaithAndReason

    A second issue is that non-response bias cannot be measured.  If a person decided not to respond to a question or process YOU CANNOT MAKE ANY ASSUMPTIONS AS TO WHY THEY ARE NOT RESPONDING.  Not only is it impossible (it demands a certain amount of mind reading), it is unethical.  You are supposed to report non-responses and then use statistical techniques to determine if that percentage of non-response skewed the outcome, but that is all you should do, and as a researcher all you are allowed to do.  So pollsters should be recordning non-response (do they do that).  But this is why the idea of non-response more by Republicans is not only a red herring but goes against many of the canons of the scientific method.  

    •  same question above (1+ / 0-)
      Recommended by:
      AllisonInSeattle

      how does one account for this issue then- I understand what you say- scientific techniques- but can you specific what that means?

    •  huh? (0+ / 0-)

      Right, we can't verify "non-response more by Republicans" directly. If that makes the very idea a "red herring," then I think a great deal of science just went out the window.

      •  You can't do it (0+ / 0-)

        It's actually pretty easy.  You can't assume the reason behind a non-response bias because there's no way you can know it.  You are basically not allowed to ask why when somebody does not respond because that is their business (and very often they do not tell you the truth anyway - and there have actually been studies to prove this).  You also can't make assumption because you are putting your bias on any assumption, and it is not part of the actual experiment that you are doing.  All right, let's say I'm trying to measure reaction to pornography on the internet.  I have them sit down at the screen and the dirty pictures flash on and they either have to register pleasure or disgust.  After one picture a person gets up and says I don't want to do this.  Now there might be all sorts of reasons for this but I can't know what they are and it is not part of my design to know what they are.  I can make a personal assumption but I have to understand my personal assumption comes from one of two venues - either it is related to what I want to find or it is based on my belief system.  I simply have to report the number of non-responses. If it reaches a certain level it skews my data and I get called on it and have to either get more subjects or find a different method of recruitment.  The idea that non-responders are more likely to be Republican is ad-hoc and the worst possible social science.

        •  I say again, huh? (0+ / 0-)

          How is it any more ad hoc to hypothesize that the exit poll discrepancy is due to differing completion rates among Kerry voters and Bush voters, than to hypothesize anything else about it?

          Of course, if we don't have to hypothesize anything about it, then I guess we're done.

          Certainly it's unfortunate to have a hypothesis that one can't test directly, but a great deal of science proceeds under this handicap. To declare an entire idea "the worst possible social science" because a direct test is impossible is... strange.

        •  Actually, it is rare (1+ / 0-)
          Recommended by:
          matt2525

          in science that we measure anything directly.  Certainly in psychology, which is my field.  We cannot see into the mind, even with fMRI - what we observe are neural correlates of mental events. But it is perfectly good science to examine those correlates, and from them to infer what might be going on.

          Non-response bias is difficult to study, but exit polls give a good opportunity to study it because we have two independent measures - the poll and the count.  If you can truly rely on the poll, that is perfect of course, because then you really can measure non-response bias (or at least bias - it may be selection bias) pretty directly - you can't see inside people's heads, but you can see whether you've got more of one kind of voter than you should have, given the "true" count.

          However, if you don't know that your other measure is right either (in other words if you can't rely on the count) then you have to be more subtle.  You need to find out whether the discrepancy correlates with something likely to be associated with fraud, or with something associated with either non-response, or selection bias.  And that is what Edison-Mitofsky did, and what I did, subsequently, using what I would argue is a better measure of discrepancy.  And I found absolutely no correlation between discrepancy and change in Bush's vote share (i.e. with a variable that is also likely to reflect fraud) but very substantial correlations with variables likely to be associated with selection bias.  This is fairly strong evidence that selection bias was present in the poll, and that fraud is unlikely to have been a major contributor to the discrepancy.

          The "idea that non responders are more likely to be Republican" is not ad hoc at all - it isn't even a priori.  In fact there is fairly little evidence for "non-response bias" in the data. There is evidence for selection bias, and thus supports the inference that Republicans were less willing to be interviewed, and thus less likely to be selected where it was possible for the selection process to be evaded.

          But the a priori hypothesis was simply that methodological factors would be correlated with bias.  And they were.  Why they should have been is a matter for interpretation.

          •  Poll shows R's reluctant to take polls (0+ / 0-)
            Pollster.com relates analysis of this Fox News/Opinion Dynamics poll question:
            Every election, the television networks conduct exit polls ... how likely is it you would be willing to spend 10 minutes filling out a questionnaire?
            Democrats:
                 72% "likely" and 44% "very likely".

            Republicans:
                 66% "likely" and 35% "very likely".

            More where that came from.

            None Dare Call It Stupid!

            by RonK Seattle on Sat Nov 04, 2006 at 04:34:42 PM PST

            [ Parent ]

            •  The poll you (0+ / 0-)

              cite was taken recently.

              How do you know that the jury pool wasn't tainted, so to speak, about the issue?  With all of the controversy surrounding polls in the last few years, how does one know that repubs just aren't performing to the question?

              Are there older sources that indicate that dems are [were] more likely to respond than repubs [and if so, I guess this would beg the question as to why pollsters weren't correcting for this factor as they do for so many other factors]?

              Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

              by dfarrah on Sun Nov 05, 2006 at 07:45:52 AM PST

              [ Parent ]

            •  Not so according to Pew (0+ / 0-)

              Pew research has done (twice at least) a study of polls in which they take a standard telephone poll (which gets the usual 20% or so of completed responses) and then simultaneously, a very thorough poll with post cards and follow-up that captures responses from about 75% of those selected. One of their questions is about party affiliation.

              Republicans are actually more likely to complete standard telephone polls than Democrats, and the difference was statistically significant.

              When Fox asks the question in the way they did it "primes" the response. Pew's approach is scientifically more appropriate.

              ... and neither poll sees the kind of bias or the pattern of non-respnse that Edison/Mitofsky would need to account for the differences between their exit polls and the votes in Ohio and Florida (and PA also in 2004).

  •  Hey, it isn't an election without Lizzie around (2+ / 0-)
    Recommended by:
    Eddie Haskell, frenchman

    Good to see you again, Lizzie.

    Still in that damn Labour Party of yours, I note.

    Even if you ever get rid of Blair, you would still have Brown. And of course, the ugliest one of them all: John Reid.

    What a waste of a great Liberal heart you are, gal.

  •  Thanks for the information (1+ / 0-)
    Recommended by:
    Flinch

    but I was struck by the number of times that people used the non-correlation of exit polls and final vote count to conclude that there was no fraud.

    Okay, for the sake of argument I'll buy the fact that you can't use exit polls to say there was fraud but that's not the same thing as saying that proves there was no fraud.

    It only says use something else: like errors in vote count reality exceeding registered voting, like down-ballot Democrats getting more votes than Kerry, like every anomoly favoring Bush over Kerry to some astronomically high improbability.

    There is more "etc" but for a comment to a diary I'm not doing the research to bring them here. Besides, I'm supposed to be working.

    Still, I appreciate the diary which I read in full and even read some sections 2-3 times to try to understand. I failed here

    "There is no evidence that Republicans were less likely to respond to pollsters in 2004"

    There are strong correlations in the exit poll data between methodological factors (such as interviewing rate) and the magnitude of the precinct-level discrepancy.  Where factors were present that would have made it easier for unwilling voters to avoid being polled (or for eager voters to volunteer) the greater was the observed discrepancy.

       I've read this 5 times and while I'm sure of the conclusion due to the fact that it's a myth-buster I still don't see how the conclusion was reached.

    There are others but I gotta get back to work.

    -4.25, -6.87: Someday, after the forest fire of the Right has died we'll say "Whew, I'm happy that's over."

    by CanYouBeAngryAndStillDream on Sat Nov 04, 2006 at 12:27:38 PM PST

  •  Your arguments are convincing (0+ / 0-)

    and I am not qualified to refute them.

    But on the other hand, they are the Mitofsky party line.  Other analyses I have read are equally convincing, but come to different conclusions.

    Can you comment on this statistical analysis of Ohio by US Count Votes', which was the first to convince me that voter fraud may well have occurred?

    They concluded that:

    Therefore the Mitofsky/Edison ypothesis of reluctant Bush poll responders is irrelevant to explain the discrepancies between the exit poll and election results in the presidential race.

    And a later reanalysis by the same group:

    Ohio's exit poll discrepancy pattern is consistent with a hypothesis of outcome-altering vote miscounts primarily favoring Bush.44 In other words, Ohio's exit poll discrepancies are consistent with the hypothesis that Kerry would have won Ohio's electoral votes if Ohio's official vote counts had accurately reflected voter intent. The patterns of Ohio's exit poll discrepancies are similar to the patterns in the national exit poll sample shown in the January 19, 2005 Edison/Mitofsky (E/M) report and discussed in earlier USCV reports.45

    And finally, this criticism and request of Mitofsky

    It is reasonable to ask Edison/Mitofsky to make publicly available the raw precinct level data and weights used to calculate both their “ call-3”and "Simon" data sets, and explain to the public its selective avoidance of these data sets in their report

    Has Mitofsky released this data?

    I don't honestly know what to believe, but I do know that I want to find out the TRUTH.

    Got an issue, here's a tissue - Will & Grace

    by Flinch on Sat Nov 04, 2006 at 12:28:28 PM PST

  •  Blatant apologist (3+ / 0-)
    Recommended by:
    bdevil89, StupidAsshole, Flinch

    I'm sorry, but the gist of this diary is that those who talked about voter fraud and pointing to exit polls to support them are wearing tin-foil hats.  The fact that Kos chose to front-page this blatant voter-fraud apology piece is telling-is he expecting the exit polls to again diverge from the vote counts, as Karl Rove does?

    If that is the case, then people in this country, who would be mad as hell, and finally, just maybe finally willing to hit the streets in defense of their stolen democracy, would again have cold water thrown on them by apologists such as this diary writer for stolen elections.

    The propaganda pieces justifying a stolen election are already coming forth, and from a "Democratic activist" web site 2 days before the election.

    I am shocked and appalled.

    "The only thing we have to fear, is fear itself."-FDR

    by Michigan Paul on Sat Nov 04, 2006 at 12:32:55 PM PST

    •  Kos didn't fp it (1+ / 0-)
      Recommended by:
      Caldonia

      I did. The analysis is well done and useful, as reflected by many of the posters within this diary, including a few who disagree with lizzie.

      Take the chip off your shoulder.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Nov 04, 2006 at 12:37:18 PM PST

      [ Parent ]

      •  oh really (4+ / 3-)
        Recommended by:
        zentiger, bdevil89, StupidAsshole, BigBite
        Hidden by:
        highacidity, dmsdbo, dnta

        So you front-paged it, the last time I checked this Web site is caleed "Daily Kos", and it is front-paged on his web site.  I just assume that a piece being front-paged is done so with his authorization.

        Maybe then, if not, we can start calling this site, "Daily DemFromCt".  By the way, I do have a chip on my shoulder, and that is I resent having Republican companies with propriety software in charge of counting our votes, and Republicans co-incidentally coming out just magically winning by 1 or 2% of the vote.  I resent Republican Secretary of States deliberately shortchaning minority Democratic urban precints of voting machines, making them stand for 10 hours to vote in the rain.  

        I resent apologists for exit poll discrepancies from being front-paged on a Democratic web site 2 days before a critical midterm election where already reports of "vote-flipping" and vote suppression and dirty tricks are flying all over.

        But thats just me, I have a chip on my shoulder.

        "The only thing we have to fear, is fear itself."-FDR

        by Michigan Paul on Sat Nov 04, 2006 at 12:44:59 PM PST

        [ Parent ]

        •  do you have arguments? (1+ / 0-)
          Recommended by:
          NCrefugee

          Some of us are always happy to talk about exit polls, and even vote fraud. Wild ad hominems about various people's motives, not so much.

        •  you're outvoted (see other comments) (1+ / 0-)
          Recommended by:
          NCrefugee

          Sorry. But as for how the site works, when Front Pagers (georgia10, mcjoan, DarkSyde, etc) are on duty, they/we can elevate diaries based on their/our judgment.

          The group of retired FP posters, including myself, Meteor Blades, etc who are helping out at election time have the same privilege. You can have an opinion, but it doesn't mean squat to me or how the site functions.

          In any case, you can help yourself by understanding the difference between exit poll discrepancies and vote suppression by inadequate machines, long lines, etc  That was the real shame in Ohio (and elsewhere) but has nothing to do with exit polls. Concentrating on the exit polls after the election allowed too little attention to fixing the  other issues you outline (which were, alas, real). This diary should help folks to understand that.

          Besides bitch, what have you done to address the problem? Are you volunteering to be a poll watcher?

          "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

          by Greg Dworkin on Sat Nov 04, 2006 at 01:04:09 PM PST

          [ Parent ]

          •  I gave Professor Freeman's study (0+ / 0-)

            In case you did not see it yet, I gave a link to Professor Freeman's article about exit polls earlier.
            I am well aware of the difference between voter suppression and exit poll disrepancies.  The article by Professor Freeman discusses the exit poll discrepancies.  Here again is the link, I put it in HTML format because although the PDF format was more neat, I'm not sure everyone's browser supports PDF files:

            http://216.239.51.104/...

            "The only thing we have to fear, is fear itself."-FDR

            by Michigan Paul on Sat Nov 04, 2006 at 01:09:50 PM PST

            [ Parent ]

            •  all that article says (0+ / 0-)

              is that there is an unexplained exit poll discrepancy. So? I think we've learned a thing or two since late 2004 -- although of course Freeman disagrees with a lot of us about what we've learned.

            •  Steve Freeman (0+ / 0-)

              who is neither a statistician nor a pollster, wrote a great deal of things about the exit polls. I am familiar with his online work, though i did not buy his book.

              His views have changed over time. Did you know that?

              For my part, I have never treated exit polls as an absolute measure for predicting election results. Moreover, no one I know has said that the discrepancy itself indicates that Kerry must have really won the election. Rather, the evidence that cast doubts on the election results come from diverse sources. The exit polls have never been cited as primary evidence of fraud, but only as a reason to take that primary evidence to heart.

              Professor Steven F. Freeman

              "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

              by Greg Dworkin on Sat Nov 04, 2006 at 01:18:11 PM PST

              [ Parent ]

              •  Well, so? (2+ / 0-)
                Recommended by:
                bdevil89, Flinch

                He says that the exit polls are "a reaso to take that primary evidence to heart."  He also says, in your quote, "the evidence that casts doubt on the election results come from diverse sources."

                I never once said that the only reason to claim the election was stolen was exit polls.  I pointed to voter suppression also, and you made a point in another post to me about the difference.

                Professor Freeman was saying that there was an unexplained exit poll discrepancy.  What I find irritating is that so many in here, after all we have been through, are willing to dismiss this discrepancy to benign reasons.  Based on the Republican's actions on other issues-lying us into war, covering up energy policy discussions, lying about scientific studies, etc., I take the other route, and tend to side with those who see some sort of evil design in their motives.

                I predict Election night will see a great revival in this debate.  I am sorry I was a little short in my initial irritation with this diary.  You have a right, if you are in charge of the ship, to front page what you deem noteworthy. But I also have a right to dispute it.

                "The only thing we have to fear, is fear itself."-FDR

                by Michigan Paul on Sat Nov 04, 2006 at 01:35:33 PM PST

                [ Parent ]

                •  well, if you want to set up straw men (0+ / 0-)

                  to battle with, you can do that all day. The snippet I quoted suggests that Freeman finally came around to recognizing that the great exit poll discrepancy needs to be put in the proper perspective.

                  This is a diary about the exit poll issue, written by someone who has studied the question in great depth, worked with the people who do them, and has published some of her findings. Definitely worth elevating.

                  Sure, you can complain. Keep the complaints fair and focused.

                  "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

                  by Greg Dworkin on Sat Nov 04, 2006 at 01:45:36 PM PST

                  [ Parent ]

          •  I am volunteering to pollwatch (2+ / 0-)
            Recommended by:
            DemFromCT, StupidAsshole
            In Yonkers NY  where the Republicans have instituted a purge complete with threatened police visits to voters homes which was reported in the nytimes and the subject of an editorial there this past week.

            Vote suppression does have everything to do with exit polls.  It's not just voters who are prevented from voting.  It's voters who are given placebo provisional ballots which are never counted, voters given faulty equipment that undervotes or overvotes.  It's voters who cast a vote and have faith that it will be counted - when it fact it is not.

            New Mexico had a lot of these voters - the native Americans who drove miles to the polls in 04, only to vote for the down-ballot races and not for president!

            •  ah, Yonkers (0+ / 0-)

              I know it well, from the Sprain Parkway to Saunders High to the Cross County mall. Yonkers mayor John Spencer is an awful candidate this year for NY Senate.

              Good of you to poll watch, but what you mention actually has nothing to do with exit polls (but it sure has everything to do with voter suppression and election integrity)

              "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

              by Greg Dworkin on Sat Nov 04, 2006 at 06:29:59 PM PST

              [ Parent ]

    •  more about exit polls (0+ / 0-)

      Here's a good web site which gives one a little different interpretation from the diarist, who apparently is held up to be some sort of expert from so many of the comments in here.  Does she have a degree in statistics?  What are her/his qualifications.  Sure, if you sound technical enough, you sound like you know what you're talking about, but I personally feel she is way out of line.

      http://www.exitpollz.org/

      "The only thing we have to fear, is fear itself."-FDR

      by Michigan Paul on Sat Nov 04, 2006 at 12:39:38 PM PST

      [ Parent ]

      •  If you feel she is "way out of line"... (3+ / 0-)
        Recommended by:
        matt2525, Caldonia, dnta

        ... instead of posting a link to a website, and instead of saying "does she have a degree in statistics?" maybe you could enlighten us with your vast and specialised knowlege.  I mean, I assume you must have a degree in statistics, right?  And you're just too lazy to explain why Febble is "way out of line", right?


        They can be counted on to tell us who our enemies are / But they're never the ones to fight or to die... -- Jackson Browne

        by Page van der Linden on Sat Nov 04, 2006 at 12:43:33 PM PST

        [ Parent ]

        •  No I don't (0+ / 0-)

          I don't have a degree in statistics.  But I have read a hell of a lot from people who do, such as Professor Freeman from the University of Pennsylvania.  Anyone who wants to read his studies, just Google his name and you will find them.

          I don't have the time to debate mathmatical probabilities 2 days before the election.  I don't dispute her right to her opinion, I dispute why it is being frontpaged, ahead of all other opinions, 2 days before the midterms.

          And I gave a Web site with great links to all sorts of rebuttals to this if anyone is so inclined.  And no, I am not going to list point by point everything they have already said because I don't want to get carpal tunnel syndrome and the information is there on the link I provided if anyone is so inclined.

          "The only thing we have to fear, is fear itself."-FDR

          by Michigan Paul on Sat Nov 04, 2006 at 12:49:21 PM PST

          [ Parent ]

          •  Professor Freeman's article on exit polls (1+ / 0-)
            Recommended by:
            BigBite

            Here is Professor Freeman's article :

            http://216.239.51.104/...

            "The only thing we have to fear, is fear itself."-FDR

            by Michigan Paul on Sat Nov 04, 2006 at 12:52:29 PM PST

            [ Parent ]

          •  Freeman has no statistics degree (0+ / 0-)

            Most of his work is business case studies. Happy to debate on the merits. But you don't have the time to debate. So WTF are you doing? Sigh.

            •  Freeman has Ph.D. from MIT Sloan School (2+ / 0-)
              Recommended by:
              StupidAsshole, BigBite
              of Management.  He is on the faculty of the Univ of Penn's Center for Organizational Dynamics where he teaches research methods and survey design (a domain that includes polling).  

              Here are some of the endorsements his book has received:

              Guided in no small part by Steve Freeman's original analysis of exit polls nationwide, I began my own investigation of the official results... What we found indicated problems in nearly every sphere of the electoral process ...

              Unfortunately, the lesson of our past two presidential elections has become patently clear--poor decisions by election officials, whether motivated by political bias or stunning negligence, can result in the disenfranchisement of voters and the massive distortion of election results....

              Freeman and Bleifuss shape the raw data into an image of all that the Founders warned us against."
              --Congressman John Conyers, Jr.  (from the foreword)
              Ranking member of the US House Judiciary Committee
              --------------------

               "After Steven Freeman first pointed to the statistical improbability of the discrepancy between 2004 Election Day exit polls, which forecast a Kerry victory, and the officially reported results, opinion leaders accepted with relief the mea culpa offered months later by exit pollsters Joe Lenski and Warren Mitofsky.  The careful analyses presented in this book demonstrate that the pollsters' explanation is utterly unsatisfactory.  Indeed, the additional evidence that Freeman and Bleifuss develop is even more disquieting than the original discrepancy.   Their book deserves to stimulate follow-up investigations into the threat to our democracy posed by insecure electronic voting machines, and into the possibility that their vulnerability was exploited in 2004 with fateful results.  As a citizen, I very much hope that the answer is `no,' but it is time for mainstream scholars, journalists, and public officials to stop avoiding the question."

              -Jack H. Nagel
              Steven Goldstone Endowed Term Professor of Political Science, University of Pennsylvania

              ---------------------

              "Freeman and Bleifuss are true patriots. They understand that our country cannot survive as a viable democracy if our election processes are corrupted. They responsibly and insightfully investigate evidence suggesting that the 2004 presidential election may have been stolen, using as a focal research question: either the exit polls were unusually way off or the votes were not accurately counted. Concerned Americans should not ignore this intelligent book."

              -Kenneth Warren,
              Professor of Political Science, St. Louis University and President of The Warren Poll

              ---------------------

              Was the 2004 Presidential Election Stolen? discusses a contentious, but not a partisan issue. People differ strongly about whom they want in the White House, but almost everybody wants whoever is there to be seen as having been rightfully elected. Steve Freeman and Joel Bleifuss explore at length several possible explanations for the discrepancy between the exit polls and the final counts during the 2004 election. Only those who simply and reflexively assert the explanation with which they're most comfortable will dismiss this careful and judicious book as the work of conspiracy theorists.
              John Allen Paulos,
              Professor of mathematics, Temple University

              "In the aftermath of the 2004 election, I was convinced that the exit polls had got it wrong, that George Bush had won the popular and electoral vote, and that arguments to the contrary were little more than a combination of sour grapes and conspiracy theory. After reading this book, I'm no longer so sure. When asked `was the 2004 election stolen?' the only honest answer I can now give is `I don't know.'  That is a sad commentary on the state of elections, election polling, the news media and academic research in the United States and it needs to be remedied."

              Michael X. Delli Carpini
              Dean, The Annenberg School for Communication, University of Pennsylvania

              •  yes, Freeman has a Ph.D. in organization studies (0+ / 0-)

                Most of his research is qualitative, which is fine, but surely it is misleading to refer to him as having a degree in statistics (which was not your formulation).

                That's great press copy, but for better or for worse it isn't representative. Freeman might have had more impact if he had junked the exit poll arguments entirely, which admittedly would have been a bold move.

          •  You dodge questions almost as well as Bush! (0+ / 0-)

            And no, I don't know Febble's educational background.

            As for the "don't have the time" thing, maybe you could post less here, do the analysis, and then return with your definitive, bookmark-worthy findings.  There's a thought.


            They can be counted on to tell us who our enemies are / But they're never the ones to fight or to die... -- Jackson Browne

            by Page van der Linden on Sat Nov 04, 2006 at 01:09:29 PM PST

            [ Parent ]

        •  Well, does she (2+ / 0-)
          Recommended by:
          Michigan Paul, StupidAsshole

          have education in the subject or not?

          I recall many people on this website dismissing various analyses by people in the hard sciences about the election because they weren't professional pollsters.

          And if I remember correctly, Febble does not have education in this area either--I think I posted a comment about the double standard--how people here were quite taken with her analysis while challenging other non-pollsters' analyses on the basis that the non-pollsters did not have the appropriate background.

          I really don't care what her background is--her analysis is interesting.  But if it is true that she doesn't have a pollster's background, IMO, then I don't think her work should be given any more [or less] weight than any other non-pollsters' [in the hard sciences] analyses.

          Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

          by dfarrah on Sat Nov 04, 2006 at 01:00:17 PM PST

          [ Parent ]

          •  shrug (0+ / 0-)

            Febble's training in psychology is more on point than many people's training in math or physics. As a political scientist, I observe that she doesn't make strange initial assumptions that some other analysts seem inclined to make. But whatever -- Febble has never asked anyone to kneel and kiss her credential ring.

          •  Since you ask (1+ / 0-)
            Recommended by:
            matt2525

            which is fair enough, I am a psychologist, so my training is in social sciences, which includes survey research, although my speciality is cognitive psychology.  However, my background is in all kinds of things, including music and architecture, and writing for children.  If you google me, I am most of the people that come up.

            However, that is irrelevant to my arguments. If they are flawed, it doesn't matter how many degrees I have.

            •  Thanks--believe me, (0+ / 0-)

              I don't want you to kneel or kiss rings.

              I just recall that some posters here were quite hostile toward non-pollsters' science [in particular, the ones that reached different conclusions from you].

              It was just the different standards that bugged me.  [Cognitive psy was one of my favorite psy courses]

              Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

              by dfarrah on Sat Nov 04, 2006 at 01:20:04 PM PST

              [ Parent ]

        •  She is out of line about the "uncorrected" results (2+ / 0-)
          Recommended by:
          StupidAsshole, BigBite
          The diarist is implying that it is wrong to say that "uncorrected" poll results won't be released in 2006.

          It is not wrong to say this.

          In her explanation she confuses, perhaps deliberately, incomplete exit poll results, ie results from early in the day, with what the pollsters call "uncorrected" results and what Freeman prefers to call "unadjusted" results.

          At the end of polling, the NEP pollsters adjust their polls to conform more closely to the official count, acting on the presumption that the official count is correct.  

          In 2004, unadjusted results from earlier in the evening were preserved on CNN screenshots which are reproduced in Freeman's book WAS THE 2004 PRESIDENTIAL ELECTION STOLEN?

          In 2006 we are never going to get a chance to see these unadjusted results.

          •  evidence? (0+ / 0-)

            As far as I can tell from the public record, all we know will be different is that they will try to prevent leaks before 5 PM Eastern. That is a far cry from saying that no results will be available until they have been conformed to the official count.

            The diary states, "Whether the networks will post close-of-poll cross-tabulations remains to be seen, but I have seen nothing to indicate that they won't." You accuse her of confusing something, "perhaps deliberately," but I can't tell whether you read what she wrote.

          •  Of course I'm not confusing (0+ / 0-)

            anything "deliberately" but I am starting to wonder whether you are deliberately confusing yourself. The NEP cannot adjust their cross-tabulations at the end of polling, because at the end of polling there are no official results in.  That's why I explicitly said that if you want the unadjusted results, you need to watch what is posted at close of poll, before any adjustments can be made.  Later cross-tabulations will be adjusted, if they results diverge from the poll.

            I don't know if the close of poll crosstabs will be posted, but I have read nothing to the contrary.  As I said in my diary.

      •  What part of my diary do you find (0+ / 0-)

        "way out of line"?

        I do not argue on the basis of credentials. If you don't want to take my arguments and information at face value (which would be very wise) then check them out. The great thing about the internet is you actually have to evaluate the content.  You cannot rely on credentials.  It is the ideal form of blind peer-review.

        •  out of line (2+ / 0-)

          What I found irritating in your analysis is that you gave some good points about exit polls, and they were intersting and informative to a degree, I agree with the other posters on that.  But you then went out on a limb and issued a point by point attack on those who argue otherwise from you-that exit polls are reliable indicators of the actual vote, that exit polling discrepancies are red flags for voter fraud, etc.  

          You seemed to cloak your assertions in a degree of certainty that was not warranted by the facts.  As I have pointed out, many others with similar levels of expertise have come to different conclusions.

          What I did not like is that yours were frontpaged, while others were not who felt differently.  You have all the right in the world to get on your soap box and lecture us all on exit polls, but I still would ask just what is your qualification to do this?

          Notice that I did not write a diary on this subject, frankly I don't have the expertise to do so and I defer to others with more knowledge than I, such as Professor Freeman.  But so much has been coming out lately about the possibility for hacking of these electronic voting machines, as well as the common knowledge that private companies with well-known Republican leanings are running companeis such as Diebold, that I have a gut-level suspicion of them.

          The election process is non-transparent and ripe for fraud on a massive level.  I feel that you would have served us better informing us of those issues, rather than launching a criticism of those of us who are on your side in this political battle to take back our country.

          And thank you people for not troll-rating me for a difference of opinion. That has happened so many times in the past.

          "The only thing we have to fear, is fear itself."-FDR

          by Michigan Paul on Sat Nov 04, 2006 at 01:18:40 PM PST

          [ Parent ]

          •  I did not attack anyone (1+ / 0-)
            Recommended by:
            dnta

            please link to any posts of mine that you consider an attack.

            And also any posts that you consider indicate "a degree of certainty that was not warranted by the facts".

            Yes, others have come to different conclusions.  I find Professor Freeman's analyses very flawed.  And if you'd like to know why, I'd be happy to explain.

            As for my qualification - my qualifications are irrelevant and I claim none, except that, as I said in my first post, I need to make it clear that I did actually work on the exit poll data for Warren Mitofsky.  I do not make that statement to give my diary credibility but simply to warn that readers might like to bear that in mind when assessing my objectivity.  I was declaring an interest.

            But FWIW, I am a competent data analyst, with training in the social sciences.

            •  I would like to know (0+ / 0-)

              I would be interested, what do you find flawed about Professor Freeman's assertions.  Being a layman, and not having read his article completely since right after it came out after the 2004 election, I sum up his argument that the discrepancies between the exit polls and the actual results are impossible and unexplained due to mathmatical probablilites?

              What do you find is "flawed" in that?  By the way, he also pointed to the German exit polls in his study, which he showed have always predicted the winner of the German elections with extremely high levels of accuracy, up to only about 0.036 differentials.

              "The only thing we have to fear, is fear itself."-FDR

              by Michigan Paul on Sat Nov 04, 2006 at 01:41:20 PM PST

              [ Parent ]

              •  Well, to be fair on Freeman (0+ / 0-)

                that particular article is quite old, and he has moved on from some of it.  He was quite correct that the discrepancy at the time was both "significant" and "unexplained", although there were some problems with his analysis, critiqued by Rick Brady here:

                http://stonescryout.org/...

                But Freeman has since written a book, together with Joel Bleifuss, in which he makes several arguments for the case that the correlations within the data are suggestive of fraud.

                I don't agree, but rather than go through them all now, I will refer you to this paper

                http://inside.bard.edu/...

                by my colleague Mark Lindeman, who addresses all Freeman's points in detail.  But if you haven't read the book, there is probably no point in giving you what I consider are refutations here.

                If you have, I'm happy to take them one by one.

                •  you're right (0+ / 0-)

                  I haven't written the book, but thank you for referring me to the sources you indicate.  Yes, that is what I thought, that the discrepancies in the exit polls were indicative of fraud.  That is why I was a little perturbed when I read your diary.

                  Btw, you are right, you did not attack anyone.  You just sort of dismissed them by referring to their arguments as "myths".  thanks for the links, I always like it in here because we have so many knowledgable people.

                  "The only thing we have to fear, is fear itself."-FDR

                  by Michigan Paul on Sat Nov 04, 2006 at 02:20:03 PM PST

                  [ Parent ]

                •  The differences in Freeman's conclusions (0+ / 0-)
                  were the result of his access at different times to different data.

                  His first online writings were written in Nov and Dec 04 using the data then available - including screenshots from CNN's election eve coverage that were inadvertently and luckily saved, before NEP changed their data to conform more closely to the official count.

                  He wrote his book using the data made available in the Evaluation of Edison-Mitofsky report released publicly in Jan 2005, which I gather Febble worked on.  So of course Febble and Freeman disagree, as Freeman uses the data to raise questions both about the conclusions drawn by the Evaluation and the official count.

                  Edison-Mitofsky did not release the precinct level data however, and will not release it.  So Febble has an unfair 'advantage' if she has seen data not made available to Freeman.

                  Full disclosure -I am Freeman's literary agent, so I am deeply familiar with the work that went into his book.

                  I do find it curious that Edison-Mitofsky and Febble both defend the NEP methodology at the same time that they try to explain away the discrepancy the NEP exit polls documented as some kind of error.

                  •  I find it extremely curious (0+ / 0-)

                    that you can so misrepresent my diary, for a second time.  

                    This is not some competition.  I do not have an "unfair" advantage, and clearly I cannot make data available to Freeman that is not mine.  I do not "defend the NEP methodology" - I explain both it, and its limitations.  If you are not interested in the information, you are free to ignore it.  But to try to paint it - for the second time - as some kind of propaganda exercise is absurd, offensive and baseless.

                    But more to the point: I have the access to the same data as Steve has access to, and yet I disagree with his conclusions from those data. I disagree, not because I know something that he does, but because I think his statistical arguments are flawed.  If you are familiar with his book, then tell me which of his arguments you find convincing, and I will tell you why I don't.

                  •  Oh, just noticed yet another (0+ / 0-)

                    misrepresentation.

                    No, I did not work on the January Edison-Mitofsky report.  "I gather" you are a journalist.  Did it not occur to you, as a journalist, that this might actually be worth checking?

                    You could have ascertained the answer extremely easily.  Indeed you could simply have clicked on my DKos handle and checked out my diaries, from which you would have discovered that I was actually critical of the E-M report, as I considered the measure of the precinct level discrepancy they had used was flawed - and I called for a re-analysis of the data.

                    As it turned out, I was actually contracted to run that reanalysis myself, for Warren Mitofsky.

                  •  really, that's not gathering, that's invention (0+ / 0-)

                    It's hard for me to understand how you could work closely with Steve Freeman over a period of time, and have the impression that Febble was working for Mitofsky as early as January 2005. Now we really are in tin foil hat territory, at least those of us who have any familiarity with the chronology.

                    I don't know whether you are a journalist, and I don't care. If you're going to make allegations about people, you ought to be prepared to back them up.

    •  Great! (3+ / 0-)
      Recommended by:
      DemFromCT, Plutonium Page, Caldonia

      I'm glad to see you refute all that is said here with actual arguments.

      I do not have my own blog.

      by Frank on Sat Nov 04, 2006 at 12:40:02 PM PST

      [ Parent ]

    •  um.... (0+ / 0-)

      Well, it's my diary, and I don't recognise that "gist".  Dammit, the only reason I got involved in this stuff was that I thought the election had been stolen.

      http://www.dailykos.com/...

      I wear my tinfoil with pride.  

      •  If you can help me clarify your arguments... (0+ / 0-)

        please do.  It appears from reading your well thought out diary and comments,that you are just trying to point out some reasons why exit polls may be flawed, but you are not trying to say that everything that ocurred in the last election was completely kosher.

        I am not a social scientist myself, but as you said in an earlier comment, proof is for math and alcohol.  I would imagine that as a social scientist, you observe a wide range of behaviors, and then try to use the collected data to form some sort of genral conclusions.  For example, I've heard studies that say people exposed to abuse as a child are more prone to commit abuse themselves as an adult.  From this information, one can show a trend, but one cannot say with any level of absolute certainty that a certain individual will commit abuse simply because they were abused themselves.  They may be prone to it, but they could grow up to be Mother Teresa as well.

        By using this methodolgy on voters and exit polls,you could conclude that a person's psychological makeup may predispose them to one political party or another, and this same psychological pattern may make them more or less apt to answer pollsters questions. This data alone is not enough to prove or disprove fraud, especially in races where preelection polls showed a pretty close race to begin with. But with enough data and polling we can at least see some interesting trends.

        However, if one candidate led 60-40 in preelection polls and exit polls, and then lost 40-60, we would know something is up if results are that far from the margin of error.

        This is what I'm getting from your arguments, please correct me if I'm wrong because I find this stuff fawscinating.

        The meek shall inherit nothing. -F.Zappa

        by cometman on Sat Nov 04, 2006 at 01:48:09 PM PST

        [ Parent ]

        •  I certainly don't think that everything (1+ / 0-)
          Recommended by:
          Eddie Haskell

          in the 2004 election was kosher, and in fact posted a couple of papers myself on things that patently weren't.

          But I should clarify that nothing in my conclusions involved psychology (even though I'm a psychologist).  I don't know why Republicans might be less willing to co-operate with exit pollsters, although I could hazard a guess.  But this stuff has nothing to do with guessing.  The hypothesis was simply: was the discrepancy greater were factors likely to allow any underlying differential willingness to be translated into bias, present.  And the answer was yes.  It could have been no.  I might have found that the bias was actually bluer where interviewing interval was longer, in which case I would have been forced to conclude that if anything, it looked as though it was Kerry voters that were more reluctant.  Which would certainly have made the overall redshift very suspicious.

          But in fact I didn't. Redshift was greater where interviewing interval was longer, which allows the inference that Bush voters were less willing to participate, and were therefore more likely to avoid selection in conditions where avoiding selection was more likely to be possible.  And it also accounts for some of the redshift.

          If the exit polls were, in some future election, to be as far out as in your example, then certainly it would be fair enough to raise a red flag.  But again, it would be difficult to rule out bias in the poll so you'd want to have strong evidence from elsewhere.  And if you wanted to investigate the possibility of bias in the poll, you'd do the kinds of analysis I did, which is to find out what correlated with magnitude of the discrepancy.  And if variables likely to be associated with fraud were correlated with the discrepancy you might have a case that fraud had occurred. I certainly wouldn't rule this out - and indeed I tested just this kind of hypothesis on the 2004 data.  But what I did find, as E-M reported in their original evaluation, was strong evidence for bias in the poll.  There are likely to be methodological improvements this time to try to prevent selection bias being as big a factor.  But what the data also indicate is that the tendency for Democrats to be more willing to participate in exit polls than Republicans appears to be real.  So it must always remain possible that non-response bias favoring Democrats will occur, perhaps especially given the debate about the 2004 polls.

  •  Question concerning absentee (1+ / 0-)
    Recommended by:
    frenchman

    Some states are going vote by mail.
    OR has already and WA is at 70+ %.

    New this year (afaik) each county releases (every couple of days in large counties) voter id numbers of those whose absentee ballot has been received. By comparing with the voter role last released prior to ballot mail out, it seems that absentee voters could be sampled in fairly large numbers and confidence by phone. I am using the data to narrow the scope of my final GOTV walks to those who have not mailed their ballot yet and poll voters.

    Of course this may aggravate some of the other factors that throw off exit polls... I suspect lots of people are gonna be shocked that you would know that their ballot was received and may react poorly to survey questions.

    Is there any type of polling being done that mimic the results of exit polls in states where mail in is the primary method of voting?

    The biggest threat to America is not communism, it's moving America toward a fascist theocracy... -- Frank Zappa

    by NCrefugee on Sat Nov 04, 2006 at 12:33:33 PM PST

    •  Interesting note (maybe) (0+ / 0-)

      In my precinct, those who have self identified at least once as repub in the past, sent in their absentee ballots very early. This precinct is 78% Dem and about 80% permanent absentee (vote by mail).

      These are people I have talked to and reported their affiliation in the past. I have not run into any that are admitting to switching parties yet.

      I suspect that many Dems are waiting for election day to drop in the box at the polling place.

      The biggest threat to America is not communism, it's moving America toward a fascist theocracy... -- Frank Zappa

      by NCrefugee on Sat Nov 04, 2006 at 01:03:37 PM PST

      [ Parent ]

    •  Oregon is a phone poll, and (0+ / 0-)

      there are supplementary phone polls in other states with large proportions of absentee/early voters. I'm not sure off-hand whether they work from voter lists, or just call lots of people and try to winnow the early voters.

    •  CA going as high as 50% absentee (1+ / 0-)
      Recommended by:
      StupidAsshole
      I heard today.

      It's a dilemma.  Obviously most people don't trust the voting machines. And it's partisan.  According to a Pew Research [report http://people-press.org/... only 45% of Democrats - vs. 79% of Republicans - are very confident that their votes will be counted accurately.

      And only 30% of Black voters are very confident (NYTimes did a front page article on this that seemed to blame the perception of black voters instead of the documented fact that their votes are often not counted, which the article did get around to discussing at the end.

      Re absentee ballots - many have raised concerns about their custody and how they are handled and counted (can be declared ineligible without the voters knowledge in many places)

      Kossacks, we have a problem here.

  •  and a question about margin of error (0+ / 0-)

    Interpreters of polls on this and many sites are accustomed to saying that a pre-election poll result that places the difference between two candidates' support within the margin of error is "not statistically significant."

    My understanding is that this is wrong. The margin of error establishes a range for each candidate's result, and ordinarily the actual state of affairs in the electorate as a whole is to be found within that range 95% of the time. Nonetheless - let's say we have a MoE of 3%, and one candidate gets 50% and the other 49%, with only 1% undecided.

    Is it not the case that while a reversed order of finish - Ms. 50% ending up with 49, and Mr. 49% winning - is quite possible, the original 50-49 result still indicated that there were at the time of the sampling MORE chances that Ms. 50% was at that moment ahead?

    In other words, one may be encouraged to some extent by a winning sampling result of 1%, even though with a 3% margin of error, a lead of 3% or greater would be necessary to be certain, within the confidence interval, that the electorate preferred the poll winner at that moment in time.

    We all realize that bias in polls stems from a myriad of factors, but I'm inclined to think that even a small lead in a poll is "statistically significant" to SOME degree. Am I wrong?

    -10.00,-10.00. Beat that, motherfuckers.

    by frenchman on Sat Nov 04, 2006 at 12:34:29 PM PST

    •  Well, a margin of error doesn't have distinct (1+ / 0-)
      Recommended by:
      frenchman

      boundaries.  Where you set them is entirely arbitrary.  The closer the margin, the more likely it is to be the wrong way round.  But it's a sliding scale.

      The other point perhaps worth making is that three polls all within their margin of error, but all in the same direction are quite likely to mean that one particular candidate is ahead.  

      I was thinking that about the recent Corker-Ford polls in Tennessee.  The first one seemed like an outlier - but the next few were in the same direction.  So, even though Tennesse is close, I'm not optimistic about Ford.

    •  the phrase to hate is "statistical tie" (0+ / 0-)

      Assuming (not that we really should assume it) that a poll is unbiased, you are absolutely right that a 50-49 lead offers better prospects than (say) a 48-51 deficit. It's unhelpful to call either one a "statistical tie" or "statistical dead heat." (I suppose one might call it a "statistical photo finish," if one needed a strained metaphor!)

      But it's not wrong to say that these differences are "not statistically significant." One just has to understand that the phrase has a technical meaning, which is not that the difference conveys no information whatsoever.

  •  Great piece (0+ / 0-)

    ..explaining the pitfalls of exit polling. I missed seing your analysis here, but this sure is a good time for you to resurface.

    I do not have my own blog.

    by Frank on Sat Nov 04, 2006 at 12:41:34 PM PST

  •  ExitPolls SchmexitPolls (2+ / 0-)

    So, now we know as little as before.

    Seriously, nice article, but I think we need to be informed more widely.

    I can recommend (while you do the dishes!) to listen to  a talk  in which Mark Crispin Miller explains what needs to be done:

    1. hand counted paper ballots
    1. uniform standard for voting procedures
    1. instant run-off voting
    1. make it illegal to be a secretary of state and a co-chair of a political campaign
    1. total ban on the participation of private vendors in our electoral system

    small mp3 download... for the link see:

    http://u2r2h.blogspot.com/...

    (vote on the outcome of the election there, too)

    •  This is sick. (1+ / 0-)
      Recommended by:
      StupidAsshole

      Not your comment. The diary. This is a discussion for specialists. A citizen shouldn't have to be a rocket scientist to understand how the election system works.

      This diary is a symptom of the sickness of our nation.

      We could find thousands of high school classes that could design a transparent, verifiable election system, and with such a system, this discussion would not be taking place (except perhaps in a different form among specialists.)

      ------

      Call your county's election office and be SURE you are recognized as registered to vote. Don't assume anything.

      by We hold these truths on Sat Nov 04, 2006 at 07:16:12 PM PST

      [ Parent ]

  •  I can't buy it. (1+ / 0-)
    Recommended by:
    StupidAsshole

    I just heard Robert Kennedy make exactly the opposite case -- that 2004 exit polls were accurate -- on Air America radio.  

    Right or wrong, I trust him.

  •  Great post (0+ / 0-)

    At least you have added some sanity to address the knee-jerk reactions of the voting machine conspiracists, who seem to blame any and all Democratic losses that occur on Diebold.

    http://www.keen.com/jiacinto For DC related travel advice, please visit that link.

    by jiacinto on Sat Nov 04, 2006 at 03:40:12 PM PST

    •  no need to blame Diebold (0+ / 0-)

      it's my brother-in-law's fault.

      Whatever it is.

      jiacinto, even you must be smiling. ;-P

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Nov 04, 2006 at 04:26:20 PM PST

      [ Parent ]

  •  Discrepancy between exits & election=BOTH suspect (1+ / 0-)
    Recommended by:
    StupidAsshole

    Discrepancy between exits & election=BOTH suspect

    The title of the post pretty much states the scientific view of the matter.  Nevertheless, while the original post never specifically claims contrary to this, it implicitly does so by categorizing only the potentially explanatory exit poll errors (its claimed scope, of course) but negligently omits that it is just as possible that the error is from the election results.   While we can't conclude the election is off just from the exits (necessarily) we CAN CONCLUDE that a material discrepancy between the election results and the exit poll results merits an investigation of BOTH.

    Failure to make the above point makes this post, while informative, materially misleading to reasonable reader.  We need not raise the question of the author's intent, to say that the reasonable reader may well be misled into attributing poll and election discrepancies solely to the poll based on the above post.   We need to investigate both elections and exit polls if there is a material discrepancy!

    Just because it "coulda been" nonsampling error doesn't mean it IS.  After all, the average internet user "stuffs the ballot box" in an online nonscientific poll in order to see their horse "win" that poll.  While that's not illegal per se, if we up the stakes in the poll or election to control of the world's richest country and sole superpower GEE DO YA THINK there might be some folks with an incentive to cheat??  In fact, there's so much at stake in elections that anyone that dismisses off hand anyone doubting the veracity of election results presently counted in secrecy on electronic machines as "conspiracists" is either not thinking, or hopelessly naive about what's at stake.

    •  You say: (0+ / 0-)

      "Just because it "coulda been" nonsampling error doesn't mean it IS. "

      Quite.  Which is why the data were subjected to exhaustive analysis, first by E-M, then, later, by me. In order to conclude that non-sampling error was a factor, I had to actually run a load of analyses with specific hypotheses.  And my conclusion was that the evidence strongly supported not non-response bias, but selection bias.

      There was no intent to mislead, and I did not mislead.  It is simply not true that there is no evidence that non-sampling error accounted for the exit poll discrepancy.  There is strong evidence.  And in contrast, the hypothesis of fraud was contra-indicated by the finding that there was absolutely no correlation between redshift and swing to Bush.  None.  The regression line was actually slightly, but insignificantly negative.  You have to make heroic assumptions about the nature of the fraud in order to make the fraud hypothesis consistent with that find.

      I'm not interested in "could'a beens".  I'm interested in what the data say.  But because I know (now) the degree to which non-sampling error is present in exit poll data, I am extremely dubious about the wisdom of conducting what is an extremely expensive exercise with the sole purpose of monitoring election integrity.  I think exit polls are a very poor tool for the job.  And precisely because there is so much incentive to cheat, I'd like you guys to get yourself a hell of a better tool.

      •  Do an election integrity exit poll then (0+ / 0-)
        and not a media exit poll.  They are designed differently.  Election Integrity have posted instructions on their website www.electionintegrity.org and will release their methodology.
      •  Selection bias = blame the interviewer (0+ / 0-)
        I'll explain another of Freeman's arguments.  He acknowledges that the Evaluation of Edison-Mitofsky also cites selection bias as a reason for exit poll discrepancies.

        In other words, the evaluation finds that the discrepancy is higher when
        interviewers are more than 25 ft away from the polling place
         (remember even now Blackwell and others are trying to keep pollsters 100 ft or more away from the polls)
        interviewers are younger
        interviewers have advanced degrees
        interviewers in larger precincts

        However - a lower discrepancy is only a good thing if the count itself is accurate.  So, for instance, it could be that the interviewers with advanced degrees actually did a better job of polling and found more evidence of a corrupted count (which, again, would include provisional ballots not counted, undervotes and overvotes voters not aware of, in addition to any possible election fraud)

        •  No, I don't think blaming the inteviewer (0+ / 0-)

          is sensible at all.  If you want to blame anything, I'd suggest blaming a methodology where an emphasis was laid on sample size (if sample sizes were smaller than expected at first call, the interviewing rate was adjusted).  It was clearly difficult to get good sample sizes in 2004, and the mean was only 80 (I think the target was 100) and in some precincts the sample size was very much smaller.  So it would be quite inappropriate to blame the interviewers, and the Mitofsky document takes pains not to do so.

  •  Beware: there's a subtle message here (3+ / 0-)
    Recommended by:
    Xeno of Elia, StupidAsshole, BigBite

    Much of this posting is straight Statistics and polling. But some is obfuscation. Edison/Mitofsky refused (and continues to refuse) to release precinct-level raw results that would have allowed independent assessments of their claim that Republicans were less likely to respond. Several qualified analysts, looking at the data that were available declared their claim to be so much BS.

    All around the world, exit polls are used as a check on the validity of the voting process. That's the method used by independent observers. And exit polls in the US were very good predictors until just recently. There is no good reason to believe that exit polls here cannot be as fair and accurate as they are elsewhere in the world.

    It is true that you can't draw a Simple Random Sample, but you can get an excellent sample by other means.

    The statistically huge deviation between exit polls and votes in several states in the past 2 elections are a new phenomenon and are deeply troubling. As a professional Statistician and teacher, I've pointed out to my students that either the Law of Large Numbers is failing, or someone is cheating. And the LLN is a mathematical fract.

    We must be very cautious and very vigilant next week.

    •  you can't get precinct level data (0+ / 0-)

      because it compromises privacy. There's no conspiracy about that.

      You're right that we need to be vigilant next week, but there's lots of reasons for that which have nothing to do with exit polls.

      As to polls overseas, there are failures and successes. They are not so accurate as portrayed.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Nov 04, 2006 at 07:39:17 PM PST

      [ Parent ]

      •   precinct level data doesn't compromise (2+ / 0-)
        Recommended by:
        Xeno of Elia, StupidAsshole
        privacy.

        that's simply an excuse for not releasing it.

        How, exactly, would precinct level data compromise privacy?  Remember, voter registations are open and political contributions are publicly posted.

        Maybe in a teensy district where, say, one black Republican resides . . . maybe.

        •  true, and even if it did, democracy matters more (0+ / 0-)
        •  Exactly (0+ / 0-)

          "Maybe in a teensy district where, say, one black Republican resides . . . maybe."

          That is exactly the point.  Ethical guidelines regarding confidentiality are extremely strict for precisely this reason.  And if your sympathy doesn't extend to a lone black Republican, how a about a lone black Democratic in a Republican neighbourhood?  Or a lone Hispanic gay Democrat?  Or a lone gay married Democrat?

          There is a vast amount of personal detail in those questionnaires, which is why confidentiality is explicitly assured.  The pollsters provide that confidentiality - it is the only thing the voter gets from the pollster in return for extremely valuable personal information.

      •  There's no compromise to privacy (2+ / 0-)
        Recommended by:
        Xeno of Elia, StupidAsshole

        That's a total red herring. We haven't asked for the detailed demographic poll data (which could, in a small precinct compromise privacy for a very few). We've asked for the raw vote totals. That's what we'd need to assess the "bashful Republican" theory floated by Mitofsky.

        And that's what we need to smoke out vote fraud. The very real risk of fraud makes it essential that we have reasonable protections. The best of those would be paper trails for each voting machine (which we don't have), and random audits (which we can't have because we lack the paper trails.)

        Second best is exit polls at the precinct level. All we want is the percentages for each candidate, and that compromises nobody's privacy.

        •  how would you use raw vote totals (0+ / 0-)

          to "assess the 'bashful Republican' theory floated by Mitofsky"?

          If you ever convinced a complement of public opinion researchers that there was a viable analytical approach not already being pursued, you might conceivably get some traction. But there has been weirdly little apparent attempt to make that case.

          •  By looking for patterns (0+ / 0-)

            One reason to doubt the "bashful Republican" theory is that the discrepancies were generally greater in heavily Republican precincts. Nobody has proposed any reason why voters in a precinct where most of their friends are Republican would be bashful about admitting to voting Republican when voters in a precinct that was split or mostly Democratic would be markedly less bashful. (But a heavily Republican precinct might be an easier place to mess with the votes or the machines in a pro-Repbulican direction.)

            Another is that in some precincts the extent of the Republican refusal to respond required to account for the differences is simply implausible.

            Clearly, either the polls are in error or the vote has been messed with or both. It makes no sense to assume that only the polls can be wrong.

            •  let me take the last point first (0+ / 0-)

              No one has ever assumed that only the polls can be wrong. Or, rather, I suppose that someone somewhere must have done so, but it really isn't relevant to the debate. I rather suppose that one reason that most political scientists have moved on is that they lack my (admittedly strained) tolerance for being lectured repeatedly on the possibility that the vote count could be wrong. Doh.

              It is hardly true that "the discrepancies were generally greater in heavily Republican precincts," in the ordinary sense of "generally." See the plot here, which uses good old-fashioned WPE. The four precincts at upper right strongly influence the mean for the so-called "Bush strongholds" (which actually are not Bush strongholds if we assume that the exit poll results were accurate). Really I see no need to hypothesize a psychological or any other general mechanism to account for four precincts. But if there really are non-trivial proportions of precincts around the country that witnessed vote miscount on the order of 40 or 50 or 60 percent on the margin, at least some of these precincts should stand out as swing precincts that went heavy Bush, or Dem precincts that broke evenly. That underscores the force of this analysis (to which I only contributed the narrative). Even if people choose to disbelieve those plots, still it should be possible to use election returns to identify likely candidates for massive hacking. Much of Mebane and Herron's analysis for the DNC VRI is along those lines.

        •  The problem is that you've GOT (0+ / 0-)

          demographic poll data, which few people interested in election integrity seem to take the slightest interest in looking at.

          And because you've got those demographic details, then releasing vote totals would potentially allow precincts, and thus participants, to be identified.  That is why the vote totals for the dataset used by Election Science Institute were "blurred" before release.

          •  Why couldn't you redact (0+ / 0-)

            the demographic info that might allow someone to identify someone?

            Bush's presidency is now inextricably yoked to the policies of aggression and subjugation. Mike Whitney

            by dfarrah on Sun Nov 05, 2006 at 07:10:19 AM PST

            [ Parent ]

            •  Because it's already in the public domain (0+ / 0-)

              An alternative is blurring precinct totals, which was done for Ohio.  I had understood that further datasets were being commissioned, but I don't know the details.

          •  Not true not interested in demographics (0+ / 0-)
            see the chapter in Freeman's book How America Really Voted, much of which is about the demographics.

            As for demographics:

            Here's Greg Palast's fascinating observation:

            CNN's exit polls posted at 1:05 AM on Nov 3 2004
            Among male voters,
            51% voted for Kerry, 49% for Bush
            Among female voters
            53% voted for Kerry, 47% for Bush
            As he asks - what was the third gender that pushed Bush over the top?

            CNN's exit polls posted at 6:41 AM on Nov 3 2004
            Among male voters
            47% voted for Kerry, 52% for Bush
            Among female voters
            50% voted for Kerry, 50% voted for Bush
            As Palast puts it, "the network's polling company applied a fancy algorithm, a mathematical magic wand, to slowly transform the exit polls to match the official count."

            So yes, election integrity folks are very interested in demographics.

    •  who were these "qualified analysts"? (0+ / 0-)

      We seem to go in circles awful fast on the issue of qualifications.

      Well, we go on circles on a lot of this. The most prominent argument that supposedly debunks rBr is a neat example of the ecological fallacy. The diary points out that independent observers at the Carter Center have actually discouraged the use of exit polls to validate election results. No one, amazingly, has supported the premise that "exit polls in the US were very good predictors until just recently" -- it just gets repeated.

      At least I agree that we must be very cautious and very vigilant next week.

    •  Well, perhaps you can tell us (0+ / 0-)

      how you draw a random sample "by other means" that would avoid the problem of selection and non-response bias?  You would earn the grateful thanks of survey researchers around the world.

      I did explain in my diary, the principle of cluster sampling - actually the precinct sample is a stratified sample. And I explicitly stated that the Margin of Error due to sampling error could be computed to account for the reduced variance expected in a clustered sample.

      However, that does not mean that the sample of voters is random, because they are sampled at the precinct, and are free to refuse - and do.  Mean completion rate was only 53%.  Heck of a lot of room for non-response bias there.  Plus, of course, sampling people on the street is very different from sampling phone numbers, or addresses.  No random number generators, just an interviewer trying to boost their completion rate.  They do a good job in the circs.  But if people don't want to participate (and 53% completion rate implies that a heck of a lot of people didn't want to participate) then your sample simply won't be random.  If you are lucky, the factor affecting selection will be orthogonal to your factor of interest (who voters voted for) - but  there is no reason to suppose it will, and evidence to suggest it isn't.

      If you are pointing out to your students that the only alternative to failure of the LLN is cheating, then you are, frankly, doing them a disservice.  The fact that the discrepancy was massively "significant" indicates that something other than sampling error produced the discrepancy.

      But you cannot infer, from the discrepancy, whether the bias was in the poll or the count.  There are all too many reasons why it might be in the poll.  The only way of disambiguating the two is to run correlations with some measure of discrepancy as the dependent variable.  As a statistician, you might be interested in the paper by Lindeman, Liddle and Brady that we presented to the American Statistical Association last year.  It describes the dependent measure I used in my analyses for Mitofsky.

      http://inside.bard.edu/...

      I found strong correlations with factors such as interviewing rate, and absolutely no correlation with change in Bush's vote share.  

      •  The issue isn't the margin of error (0+ / 0-)

        I don't want to get into a technical discussion over margins of error in multi-stage sampling. The issue isn't specifically whether a given precinct's reported vote totals are too many standard errors away from the proportion found in the exit poll. The issue is the pattern of discrepancies. That's where we would have a hope of detecting fraud and that's why we should be able to see the raw precinct-level proportions for each candidate in the exit polls (but not, for privacy sake, personal or demographic information.)

      •  non-response & reluctant Bush voter theory wrong (0+ / 0-)
        If you think that non-response rates are not random and have skewed the sample, you have to be able to prove that they are not random.  

        Freeman cites Mitofsky colleagues Dan Merkel and Murray Edelman who did study non-response and found minimal differences between demographic groups - women slightly higher, elderly slightly lower.  This is in a book they published, ELECTION POLLS, THE NEWS MEDIA AND DEMOCRACY.    Political scientists Samuel Popkin and Michael McDonald had similar findings in a 1998 article.

        And Mitofsky's own 2004 results disprove the "reluctant Bush respondent" thesis.  In areas where Kerry drew 80% of the vote, response rates are lower than in areas where Bush drew 80% of the vote.

        In Bush strongholds, the response rate was higher.  US Count Votes did a paper on this in March 31, 2005 which I believe can still be found online.

        •  Look I'm happy to keep retyping (0+ / 0-)

          what is in my diary, but it would do my fingers a favour if you'd actually read it.

          There is not a great deal in the data to support the hypothesis that non-response bias was a major factor in the discrepancy, although it was a contributor.  The evidence suggests selection bias.

          Response rate itself tells you very little about bias.  What causes non-response bias is differential response rates between the two sets of voters.  In a precinct with low overall response rates, if Bush voters and Kerry voters both responded at 20%, you wouldn't get bias.  But if Bush voters responded at 15% and Kerry voters at 20% you would.  Similarly in a high response rate precinct: if Bush voters responded at 60% and Kerry voters at 80% you'd get bias.  If they both responded at 75% you wouldn't.  So the fact that response rate was no lower in "Bush strongholds" than elsewhere (the regression line was statistically flat, in fact) does not tell you anything.  What would be postulated, for the non-response bias hypothesis, is that overall, Bush voters responded at a lower rate than Kerry voters in any given precinct, regardless of the overall response rate in that precinct.

          But secondly, as I said, non-response bias does not appear to have been a major factor.  The evidence suggests, in contrast, that the major factor was selction bias - that Kerry voters were more likely to be selected for interview than Bush voters - in some precincts.  This would, in fact, tend to increase the reported response rates - if interviewer were tending to select more willing looking voters, the reported completion rate would tend to go up.  But if those willing-looking voters were also more likely to be Kerry voters, so would the bias.

          So "Mitofsky's own 2004 results" do not in fact "disprove the 'reluctant Bush respondent' thesis".   And yes, the paper is still online, although it has been revised several times.  It is also still wrong.

  •  Bush administration helped fund Ukraine polls (2+ / 0-)
    Recommended by:
    Xeno of Elia, BigBite
    In testimony before the House International Relations Committee in 2004 Dick Lugar R Ind called for State Dept to fund exit polls for the second Ukraine election to insure its fairness.  The Ambassador John Teft testified also, saying the Bush admin funded exit polls there in order to detect fraud.

    So much for the diarist's myth.

    And the links to the Jimmy Carter info, some of which are huge pdf files, can be read to support the appropriate use of exit polls overseas - as in Mexico 2000 - though he cautions in some instances - Peru, where people needed to become confident that their votes were in fact confidential - they are not the best tool to use.

    •  here's a practical pov (0+ / 0-)

      exit polls and exit polls.

      You seem remarkably insistent to the point of rudeness about the diarist's "myth" when if fact there's an interesting dichotomy.

      The people with no experience with exit polls like Freemen and you seem convinced they're meaningful re fraud, whereas the people with the most experience with them (and the link i give shows it's not just Mitofsky) don't think any such thing.

      Why is that?

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sat Nov 04, 2006 at 07:33:05 PM PST

      [ Parent ]

      •  DemFromCT, Freeman has experience (2+ / 0-)
        Recommended by:
        StupidAsshole, BigBite
        with polls.  So does Kenneth Warren, the longtime pollster working with him on the independent election verification exit poll project, see www.electionintegrity.org.

        Just because I am not a professional pollster doesn't mean I don't know a great deal about them from having worked with Freeman on his book - I know more than most journalists do now.

        Besides, a lot of it is simply common sense that any reasonably intelligent careful person should be able to understand,  

        This idea that you have to be a professional or a Ph.D. to understand and argue on behalf of something is simply ridiculous.

        Exit polling is not rocket science.  Nor is statistics either.

        Exit poll discrepancies can be a strong indicator of possible fraud.  Not PROOF, as in a court of law, but an indicator and one which has been ignored when it shouldn't have been.  Why you don't care to admit this is beyond me.

        You are simply defending a broken system. As if you were content with the Democrats' persistent #2 status.   Why you want to do this, I have no idea.

        •  i didn't say you had to have a PhD (0+ / 0-)

          that's a Kathy Dopps line from the old USCV, and one i reject.

          OTOH, Warren is the experienced pollster of the group, not Freeman. I'm very interested to see how they do and as I have said repeatedly, i hope they succeed.

          It isn't so much defending a broken system as not accepting the nonsense people post as if it were fact. That I find unfortunate.

          "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

          by Greg Dworkin on Sat Nov 04, 2006 at 07:59:06 PM PST

          [ Parent ]

          •  I do have a PhD (2+ / 0-)
            Recommended by:
            Xeno of Elia, BigBite

            and it is even in Statistics. And I used to work for a major network making predictions on election nights.

            We no longer do that because the exit polls were (or at least used to be) so good that it was decided that our extensive system of gathering precinct-level results and combining them with a complex Bayesian model wasn't worth the cost or the effort.

            Then the exit polls stopped working.
            By that I mean that they no longer predicted the official vote.

            Now, either we forgot how to take polls
            or Mitofsky is incompetent
            or the Law of Large Numbers (the Statistical result that tells us that the polls estimate the way people really voted) has stopped working.

            or the voting system is broken.

            I don't believe any of the first three, and the risk of the last of these to our Democracy is so great that I can't understand anyone not taking it seriously. Especially those with precinct-level raw data, which could help identify the trouble spots.

            •  Well, as someone with a PhD (0+ / 0-)

              in statistics, you will know that the Law of Large Numbers applies to a random sample, and so that if it appears to have stopped working, one possibility is that the sample is not random.

              It is possible that the polls used to work better, although the evidence is that they have had bias since at least 1988.  However, it may not be the methodology that has changed.  Do you know what the response rates were in your day?

              And can you seriously argue that a mean response rate of 53% indicates a random sample?  

              And can you say what analyses you would apply to the precinct level raw data, were you to have access to it?  What hypotheses would you test?  And how would you account for a slight but non-significant negative correlation between redshift and swing to Bush (N=1250) if fraud was responsible for variance in both?

              It's all very well arguing that the data need to be examined for fraud.  But they were.  I turned it upside down and inside out.  What would you do with that data?  

              •  Look at the data (0+ / 0-)

                There is always some response bias in polls. But it is usually possible to look at patterns in the biases and understand them. I don't know whether the response rates have changed, but a 53% response rate is far better than the response rate of published telephone polls. Those coontact about 75% of the randomly generated tlelphone numbers, but complete only about 38%  (Pew's reported numbers; less for many companies) of interviews with those contacted -- less than a 30% response rate (and much lower than that for SUSA-style automated polling; see their site.)

                And exit polls have the huge advantage over pre-election polls that they don't need a "likely voter" model; we know they are only talking to voters. Predicting who will turn out is a huge source of bias in the pre-election polls, and a major reason why polls from different companies taken at the same time can disagree.

                The LLN applies formally to simple random samples. But there's ample theory that other methods (including the systematic sampling often used in exit polls) will work the same way. Modern polling uses a variety of methods to reduce variance and bias, but none of those invalidate the LLN (or they wouldn't be used).

                If I had more of the raw data, I'd look for patterns. Any theory that accounts for discrepancies between exit polls and reported votes based on social or political forces will predict some kind of pattern; more bias in richer or poorer precincts, more in white or black areas, etc. If a precinct shows a large error, then others like it should do so to and in the same direction.

                But if the errors are due to vote fraud, we shouldn't expect that pattern. The fraud will take place according to the kind of machines used for voting, or in isolated hot spots, or in ways that will relate to the paths followed by the data and the custody and aggregation trails.

                And none of this would compromise anyone's privacy (except maybe the reputations of some election officials.)

                •  I looked at the data (0+ / 0-)

                  and I looked at patterns in the biases.  Did you actually read my post?

                  And it doesn't matter what the response rate is for telephone polls, we are talking about face-to-face sampling.  If the response rate is 53% you do not have a random sample.  You have a volunteer sample.  Moreover, if your response rate is 53% you can infer absolutely that a large proportion of voters do not want to be polled.  From which it is perfectly reasonable to postulate that a proportion of those "reluctant" voters (which we KNOW exist) may actually evade selection.  This cannot happen in a telephone poll, but there is no reason to suppose it won't happen in a face-to-face selection process - and we have strong correlational evidence that it does.

                  You are right that the exit polls do not need a "likely voter" model - it is one of the things that makes the cross-tabulations - adjusted or unadjusted - so valuable.  But it does not alter the fact that the selection process itself is quite different to that used in telephone polls, which are conducted over several days by experienced interviewers, who can call back if a voter is unavailable.

                  And we are not, as you must be aware, talking here about systematic sampling.  The MoE of a clustered or stratified sample is perfectly possible to calculate.  And as, again, I explain in my diary, methods are available to reweight the data to compensate for bias.  In fact, that was one of my major points.  I can't believe you missed it.  But what method do you propose for compensating for bias you don't actually know you have, namely bias in the variable you are actually interested in?  By definition, no-one knows who the non-respondents voted for.  So the only source of data to reweight by this variable is the vote-returns.  Which of course is exactly what the pollsters do.

                  Check my other posts for details on what I found, but in short:

                  Discrepancies were correlated with a number of methodological factors likely to make non-random sampling more difficult, e.g. interviewing rate. A relatively small multiple regression model accounted for all the redshift.  This actually surprised me, as there is no particular reason to expect it would, as it is not the methodological factors that are postulated to cause the red-shift -  they are merely postulated to cause variance in bias that is actually due to a postulated differential propensity to take part in the poll.  But it is good news for the pollsters, as it suggest that with careful methodology, the underlying tendency can be overcome.

                  But much more tellingly I correlated the discrepancy with change in Bush's vote share, and found absolutely nothing.  I think I've given you the link already, but here it is again:

                  http://inside.bard.edu/...

                  as it was written up by my colleague, Mark Lindeman.

                  I'd actually be genuninely interested in any fraud mechanism leading to widespread vote-theft you can postulate that would be consistent with that finding.  It's not often I get to talk to a PhD statistician.

                  •  I might note that we have hard evidence (0+ / 0-)

                    of non-response (or participation) bias with respect to age -- rather extreme bias, since the reported completion rate for voters aged 60+ is twelve points lower than that of younger voters (43.0% versus about 55.5%, estimated from p. 54 of the evaluation report). That bias is (crudely) measured, so it can be compensated for.

                    The wild leap is to argue that since we can't directly observe participation bias with respect to vote choice, we should assume that it doesn't exist. It would be more reasonable if we actually had reason to believe that U.S. exit polls have been rock-solid reliable in the past. Actually, we have reason to believe otherwise, as Charlie Cook has testified elsewhere in the comments.

                  •  I'm not a PhD (0+ / 0-)

                    though I do have a degree in Math & Physics from Cornell, so I'm probably at least as bright as your average Kossack and I can't help but think that entire story line - and yes I read all of your post as well as a good bit of Kathy Dopp's analysis debunking Mitofsky's and your rationalizations, (see her post here) - about response bias is a concocted cover story for real and unexplained discrepancies between the exit polls and the votes.

                    Further, the assumption that the vote total is correct and the exit polls must be modified to match sounds ever so much like how I used to fudge my lab results in college. Either the polling was done incorrectly or the results point to serious problems. If you're going to modify the exit polls to match the vote after the fact, why bother with them at all?

                    Democracy is a contact sport...

                    by jsmagid on Mon Nov 06, 2006 at 07:15:25 PM PST

                    [ Parent ]

                    •  So, you have a degree in Math and Physics (0+ / 0-)

                      Have you actually read Dopp's paper, or are you taking her assertion that I am a liar at face value?

                      And if you used to fudge your lab results in college, then what you were doing was nothing like what the pollsters do in assuming the vote count is correct.  All science proceeds on assumptions.  It is only the undeclared assumptions that can be described as "fudge".  The pollsters do not claim - do not intend- to monitor the integrity of the election.  That is why they assume the count is correct for the purposes of their projections.

                      However, if you are implying that they - or I - assumed the count was correct for the purposes of analysing the reasons for the discrepancy in 2004, then you would be quite wrong, as is easily ascertained from the fact that they actually tested at least one miscount hypothesis in their evaluation paper.

                      But frankly, if you can give credence to the post you link to, despite the fact that the poster accuses me of lying on the basis of a misquotation that can be easily verified as such, then I don't have much hope of a rational debate with you.  If, on the other hand, you want to read her argument, and present it here, I am happy to explain why I think it is completely irrelevant to anything I or Mitofsky have claimed.

            •  some detail might render this useful (0+ / 0-)

              What time period are you talking about? Are you suggesting that the exit polls were accurate until 2004 (or maybe until 2000 in Florida)?

              So accurate that your network decided it could call every state from the exit poll results without waiting for vote counts? Or do you mean, accurate enough that if a race was too close to call from the exit poll, there was little value added in integrating the exit poll interview data into the vote count analysis? Just how "so good" were the exit polls, before they weren't?

              If you don't understand why there isn't more professional urgency about releasing exit poll data, well, I've seen little effort to make the case in a way that would convince academics that there is an issue here. What have you been doing for the last two years? Whatever it is, it hasn't worked. It's never too late, I guess.

        •  You know something? (1+ / 0-)
          Recommended by:
          DemFromCT

          I never met Warren Mitofsky, and now I never will.  But I got to know him a bit through our email exchanges, and one thing that struck me extremely forcibly was that, cantankerous though he could be, he was prepared to learn from absolutely anybody, including, to my astonishment, me.

          He had more experience and knowledge of exit polls than anyone in the world - he invented the system. And he knew both how powerful the information they could provide was, and the nature of its limitations.  And yet he knew that there was always more to learn, and was prepared to learn it from anyone whose arguments made sense.

          No, exit polling isn't rocket science, and nor is statistics, and you don't need a PhD to understand either of them.  What you do need, however, to understand anything, is the kind of ruthless honesty and intellectual humility that Mitofsky exemplified.

          I'd like to see more of it around.

          •  Freeman also found Mitofsky (0+ / 0-)
            open to discussion.

            And as he states in his book, Mitofsky's experience with exit polling is unparalleled.  Which makes it all the more curious that he repudiated his own polling in 2004.

            •  Well, the curiousness (0+ / 0-)

              is easily accounted for by the fact that he was aware, as no other, of the limits of his own data, and was aware, because he'd examined the data in detail, and hired me to examine it further, that it supported his hypothesis, and did not support the hypothesis that fraud was responsible for the discrepancy.

              It's called honesty.

      •  Evidence is evidence is evidence (1+ / 0-)
        Recommended by:
        BigBite

        It is irrelevant whether or not the exit poll is designed "to detect fraud or not."   If you doubt this principle, then I suggest you go to law school, become an attorney, represent a criminal defendant, and make the argument IN COURT that your client's fingerprints on the weapon are NOT evidence of possession of that gun because his fingerprints were NOT INTENDED to prove possession.

        No matter what the stated purpose(s) of a poll are, its use as evidence is unlimited so long as it is relevant to a point someone's trying to prove.  Since exit polls do try to state what numbers are for election results (or can be used that way) the exit poll resuls are indeed relevant.   Depending on specifics, they may not fully prove anything, but they are some evidence on one side or the other of the scale of justice.  Moreover, even if two experts of equal or unequal qualifications directly disagree with each other, they DO NOT CANCEL OUT, nor make any issues go away. In court, in the aforementioned event, the jury or judge or the American public get to choose sides...

        •  sure, you can believe whatever you want (0+ / 0-)

          But a rationale would be nice.

        •  Oh, I agree (0+ / 0-)

          And I spent a large amount of time myself trying to reverse-engineer the polls so that they might tell us something about fraud.  

          That's why I conducted this analysis for Mitofsky:

          http://inside.bard.edu/...

          And inference I drew from it was that if anything it contra-indicated the case for massive widespread vote-switching fraud, and at best, required heroic assumptions about the nature of the fraud in order for the case for large-scale fraud to be consistent with it.

          Whether a jury would understand the implications of that analysis, I don't know.  But that wouldn't make any difference to its validity.  Check it out yourself.

  •  Freeman takes issue with every one of the "myths" (3+ / 0-)
    Recommended by:
    Xeno of Elia, StupidAsshole, BigBite
    that the diarist attempts to debunk here.  

    I think it's important that people who read this diary know that there is an alternate view of the Edison-Mitofsky NEP polls than is presented by Febble.

    In fact, in many ways Freeman gives greater credence to the findings of the NEP exit polls than do the pollsters themselves.

    It's bizarre that Febble promotes the NEP polls at the same time that she apologizes for them, and disturbing that she puts down independent election verification exit polls with transparent methodology and data while defending privately owned media exit polls based on unreleased data.

    Voters and Democrats deserve better!

    •  I don't see Freeman actually giving credence (0+ / 0-)

      to the NEP polls. He has made no attempt to convince me that he actually believes them. He hasn't explained the lack of correlation between shift and swing; he hasn't explained the lack of correlation between shift and pre-election poll deviation. The "wonderland of numbers" shtick in his text creates a strong presumption that he never bothered to look at any exit poll dataset with the possible exception of 2004, since he doesn't report on how people reported their past votes in any earlier exit poll. It is hard to figure.

    •  Well, would you like to explain his arguments? (0+ / 0-)

      And why my responses are inadequate?

      I am happy to debate issues.  But you seem to be here to stir discord, mostly by completely mis-stating what I have said, and inventing reasons you think I might have said it.

      I did not "promote" the NEP polls.  I said that they were not much use for monitoring election integrity, which is true.  I said that polls designed for the purpose would be better, which is true.  I expressed some concerns about those polls, which are genuine.  If the people conducting the polls can't take a critique fo their methodology, then they won't learn anything about polling.  Fortunately, all the exchanges I have had have had with them indicate that the are keen to learn as much as they can.  I spent several hours with Freeman in Montreal, during which he took notes, and I have been in constructive communication with several people involved in independent polls designed to monitor election integrity.

      So would you kindly lower your weapon, and get out of my diary until you can actually present an argument, rather than a series of faulty assertions.

      •  I stand by the explanations I have already posted. (0+ / 0-)
        •  You haven't posted any (0+ / 0-)

          You haven't raised a single one of Freeman's arguments - just stated that he disagrees.

          When you want to discuss an actual argument, I am ready.

          •  Febble, that's not so - I do explain Freeman (0+ / 0-)
            Read my posts upthread in which I discuss what Freeman says about
            the difference between media polls and verification polls, his explanations of how the media pollsters adjust the results,
            his analysis of the reluctant Bush respondent theory,
            his observation of the overwhelming and statistically impossible PATTERN of the red shift
            his observation that the media pollsters assumed the official count was correct and unscientifically did not test for the possibility that it was corrupted somehow (which could be by undercounts and provisional ballots, not simply by actual fraud)
            his statement that the exit poll results are not PROOF of fraud but as Conyers says in the intro to his book, one indicia that something went wrong and ought to be fully investigated
            and more.
            •  Well, I agree with you (and Freeman) (0+ / 0-)

              about the difference between media and verification polls. It's why I made the point at the outset of my diary.

              And if by "his analysis of the reluctant Bush respondent theory" you mean his theory about non-response bias, then you do not seem to have read my response to that - overall response rate is largely irrelevant to the issue of differential response rates.

              I am not sure which "overwhelmingly and statistically impossible PATTERN of the redshift" of the redshift you are referring to.  Please pick one, and explain what you understand by its being "overwhelmingly and statistically impossible".

              The statement that the media pollsters did not test for the possibility that the count was corrupt is also simply not true.  Freeman saying it is does not make it so - the E-M report details the results of tests of differences in discrepancy by machine.  Why would they have done that if they had "assumed the official count was correct"?  And, moreover, Mitofsky gave me a free hand with the data, and I certainly did not assume that "the official count was correct".  Nothing in my exchanges with Mitofsky led me to believe that expected me to, and, indeed, I tested several rather important fraud hypotheses.

              It is, however true that Mitofsky was confident that the discrepancy was probably due to bias in the poll from the outset because, again, as I point out in my diary, he was already aware of indicators of bias before a single result was available.  So corrupted count could not have accounted for those discrepancies - whereas bias in the exit poll could have.  He was also keenly aware of similar problems in previous elections.

              So, sure - Mitofsky was pretty sure he was backing the right horse.  But it didn't stop him checking out the alternative.  And it didn't stop him hiring a DKos poster with a history of investigating election irregularities to check again.

  •  Tester listed as Rep on their site (0+ / 0-)
    I was checking the states on their site that they'll be taking exit poll info and found for Montana they list Tester as as a Rep and Burns as a Dem...they can't be all that. I emailed them and told them. I didn't look at any more 'cause it wasn't like they spelled a name wrong, they got their party wrong!
  •  Charlie Cook on Better Use of Exit Polls (4+ / 0-)

    Having worked at networks for every election since 1992, my advice on exit polls?  Ignore them.  They aren't very good.  Any contest that is so close than you couldnt figure out who was going to win by reading the previous week's Time or Newsweek is too close for an exit poll to tell you anything.  It's a poll, it's not a count.

    In 2004, while you guys all got your panties in a wad over the exit polls, I stayed away from the network where I worked until 5:00pm, did not allow anyone to tell me over my cell phone or open any emails on my blackberry that appeared to have anything about exit polls and any emails that looked like they had any exit poll data in it, I immediately closed the email before seeing the results.  

    As a result, I entered the vote count with a clear mind, no preconceived (erroneous) thoughts about what was going on and did a better job.  If you guys had seen as many screwball exit polls as I have, you would too.

    The best use of exit polls is to be able to look and say, approximately, how various demographic groups voted, after the election is over and after the data have been cleaned up.  Anything more and you are simply engaging in mental masturbation.

    This year, no data will be out until 5:00pm.  My advice, go to a 4:00pm or 5:00pm EST movie, have dinner, relax, and ignore anything until after maybe 8:30pm or 9:00pm, then settle in and enjoy watching the real returns come in.

    •  thank you (0+ / 0-)

      Check the House at 8:30 for IN and CT, take a nap (at least I will) and come back at 10 and beyond for the Senate. The Senate may take a while.

      I look forward to your analysis (and that of your team, Amy and Jennifer et al) the day after.

      "Politics is the art of looking for trouble, finding it everywhere, diagnosing it incorrectly and applying the wrong remedies." - Groucho Marx

      by Greg Dworkin on Sun Nov 05, 2006 at 06:14:01 AM PST

      [ Parent ]

    •  Just remember the differences between (0+ / 0-)
      the media exit polls and election verification exit polls.

      results for the latter probably won't be available until the following morning.

      One of the fallibilities of our system is the need to know (or be told) immediately what has happened.  We have no patience for a careful count, and that is our loss.

  •  Great, lets let them steal the midterms. (0+ / 0-)

    What's the excuse going to be this time, Saddam's trial?  The abortion cases the Supreme Court will be hearing starting on the 8th?  The amazing GOP 72 hour GOTV?  Or perhaps the tested favorite, surpising evangelical turnout because of gay marriage?  Or were the exit polls wrong.  All excuses for massive fraud.  Touchscreen DREs were practically designed for fraud, obvious glaring security errors, why is that?

    Every "glitch", every bit of suppression, the MSM dismissal of fraud, all favor republicans.

    Florida was stolen.  Ohio was stolen.

    If you don't believe this I'd like to know where you've been getting your news.

    Neocons don't believe in democracy, they believe in elite rule.

    It's not about Republican or Democrat.  It's the corporate media supporting their corporate president.  It's all about MONEY!

    Perhaps the exit polls were wrong - but why?  The shy republican hypothesis?  Still peddling that BS?

    Even if I had never heard the exit polls prove fraud theories, I would STILL believe '04 was stolen - because if you look hard enough, you find mountains of evidence.

    I suggest you all visit BradBlog once in a while...

    Much more information, and links to election fraud related books and movies here: ElectionTheft.net

    •  Sigh (0+ / 0-)

      "Perhaps the exit polls were wrong - but why?  The shy republican hypothesis?"

      The hypothesis is simply that Kerry voters tended to be more willing to participate than Bush voters.  It's a perfectly testable hypothesis.  I tested it.  It was supported by the evidence.  Much as the Theory of Evolution is supported by the evidence, even though we cannot directly observe macro-evolution.

      "Still peddling that BS?"

      Yes, because it isn't.

      "Even if I had never heard the exit polls prove fraud theories, I would STILL believe '04 was stolen - because if you look hard enough, you find mountains of evidence."

      Well, I'm happy happy to see those mountains assembled into a case.  I actually contributed to a couple myself.  But the fact that you have some good evidence doesn't make poor evidence better.  It simply dilutes your case.

  •  Febbles's Post (0+ / 0-)
    In the interest of fairness, I have been looking at an e-mail I recently received from Kathy Dopp who claims that Elisabeth's post is full of distortions and even lies.
    Click to read:
    http://www.progressiveindependent.com/...
    •  Response below (0+ / 0-)

      We already irrefutably mathematically proved that everything Liddle says in her latest Daily Kos article is wrong and our proofs are all summarized or referred to here:

      http://electionarchive.org/...

      I had hoped that Liddle was turning over a new leaf but she found a new way to present her hopelessly illogical cr--.

      Well, we are off to a nice civilized start I see. If anyone would like actually to make Dopp's argument here, I'd be more than happy to engage with it.

      Wrong stuff in Liddle's DailyKos article includes:

      "Again, if the incoming vote returns indicate a systematic divergence from the exit poll response, the pollsters have reason to suspect bias in their sample."

      Liddle's arguments re. the influence of polling factors do not take into account the "base rate" of Kerry over-estimation by polls even in the absence of these factors.

      I am not entirely clear what Dopp means here, but it does not seem to me to undermine my statement which is simply that if two polls diverge, you have reason to suspect bias in at least one of them. Perhaps Dopp feels the more likely source of bias is the pre-election poll.  If she'd like to clarify, I'd be happy to consider her point.

      Ron Baiman and I mathematically proved her other statements wrong in our academic papers on exit polls available on http://electionarchive.org (in the exit poll section on the home page.)

      Well, I disagree.  Again, if someone would actually like to make the argument, I'd be happy to respond.

      Liddle also proved (to me at least) that she is an outright liar in this Daily Kos article because she made a statement in it that she had, in many emails to me and to colleagues I worked with, asked me to remove from one of my papers, claiming that she never made the statement. Yet she makes the exact same statement again in Daily Kos.

      i.e.
      "There is absolutely no correlation between the magnitude of the precinct-level discrepancies in 2004 and change in Bush's vote share relative to 2000 (what UK commentators call "swing"). If a single factor, e.g. fraud, was responsible both for both the discrepancy, and for inflating Bush's vote, then you would expect the two to be positively correlated."

      Dopp appears to have difficult in understanding the difference between a sentence she alleges I wrote:

      If there is vote fraud, then there will be a positive correlation between Bush vote increase from 2000 to 2004 and the exit poll discrepancy.

      http://uscountvotes.org/...

      and various sentences I did write, such as the one she quotes from my diary:

      If a single factor, e.g. fraud, was responsible both for both the discrepancy, and for inflating Bush's vote, then you would expect the two to be positively correlated.

      The crucial difference is, of course, as alert readers will note, is the parts I have marked in bold.  Dopp's paper, in which she claimed to

      "irrefutably mathematically prove[d] that everything Liddle says in her latest Daily Kos article is wrong"

      takes the form of a proof that a statement of the form she alleges I made is illogical.  It is, in fact illogical, which is why I did not make it.  The statement I did make, on previous occasions, and in this diary, is perfectly logical.  I am happy to debate this further if anyone can be bothered.

      The statement that she repeatedly tried to get me to remove from my paper which I quoted her as saying was;

      "If there is vote fraud, then there will be a positive correlation between Bush vote increase from 2000 to 2004 and the exit poll discrepancy."

      Exactly.  This is a misquotation, embedded in a mis-citation with a mis-attribution: I did not make the statement; it does not appear in the cited source; I was not the author of the source.

      Or is she claiming that sometimes vote fraud favoring Bush does not inflate Bush's vote from what the voters cast it?!! Really?

      Well, no.  I don't even know what that means.  It is certainly true that when variance in two measures share a cause they will not always be correlated - indeed it is why we compute probability values and effect sizes.  This is why I have never said, as a statement of necessity that if Bush's change in vote share and the exit poll discrepancy shared a common cause there must be a correlation. But if there isn't a correlation you cannot infer a common cause, and with a great deal of statistical power you can put a maximum probable value on the size of the effect.  What you can say, from the data, is that IF the two things shared a common cause, the maximum probable size of the effect was small.

      There are alternative inferences; for example if one makes heroic assumptions about the uniformity of the fraud; or one postulates that the fraud was carefull targetted only in precincts in which Bush was anticipated to do worse than his average, then the failure to find a correlation might be consistent with fraud.  But the data do not actually support the inference, and indeed, the assumptions made run counter to the kinds of fraud mechanisms that have, to my knowledge, been postulated.  But sure - it would be possible for there to be fraud, and yet for there to be no correlation. But it is not what we would expect if the fraud was on a massive scale.

      In the Lindeman docs she links to support her statement, the definition of "inflating Bush's vote" is an increase in the Bush vote in 2004 as compared to the 2000 election.

      Well, no. Lindeman doesn't even use the words that Dopp places in quotation marks (Dopp seems to have difficulty with the cut and paste function).  So I am not at all clear what she thinks Lindeman is defining. But the measure I defined as "swing" was, essentially, Bush's margin in 2004 minus his margin in 2000 (actually I used the standardized residuals from a regression, but the differences are minimal).  A "positive" value for "swing" means that Bush improved his vote-share; a "negative" value for swing means that his vote share decreased.  The correlation tells the extent to which variance in "swing" was correlated with variance in "redshift" - the magnitude of the exit poll discrepancy in his favor.  If Bush had tended to do better (or less badly) in precincts where the discrepancy was large, and worse where the discrepancy was small (or in Kerry's favor) it would have been suggestive, at least, of fraud. One could have concluded that variance in both measures shared a cause.  However, no such tendency was observed.  

      Liddle KNOWS that this hypothesis has been proven mathematically incorrect many months ago, so why revive it NOW?

      Well, I do not agree that it has been proven mathematically incorrect.  I agree that Dopp proved that something was mathematically incorrect, but not anything that I actually did or said.

      Why right before the election?

      Because I thought it would be useful for people to have a primer on how the NEP polls work, and the sources of error they are likely to contain.

      Why simultaneously with the NY Post announcement?

      I know nothing about this announcement (not being an American).  Perhaps someone would enlighten me.

      Why on the Daily KOS where anyone who can present convincing facts to refute her has been banned?

      Because I thought Daily Kos readers might appreciate the information.  Clearly a great many did.

      Liddle also makes the case for not revealing any exit poll data until after the polls close and the New York Post today informs us that the exit polls will be kept secret until after polls close!

      Well, not exactly.  I explained the case for not revealing data until after the polls close, and therefore specifically recommended that people take note of the data at close of poll if they are interested in the magnitude of subsequent adjustments.  

      Coincidence. Me thinks not!

      Here is the Post article:

      http://www.nypost.com/...

      Ah.  Oh well, that's exactly what was in the Lenski link in my diary.  Yes.  Looks like no leaks before close of polls.  Nothing about not posting close-of-poll cross-tabs though.

      I think we know who Liddle is working for, and why this post was made just three days prior to our election.

      Well, I made it clear in my first post that I had worked for Mitofsky.  I'm not working for anyone right now apart from my regular employer.  But sure, the reason I posted just three days prior to your election was because I thought the information would be useful.  As I said.

      •  wow, you really worked this one (1+ / 0-)
        Recommended by:
        Febble

        I thought you might have stopped with this:

        "We already irrefutably mathematically proved that everything Liddle says in her latest Daily Kos article is wrong...."

        Riiiiiiight. Every word of it, mathematically falsified! And Dopp threw in a mathematical proof of immortality for good measure. (People give all the credit to Frank J. Tipler, but actually, he stole the ideas Dopp was about to have, if you hadn't been preparing to distract her with all that Fancy Function bunk.)

        The other thing worth underscoring is that what the New York Post story says about the results being secret until 5:00 Eastern matches what Lenski told Andrew Kohut in an interview published on September 28. Hmm, September 28 -- the same day you posted a diary about... umm, nothing. Coincidence?

        Jeepers.

  •  While this was informative (0+ / 0-)

    it seems to be rehashing the Edison/Mitofsky report.  I wonder if you could please comment on the findings of the consortium of PhD mathematicians and statisticians from USCountVotes.org.   If you are not familiar with their work, you can find a description here.

    The March 31 analysis by USCountVotes.org, however, demolishes the Edison/Mitofsky conclusion. The Ph.D.s who conducted the analysis found, using NEP's own data, that Bush voters actually had a slightly higher rate of participation in the exit polls than Kerry voters.

    The signers of the critical analysis include such experienced hands as Frank Stenger, professor of numerical analysis at the University of Utah; Richard Sheehan, professor of finance at the University of Notre Dame; and Campbell Read, professor emeritus of statistical science at Southern Methodist University. These experts say in their paper, "No matter how one calculates it, the discrepancy cannot be attributed to chance."

    Ron Baiman, Ph.D., is associated with the Institute of Government and Public Affairs of the University of Illinois at Chicago and is a signer of the report. He found that "statistically significant discrepancies of exit poll results from reported election outcomes were concentrated in five states, four of which were key battleground states," such as Ohio, Pennsylvania, and Florida. As most Americans know, the presidential vote in these states was close, and a shift in any one state would have tipped the election.

    I have always found the E/M results to read basically like: "the election had different results than the exit polls, and therefore the exit polls were inaccurate."

    As a mathematician, I find the full results from USCountVotes.org (found here) to be much more thorough and more troubling.

    •  Certainly I can comment (0+ / 0-)

      as I was one of the people who had a little input into the early stages of the report.  I declined to put my name to it, however, as while I found its intentions and recommendations and laudable, I found two of its key arguments flawed.

      One of those arguments I addressed in detail in a paper I presented on Daily Kos:

      http://www.dailykos.com/...

      which is posted here:

      http://www.geocities.com/...

      In it, I argued that the aggregate data points presented in the E-M report were not sufficient to draw the conclusion that "exit poll discrepancies rise with the concentration of Bush voters" as the measure used by E-M to quantify the magnitude of the discrepancy (the "Within Precinct Error" or WPE) would tend to introduce an artefactual slope in the data.  I suggested an alternative metric, which Mitofsky subsequently used to re-analyse the data.  What he showed was that while if the the WPE was used as the measure, there was indeed a slight but statistically significant monotonic relationship between discrepancy and concentration of Bush voters, when the measure I suggested was used, this relationship was no longer significant.

      He presented this finding at AAPOR 2005, and it is written up here:

      http://www.mysterypollster.com/...

      I do not (and did not) argue that this refutes the hypothesis that fraud was responsible for the discrepancy, merely that we cannot deduce from the apparent positive monotonic relationship between discrepancy and Bush's vote-share that fraud was responsible for the discrepancy, as the relationship seems to depend on the measure used (and I'd argue that WPE is a poor one).

      The other issue was the argument from response rates.  As I say in my diary (and this is not a rehash of E-M - it is not a point they make), overall response rates are unlikely to have a simple relationship with differential response rates. In any case, the E-M report stated that there was no significant difference in response rates between precinct partisanship categories, and at AAPOR Mitofsky also presented a plot indicating that there was no monotonic relationship between refusal rate and partisanship.  See this DKos diary for further details on Mitofsky's AAPOR plots:

      http://www.dailykos.com/...

      However, as a result of my DKos paper (in which I criticised the E-M report, and only USCV by way of the data they had abstracted from the E-M report) for its use of the WPE, Warren Mitofsky contracted me to reanalyse the data using the measure I had proposed.  In fact, working together with Mark Lindeman and Rick Brady, we found that that measure too produced artefacts, and we devised a modified version.  This was presented at the 2005 meeting of the American Statistical Association, and, if you are interested (as a mathematician  - it's fairly technical) you can read it here:

      http://inside.bard.edu/...

      And this was the measure I used for the analyses I conducted for Mitofsky.

      As for my diary: no, it isn't really a rehash of the E-M report, although that was certainly one of my sources of information - another was simply what I learned through my exchanges with Warren Mitofsky, and through immersion in the ins and outs of the NEP exit polls.  The diary was designed primarily as a compendium of information about exit polls in general and the NEP exit polls in particulary that I didn't have in 2004, and would like to have had.  Which is why I presented it here, now.  However, I also thought it was time I updated Kossacks with the key findings that emerged from my work for Mitofsky. One is that, using what I consider a more robust measure, I was able, using multiple regression models, to demonstrate that there was indeed strong evidence to support the hypothesis that the discrepancy was associated with methodological factors that were likely to be associated with departures from strict non-random selection (and that selection bias, rather than non-response bias, was likely to have been the major contributor to the discrepancy).  The other was that I found absolutely no correlation between the precinct level discrepancy and "swing" to Bush - in other words, there is no hint of a tendency for Bush to do better where the discrepancy was greater.  If you check my other comments in this diary,  you will find more details, but essentially, heroic assumptions are required to make this finding even consistent with widespread fraud of a type that would produce an exit poll discrepancy, and it certainly does not support it.  The finding, I would argue, if anything, contra-indicates the case that massive widespread fraud occurred.

      The finding, with commentary, was presented by Warren Mitofsky at a meeting of the American Statistical Association in Philadelphia with Steve Freeman, and details of the finding, with commentary by Mark Lindeman, are given here:

      http://inside.bard.edu/...

      I am happy to check back to respond to any comment you may have.

      •  Thanks (0+ / 0-)

        This does clarify a few issues.  Specifically, I am intrigued that the correlation is dependent on the metric used.  This either argues for the sensitivity of certain metrics or for their bias/inappropriateness for the job.  Thanks for the posting to your work presented at the ASA.  I will look that over this week and digest it (probably on Tuesday after voting!)

        Cheers!

        •  just a caveat (0+ / 0-)

          with the open disclosure (in case you haven't figured it out already) that I am Febble's friend and colleague, although we have only met once.

          The correlational issue is pretty cool, but really not much should depend on whether the 'true' correlation between red shift and apparent Bush vote share is more nearly 0 or 0.05 or 0.1 (or whatever). There are certain fraud scenarios for which that might be a diagnostic, but as Febble said, it isn't a clincher either way. However, I think the scatterplots illustrate the problem with pondering the mean WPE in precincts with 80%+ Bush vote share: the arbitrary partition leads to misleading results. (This should be apparent even from the reported median for these precincts, which appears to be smaller in magnitude than the overall median [not reported].)

          You may want to check out the scatterplots linked via my home page (the first two or three), since the ones on Mystery Pollster don't indicate the Y-axis scale. The scatters are also pretty nifty for visualizing the source of the artifact: points at upper left and lower right are arithmetically impossible. Of course things get more complicated from there.

        •  Just one comment to clarify (0+ / 0-)

          The correlation (or rather lack of correlation) between discrepancy and "swing" is not sensitive to the measure used.  It is the correlation between discrepancy and vote-share that is (the one in Mitofsky's AAPOR plots).

          Cheers

          Lizzie

    •  PS (0+ / 0-)

      I had not followed your first link when I posted my response.

      I should therefore make a couple of additional points:

      One is that it is not correct to say that

      Bush voters actually had a slightly higher rate of participation in the exit polls than Kerry voters.

      In fact, there was no significant difference in refusal rates, but more to the point, participation rates are not simply a function of completion rate (completion rate = responders/[responders+refusers + "missed"], where "missed" are voters who were selected for inclusion but "missed" because the interviewer was still busy with the previous voter).

      "Participation rate" would be the rate at which Bush voters took part in the survey, which would also be a function of the rate at which they were selected.

      Second, is that I am not at all sure how Baiman arrived at his conclusion that

      "statistically significant discrepancies of exit poll results from reported election outcomes were concentrated in five states, four of which were key battleground states, such as Ohio, Pennsylvania, and Florida."

      but it is somewhat misleading to imply that the largest discrepancies were in battleground states.  According to the E-M report (pp 21-22),

      http://www.exit-poll.net/...

      of the three battle ground states Baiman mentions, the largest discrepancy was in Pennsylvania (11.5 points) and the following states had greater or equal discrepancies:

      Vermont
      New York
      New Jersey
      New Hampshire
      Missippi
      Delaware.

      and several others had discrepancies at least as large as that in Ohio.

  •  Liddle, what would it take... (0+ / 0-)

    Liddle, I'm interested in understanding what sort of evidence would convince you that vote fraud had occured?

    For me, the discrepancy between exit polls and results, combined with the baffling refusal of government officials to provide transparent voting proceedures, combined with the certain knowledge that vote fraud in America *has8 occured in the past, is enough.

    But I'm neither a mathematician nor as scientist.

    What is the minimum anount of evidence that would convince you?  In other words, is the hypothesis that "Exit polls are not reliable" at all falsifiable?

    •  Well, some of the evidence there actually is (0+ / 0-)

      convinces me that fraud occurred.  There isn't a "minimum amount" that would convince me.  It would depend on what the evidence was.  My overall take on the 2004 election is that it was manifestly unjust - and I use that word advisedly.  Much of what went wrong may not have been illegal, but was immoral (such as many of the voter suppression tactics that went on, including Blackwell's absurd "80 gram paper" registration ruling; and much of what also went wrong may not have been malicious but was certainly (IMO, but IANAL) negligent, such as the under-supply of voting machines to Democratic precincts in Franklin County, Ohio.

      I'd be convinced by anything that was, well, convincing.  I would be extremely surprised if no outright fraud occurred in Florida or Ohio.

      And quite apart from all that, the very fact that electronic voting systems are vulnerable to hacking, and also simply unreliable means that you have no basis for confidence in your vote-counting system, and, in many cases, no basis for confidence in the checks that should be available to you, such as fair recount procedures.  I am appalled at the laxity regarding such things as custody of ballots (or voting machines for that matter).  If there was no fraud in 2004 it would not be for want of opportunity.  The system is either a crime waiting to happen, or a crime that has already happened, in many places, in many ways.

      But I don't think you can simply, at this stage, add the exit poll discrepancy to other evidence, or, for that matter, multiply other evidence by the exit poll discrepancy.  It was a reasonable stance immediately after the election - indeed it was one I took - that given evidence of malpractice, and the unprecedented (apparently) size of the exit poll discrepancy, that fraud might have been committed on a scale to have contributed measurably to that discrepancy.

      However, although the E-M report was short on the kinds of statistical detail a statistics geek like myself would have wanted, it did, contrary to widespread assertions to the contrary, report evidence in favour of the hypothesis of bias in the poll.  The part that struck me as most convincing was the part that demonstrated that the discrepancy was greater where the interviewing interval was long.

      But it didn't exactly satisfy me, not least because I wasn't happy with the actual measure of precinct level discrepancy.  Which is why, to cut a long story short, I wrote my paper and posted it on DKos, and why I ended up being hired by Mitofsky to re-run the analyses.

      And it was those that convinced me that a) there was strong evidential support for the selection bias hyopthesis and b) that the fraud hypothesis was not only not supported, but that the evidence if anything contra indicated widespread, vote-switching fraud, on a scale of millions at least,.

      It does not rule it out fraud at any scale - there is plenty of slack in the data for ad hoc fraud.  And there are many forms of fraud - or electoral injustice - that wouldn't show up in exit polls anyway, including voter suppression, which I think was a big factor in 2004, and probably always is.

      And to take your final question:  "is the hypothesis that "Exit polls are not reliable" at all falsifiable?": well, it's an oddly formulated hypothesis.  Generally it is difficult to falsify a negative.  It would always be possible to conduct an unreliable exit poll, so as a general statement, the hypothesis is true.

      As a hypothesis it also suffers from the fact that "reliability" itself is something that can always be falsifiable.  A car can be 100% reliable only until the day it fails to start.

      What you probably mean, though, is "is it possible to falsify the hypothesis that an exit poll discrepancy was due to bias in the poll?"  And I think the answer is probably: only where the discrepancy was extremely large.  This is why I, essentially, I think they are poor tools for monitoring election.  If, for example, you were 100% confident that you had truly sampled every Nth voter, and you had 80 responses out of 100 voters selected, and 60 of those responses were "Democrat" and 20 were "Republican", and the vote count at the precint was 60% republican, your poll would give Democrats a winning margin of 50 points, and your official margin would be a winning Republican margin of 20 points.  Even if all your refusers were Republican, then at most you would have had a "true" sample of 60 Dems and 40 Reps (Dem margin of 20).  And that would give you pretty slam dunk evidence of fraud.

      But you would, as I said, have to be 100% confident that you had no selection bias - and I'm not sure you could ever have that confidence.

      So, in conclusion, my view is that while an exit poll that was actually designed to monitor an election, and conducted with great care might give you reason to postulate fraud, if their was a significant discrepancy, the fraud would have to be pretty egregious for you to be confident that the bias wasn't in your poll.  Which is possible.  But there are better ways of monitoring elections, which is why I support HR 550 and mandatory random audits.

Meteor Blades, Chris Bowers, emptywheel, Al Rodgers, Greg Dworkin, Spit, JekyllnHyde, RF, CJB, Sean Robertson, Ben P, Jackson L Haveck, CalifSherry, Trendar, Adam B, lipris, Tulip, David, Alan in Phoenix, Dump Terry McAuliffe, thebes, Rolfyboy6, Pandora, FaithAndReason, HarlanNY, Mullibok, maynard, Dems2004, AwsiDooger, leberquesgue, cos, Blue Shark, martianchronic, Jackson, DCCyclone, philinmaine, recentdemocrat, WI Deadhead, ilona, julatten, bumblebums, exNYinTX, Jerome a Paris, louismg, TrueBlueDem, Plan9, sardonyx, kissfan, bara, smintheus, Davidsfr, Doc Allen, Janie, Welshman, srkp23, Ryno, javelina, masoregonian, BruinKid, matt2525, Terre, David Boyle, DoctorScience, dmsilev, Cedwyn, nio, mayan, Eddie in ME, RallyPoint, brainwave, cometman, cdmc2, niteskolar, Catte Nappe, hazzcon, renaissance grrrl, Eddie Haskell, kismet, DMiller, outragedinSF, AnonymousArmy, KateCrashes, davybaby, CanYouBeAngryAndStillDream, Marcus L Salyer, d to the f, bablhous, Grahamdubya, weirdmusic657, thereisnospoon, ZaBlanc, Julie Gulden, gradinski chai, Fabian, Duke1676, cantwait08, red clay dem, Lenawee Liberal, ignorant bystander, PBen, ejmw, Cmyst, Brooke In Seattle, YucatanMan, reflectionsv37, AnnArborBlue, Morrigan, lofti, jmonch, sheddhead, Phil S 33, SheriffBart, hilltopper, aaronburr, Marcus Junius Brutus, power model, sbdenmon, Shaking the Tree, dsteffen, doe, Land of Enchantment, Asinus Asinum Fricat, CCSDem, Denny in Seattle, Starseer, kestrel9000, dougymi, scubaix, NBBooks, arbiter, plf515, BarbaraB, oakroyd, blitz boy, va dare, Statusquomustgo, 73rd virgin, frenchman, pinkhardhat, Abraham Running For Congress When I Turn 25, wildNwonderful, possum, gloriana, vets74, BruceMcF, martin o gonz, kath25, foreign noise, RosyFinch, DemJack, bruinbown, Big Tent Democrat, wakemeup7nov06, Wreck Smurfy, Progressive Chick, JML9999, FatOldGuy, ucla grad102

Subscribe or Donate to support Daily Kos.

Click here for the mobile view of the site