RESEARCH/Steven Freeman says exit polls are reliable. So he can’t help wonder why they were so wrong on Election Day.
Steven Freeman had the flu in early November, which forced him to lounge around, watch television and surf the Internet more than he usually would.
It just so happens one of those lazy nights was Election Night, when Freeman watched, perplexed, as George W. Bush won re-election—despite exit poll numbers that had early that day all but predicted a John Kerry win.
“I know exit polls have a lot of advantages over most other surveys, and so I was kind of wondering how they could be that far off,” says Freeman, a visiting scholar and affiliated faculty member in Penn’s Center for Organizational Dynamics. “I was waiting for the news to report on it, but there were no reports.”
So Freeman decided to write his own. A couple of months later, he found himself knee-deep in research about the discrepancy between the exit poll results and actual voting returns in three key battleground states—Pennsylvania, Ohio and Florida. His work has been cited by newspapers across the country and he’s been a target for both criticism and praise. “It’s really not my area, but I know how to do research, so I researched it,” Freeman says. “I figured there was something wrong with the polls, like everybody … As I investigated more, I pretty much became more and more perplexed that there didn’t seem to be any good reason for the exit polls [results].”
Freeman has reached some conclusions: He says exit polls are fundamentally sound; the deviations that occurred on Election Night could not have occurred strictly by chance or random error; and no “solid explanations”have yet been offered to explain those discrepancies. Though Freeman points out he’s not alleging election fraud, he does say the issue warrants further study.
Freeman plans to produce a total of four papers on the subject. In the first paper, which has been updated since it first appeared on his web page, he writes, “as much as we can say in social science that something is impossible, it is impossible that the discrepancies between predicted and actual vote counts …could have been due to chance or random error.”
With conclusions such as that, it’s no surprise political blogs soon picked up on the research. The bloggers were followed by mainstream media outlets. Still, Freeman says he’s disturbed that no major media outlet researched the exit-poll discrepancy on their own—and that some of them did no more than attack his work. He says there may be a lesson in the media’s reluctance to investigate the story.
Penn, he says, filled an important niche when it launched FactCheck.org—which monitors the accuracy of political claims—and he says another niche may be waiting to be filled. “The level of discourse on the subject was just so low in the news,” Freeman says. “To anybody who knows basic surveys of basic statistics, it was just absurd what was being put off as an explanation. It makes you think there’s a little bit of a hole in our society … and maybe the university should offer to fill it.”
Originally published on January 13, 2005