TORONTO — The demand for exit poll data has always outweighed the exit poll’s predictive accuracy.
The network-sponsored, election-night exit polls serve two purposes: guiding the news media as it covers national elections in real time and serving as a standing record of the composition of the electorate and what they thought about the candidates and issues.
Story Continued Below
But their problematic history has spurred a schism among America’s top news organizations, which have spent decades pooling their resources to conduct one universal exit poll. Now some outlets have splintered off to launch their own project, which debuted in last year’s midterm elections.
The 2020 elections will feature two parallel polls of voters designed to measure who voted, for whom they cast their ballots and why. And both providers — the National Election Pool and its exit poll, along with The Associated Press’ new VoteCast project — claim they’ve solved many of the problems that have confronted exit pollsters for decades.
It’s led to something of an exit polling arms race — with both camps, while collegial and collaborative in general, claiming their recent work has produced more accurate data for news organizations to project elections. The war of words kicked off here at the American Association for Public Opinion Research’s annual conference, where papers were presented at back-to-back panel discussions on Saturday.
“I do believe we have better data,” David Scott, AP’s deputy managing editor for operations, said in an interview.
“We achieved what we wanted to do in the two things that VoteCast is really designed to deliver,” Scott said. “We need really great data on election night to power the AP’s decision desk and to tell the story of the electorate and why the winners won.”
But new doesn’t actually mean better, said Joe Lenski, vice president of Edison Research, which conducts the exit poll for the TV networks.
“I think the exit poll data is the data of record for election analysis,” he said, referring to the work by media outlets and academics well after the election occurs.
The history of past exit poll failures is well-documented, even if the slip-ups aren’t actually that frequent. If the exit polls in 2004 were accurate, John Kerry would have been elected president. More recently, exit poll data suggested Hillary Clinton would defeat Donald Trump in 2016.
As a record of the composition of the electorate, the exit polls have been plagued for years by a number of historical biases — namely the over-representation of younger and more-educated voters, which, in recent elections, has led to a Democratic skew in the results.
And the exit polls have been under siege from external forces, too, with more voters nationally casting ballots before Election Day or through the mail, making them unreachable by interviewers positioned outside polling places. In general elections now, roughly four-in-10 voters cast their ballots in ways other than appearing in person on Election Day.
So now there will be two exit polls. If you watch the election returns on the three major broadcast networks, CNN or MSNBC, you’ll see the traditional exit poll that mostly consists of interviewers approaching voters leaving their polling places on Election Day. But if you’re watching Fox or the Fox News Channel or consuming news from any of The Associated Press’ thousands of affiliates across the country, you’ll get VoteCast.
Moreover, the members of the National Election Pool — ABC News, CBS News, CNN and NBC News — also get their election results from Edison, while the vast majority of other news organizations (including POLITICO) get their results from The AP. That means if the polls or the election results differ, Americans might get two diverging pictures of the election as results pour in that Tuesday night in November 2020.
But researchers on both projects say they’ve cracked the code, producing results on Election Night 2018 that were, on average, only 1 percentage point off from the final vote count. At a panel discussion here Saturday, Courtney Kennedy, the director of survey research at Pew Research Center, proclaimed that 2018 “improvements” and “innovations” to the traditional exit poll were “tremendous.”
“You made the national exit poll more accurate,” said Kennedy.
Meanwhile, the AP and its partner at NORC, a research institution that’s part of the University of Chicago, are celebrating what they consider a wildly successful 2018 launch here this week. They’re throwing a party at the Hockey Hall of Fame, featuring with an open bar and photo opportunities with the Stanley Cup, the sport’s iconic championship trophy.
While both are polls of voters and serve similar purposes, there are important differences between them. The National Election Pool’s exit poll combines the traditional exit polling approach — interviewing voters on their way out of the polling place — with telephone surveys in states with all-mail voting or robust in-person early voting.
Edison made two key changes in 2018, Lenski said. It experimented with in-person interviewers at large, early-vote centers in Nevada and Tennessee. And most importantly, Lenski said, Edison tweaked the way it asks voters about their own levels of educational attainment and adjusted the survey to reflect a truer balance along educational lines, producing a more accurate result early in the evening that carried through until all the votes were counted.
“We felt that those adjustments we had made based on education have done a really good job of giving us top-line numbers at 5 o’clock and at poll closing that told the story,” Lenski said in an interview. “Our members had the story on election night. We knew at 5 o’clock it was going to be a really good night for Democrats in the House and not-so-good night for Democrats in the Senate.”
AP VoteCast, on the other hand, looks more like a typical pre-election poll, combining traditional, probability-based polling with an online, opt-in survey of voters. VoteCast — which involves more complicated statistical modeling than the traditional exit poll — includes both those who say they voted, and those who say they didn’t.
The 2018 midterms marked VoteCast’s debut, and Scott, the AP editor in charge of the project, says it couldn’t have gone much better. AP and NORC researchers will present their results here on Saturday; in a public report, they’re calling VoteCast’s launch “an impressive success.”
“At that period between 5 o’clock on the East Coast and when polls close, the data for VoteCast was showing the right winner in 92 percent of the races that we looked at,” Scott said in an interview. “And our average margin of error was around 1 percent — still leaning toward the Dem, but still just a 1 percent margin of error. At that same time in our history with the exit poll — especially our recent history with the exit poll — you know, we’re looking at average errors of double digits in some states.”
The next test for VoteCast will be the 2020 presidential primaries and caucuses — contests that pose unique challenges. Unlike in general elections, primary electorates shift drastically from cycle to cycle, and there are few historical benchmarks to use in calibrating to get the right mix of voters. And since VoteCast is conducted beginning in the days before the vote, late-breaking events in the campaign — a candidate dropping out in the days between primaries in different states, for example — may be harder to measure.
“The scalability and infrastructure works a lot better when we’re doing all 50 states all in one night, as opposed to a series of events,” Scott said. “There’s also the technical: We’re in the field, looking for early voters at the same time that you might have an election going on.”
Lenski, who leads the traditional exit poll, thinks he has an advantage come next year’s primaries.
“I can’t speak for how AP VoteCast is planning on doing this, but I know the type of modeling they use in a general election is much tougher to do in a primary because there’s no really good history for a 23-candidate Democratic race to model what the electorate is going to look like,” he said.
“I know what we do works for a primary,” Lenski added. “It’s worked for 40 years. And we have the infrastructure in place to do event after event, week after week. It’s going to be a real grind. Starting February 3 with Iowa, going through March 17 — it’s just about every Tuesday, and sometimes Saturdays, and sometimes Mondays — there’s going to be an event. And we have the infrastructure in place to cover state after state.”
The 2020 presidential race consists not of one Election Day but a series of elections, beginning with the Iowa caucuses. That increases the chance, with different sources of voter polling and election results, that the trajectory of election night could look different based on where Americans get their news — a dangerous turn given Americans’ already declining faith in institutions of government and media, sometimes amplified by political leaders.
“I think that’s possible,” said Emily Swanson, the AP’s polling editor. “And I think that we would have a lot of confidence in the story that we’re telling.”
But Lenski maintains the traditional approach used for the network pool is best.
“We still believe that the best way to report voter opinions on Election Day is to actually talk to voters right after they voted,” he said.