Civilifications Dave Denison , April 25, 2024

The Modelers Have No Clothes

We might as well flip a coin The Baffler

Yale economist Ray C. Fair, in the introduction to his 2002 book Predicting Presidential Elections and Other Things, takes note of W. H. Auden’s lines “Thou shalt not sit / With statisticians nor commit / A social science.” “I am giving the opposite advice,” Fair writes. “Come sit with statisticians and social scientists a while and see what they can do.”

In my heart, I want to go with Auden over Fair. Could there be a less enticing seduction than “come sit with statisticians”? But every election cycle, the social scientists lead me into temptation. I pull up my chair, tug at my shirt collar, and watch in fascination as they unveil their models.

Fair has been at this a long time. In the 1970s, he delved into studies on voting behavior that suggested economic conditions determine election results. For a 1978 paper in the Review of Economics and Statistics, he created a model to show the effect of economic events on presidential elections. He has continued to update the model every four years based on new election data. The key insight is not a bad one: “We must go beyond simple polling results to learn about the factors that influence voting behavior,” he wrote in his 2002 book. While a great majority of voters tend to be reliable partisans, there are always some who vacillate between the parties—and whether they are satisfied with economic conditions is likely a major factor in how they vote. Fair has developed an equation that uses the growth rate in the economy and a measure of inflation, as well as how many “good news quarters” have occurred during the president’s term, based on the theory that people tend to remember peak economic moments more than average ones. He takes into account the well-recognized advantage that incumbents have when running for reelection. Thus, he expected the Democrats would be in trouble in the 2016 election. The fact that Hillary Clinton had no incumbency advantage, along with a lackluster economy that year, suggested to Fair she would get about 44 percent of the two-party vote and lose the election. As it turned out, she won 51 percent, but still lost in the Electoral College, with 232 votes to Trump’s 306. (Clinton got about 48 percent of the total popular vote when minor candidates are included, but Fair and others often try to predict the two-party vote, to focus on the margin between the Democrat and the Republican.)

These days, there are many economists, political scientists, and data nerds who use sophisticated modeling for their “predictive analysis.” As the field has evolved, different combinations of variables have been tried. One thing that has become clear is that predicting who will win the majority of votes nationwide does not predict who will become president, due to the unfortunate existence of the Electoral College. And as important as the state of the economy is in determining the mood of the electorate, a model’s basic equation, as Fair recognized, needs to reflect the advantage of incumbency for a first-term president running for reelection but also the likelihood of voter exhaustion after the same party has governed for eight years. You would think the general approval/disapproval rating of an incumbent president would matter—but how much? (Fair does not factor in approval ratings.) What if there is a strong third-party candidate in the mix, as there was in 1992 when H. Ross Perot took about 19 percent of the popular vote? What if there is a larger than expected voter turnout, as there was in 2020? Already we see the problem: there are any number of variables that could be important, and all kinds of ways to try to quantify them and balance them against each other.

Undaunted, Moody’s Analytics has developed a model that seeks to improve on the work of Ray Fair. The Moody’s model is designed to make an Electoral College prediction, which means it needs to reflect economic and political factors in different regions of the country. Of course, the electoral result of the majority of states is not in doubt: California and New York will vote for the Democratic candidate, and Texas and Florida will go for the Republican. The key to predicting the winner of the coming election is a matter of forecasting the decision of half a dozen states. And there is no uniform “state of the economy”—voters in Arizona may not see things the same way voters in Michigan or North Carolina do. Much will depend, as well, on the enthusiasm level in the key swing states; in fact, the Moody’s model assumes Republicans will have a turnout advantage this year, as the out-of-power party often has more success in motivating its voters.

In January, Moody’s ran the numbers. The results may surprise you: “Biden is expected to win 308 electoral votes, 38 more votes than the 270 needed to win re-election.” Biden won 306 electoral votes in 2020, but Moody’s expects him to win North Carolina this time and lose Arizona, reversing the 2020 outcomes in those two states. Moody’s also expects the winner in five states to be decided by less than one percent of the vote. You might say that sounds like an election that’s “too close to call,” but then you’d be left without a prediction. Nevertheless, by Moody’s math, Biden’s prospects hang by a few threads:

If we start flipping the results of his slim­mest victories, the loss of North Carolina and Nevada would trim his vote total to 286, still enough to achieve victory. Losing Georgia, which has 16 electoral votes, would then bring Biden to the exact threshold he needs to win a second term. Therefore, Pennsylvania appears to be the key to winning or losing the 2024 election. Losing the Keystone State’s 19 electoral votes would drop Biden to 267 votes, if he also loses North Carolina and Nevada, and 251 votes, if he also loses Georgia, swinging the election to Trump. In other words, our model suggests that the upcoming presidential election will likely be determined in Pennsylvania.

As usual in the prediction game, everything is hedged. “The election could easily flip with only small shifts in the economy’s performance, [Biden’s] approval rating, voter turnout, and how well third-party candidates do.” Biden’s approval rating, in fact, has been consistently at or under 40 percent. Only one president since Franklin D. Roosevelt has averaged a lower approval rating during his presidency: Trump. That ought to doom him, but voters can have short memories. The Republican propaganda machines will have lots of people believing the economy was in good hands with Trump until inflation ran wild under Biden. It’s not even a given that voters will accord Biden the usual edge of being a steady, tested incumbent—as a former president, Trump will be assumed (by some vacillating voters) to be tanned, rested, and ready. Overall, the Moody analysis sees political factors favoring Trump’s candidacy and economic factors favoring Biden’s.

In other words, we might as well flip a coin.

Suppose we want a modeler who looks at the numbers and then makes the call without dithering. For adventures in predictive certitude, Helmut Norpoth is your man. Norpoth is a Stony Brook University political scientist who developed what he calls “the Primary Model” in the mid-1990s. He came to believe he had found the one factor that was more important than all the others: how the presidential candidates fare in the early primaries, he believes, predicts who will become president. It’s especially important in the case of the incumbent. “A sub-par primary performance by a sitting president spells doom in November,” he has written. His model ignores the state of the economy and the approval ratings of presidents and candidates, although he does factor in the incumbency advantage and the usual “time for a change” sentiment after the same party holds the White House for two terms.

Leading up to 2016, Norpoth claimed the stronger candidate in the primaries had won twenty-five out of the last twenty-seven elections. The two exceptions were John F. Kennedy’s narrow win in 1960, and George W. Bush’s contested victory over Al Gore in 2000—both of which can be seen as dubious results. So, when the New Hampshire primary in 2016 gave a strong victory to Donald Trump and a resounding loss to Hillary Clinton (who came in well behind Bernie Sanders), Norpoth made the call. In March of 2016, before either candidate had locked down their party’s nomination, he announced that the Primary Model predicted Trump would win over Clinton “with 87 percent certainty.” If Sanders were to go on and win the nomination, Trump was forecasted to win with 99 percent certainty.

Norpoth’s early prediction won him fawning coverage in the right-wing press. The New York Postcalled him a “prediction wizard” who was “scary accurate.” The Postquoted him a few weeks after the New Hampshire primary saying the probability of a Trump win was “almost ‘take it to the bank’.” Even as polls that summer and fall showed Trump lagging behind Clinton, Norpoth didn’t waver. The Primary Model had spoken, and the results were, in his words, “unconditional, final, and not subject to updating.” When Trump emerged victorious, it resulted in even more acclaim for Norpoth as an election oracle. Lost in all the hype was an important detail: Norpoth’s actual prediction was that Trump would win the popular vote. In fact, his “scary accurate” model forecasted a final result of Trump with 52.5 percent of the two-party vote and Clinton with 47.5 percent. (Actual result: Clinton 51.1 percent, Trump 48.9 percent.)

While Norpoth quietly acknowledged that error, he moved forward as if he believed his own press. For the 2020 election, he decided he would use the results of the early primaries—New Hampshire for both parties, with the addition of South Carolina for the Democrats—to predict the presidential winner in the Electoral College. (The GOP cancelled its South Carolina primary that year.) Trump had the advantage of incumbency and he had only token opposition—former Massachusetts governor William Weld, who received about 9 percent of the vote in New Hampshire to Trump’s 84 percent. Meanwhile, Biden came in fifth in New Hampshire with a dismal 8.4 percent of the vote—Sanders led with 25.7 percent. Biden rebounded in South Carolina, but still came in with less than 50 percent of the vote there. With that, Norpoth had seen enough: on March 2, eight months before the election, he announced that the Primary Model gave Trump a 91 percent chance of besting Biden. Trump would get 362 electoral votes to Biden’s 176. (Actual result: Biden 306, Trump 232.)

Norpoth retired from his Stony Brook University post last year, yet the Primary Model lives on. After this year’s New Hampshire and South Carolina primaries, it was clear that Biden’s results were much stronger than Trump’s. While Biden won almost 64 percent of the New Hampshire primary vote, Trump took only 54 percent. And in South Carolina, Biden won 96 percent while Trump came in just under 60 percent. Consequently, the Primary Model, updated on Norpoth’s website, gives Biden a 75 percent chance of winning reelection, with an expected 315 to 223 victory in the Electoral College. Oddly, the New York Postand Fox & Friends, et. al have taken no notice of this latest Primary Model forecast.

The notion that social scientists can predict the future with more accuracy than those who go on mere hunches—or those who read aggregated polls, or tea leaves—is most dearly held among those who see themselves as trained experts or political professionals. It rests on the same idea of expertise that propels the business of political consulting, public opinion polling, and journalistic punditry: that if you study patterns enough you can understand why things happen the way they do. Understanding that, you can predict the future. And most people like the feeling they can see where things are going. Think again of the Moody’s analysis that states the presidential election “will likely be determined in Pennsylvania.” If you confidently repeated that in a dinner conversation, you would sound like you are “in the know.”

Do these predictions have an ill effect on the body politic? There’s an annoying hubris to them, but it’s not markedly worse than that of the guy with a podcast who sees exactly how events are lining up and knows “in his gut” how the election will play out. It’s probably true that the people most likely to give credence to a modeler’s forecast are the ones who want to believe in what it says. But that can have real-world effects. It’s easy to imagine discussions in the White House earlier this year about whether Biden should stay in the 2024 race. Then Moody’s Analytics releases its forecast giving him the nod. An aide rushes the news into the Oval Office. Biden nods sagely. He knew all along. If more evidence is wanted, the aide might remind the boss that, on top of that, Ray Fair—the dean of election modeling—says that because of the state of the economy the Democratic candidate is likely to get 51 percent of the two-party vote in 2024.

The game gets a little funny, though, when our supposedly scientific forecaster is forced to ask why the imperfect, messy, and sometimes chaotic world does not conform to their carefully crafted model. And that’s when you see them admit that there are many important factors that can’t be made to fit into their equations. Ray Fair touches on the problem in Predicting Presidential Elections. As he gathered data from twenty-four presidential elections, he was able to make a plausible case that economic conditions were a consistently important variable. Plentiful economic data exists for every election year. But other factors only affect a few elections—as was the case in 1992 when Ross Perot was a significant third-party factor. What Fair’s model needs is for things to happen pretty much in the way they’ve happened over the last one hundred years. As he explains it, “possible shifting behavior is a nightmare for social scientists trying to explain behavior over time because stability is needed to learn very much.”

And yet anyone watching events unfold has to be aware that, as Emory University political scientist Alan Abramowitz said in 2020, “presidential elections no longer work the same way they have in the past.” He was referring to the stronger partisanship in a time of polarized politics: there aren’t as many swing voters who are open to either party. And with the nation almost evenly divided between Republicans and Democrats, presidential elections are decided by tiny margins, making prediction much more dicey. It’s also true that, even as a rematch, this presidential election could be unlike any other. Trump is always an unknowable factor. There is a good chance Robert F. Kennedy Jr. could exceed the usual tiny vote totals that marginal candidates get. Chances are probably slim that the Republican candidate will be campaigning from a prison cell, or that the Democrat will be confined to a hospital bed. But there is a sense that just about anything could happen.

After Helmut Norpoth made his prediction that Trump was 91 percent certain to win the 2020 election, he felt compelled to offer a public explanation of why his Primary Model was so far off the mark. What he wrote on his website says a lot about the perils of prediction. “What went wrong?” he asked. “The answer is simple and apparent: a perfect storm.” Norpoth lists “a cascade of election-year ‘surprises’” that he considered to be “unprecedented”: the pandemic affected the election by causing a sudden economic downturn; there was an expansion of voting by mail, which led to higher-than-usual turnout; even the widespread protests after the police killing of George Floyd may have been a factor in energizing voters for Biden. He notes that Trump had low public approval ratings, but he doesn’t quite acknowledge another possible influence on voters’ decisions: that Trump was a singularly chaotic, corrupt, and incompetent president, for instance. Still, in attributing the election result to any number of factors that are ignored by his Primary Model, he gives away the game. If it was a “perfect storm” that hit in 2020, couldn’t you say that about the 2016 election too? Some freakish set of events put a mountebank like Trump in office. At the least, a lot of things went wrong for Hillary Clinton. And isn’t it likely that the result in 2024 will depend on some new cascade of election-year surprises?

Moody’s Analytics also wrongly predicted a Trump victory in 2020. Recently, Moody’s chief economist Mark Zandi spoke on the BeyondPoliticspodcast about this year’s Moody’s forecast. Notwithstanding his model’s current prediction of a Biden victory, he listed a few factors that might cause Biden to lose: a rise in gasoline prices, higher mortgage rates, and whether a third-party candidate emerges with significant support. Bottom line: Biden is slightly favored—unless things don’t break his way.

“I strive to be 50 percent right,” Zandi said at the end of the podcast. I’m pretty sure he was joking. But I’d say that’s a realistic goal for anyone who wants to claim expertise in predicting presidential elections.