Return of the People Machine

Even a halfway-decent political campaign knows you better than you know yourself. A candidate’s army of number crunchers vacuums up any morsel of personal information that might affect the choice we make at the polls. In 2020, Donald Trump and the Republican Party compiled 3,000 data points on every single voter in America. In 2012, the data nerds helped Barack Obama parse the electorate to microtarget his door-knocking efforts toward the most-persuadable swing voters. And in 1960, John F. Kennedy had the People Machine. Using computers that were 250,000 times less powerful than a modern MacBook, Kennedy’s operatives built a simulation of the presidential election, modeling how 480 types of voters would respond to any conceivable twist in the campaign. If JFK made a civil-rights speech in the Deep South, the People Machine could, in the words of its creators, “predict the approximate small fraction of a percent difference that such a speech would make in each state and consequently … pinpoint the state where it could affect the electoral vote.”

But you don’t hear Nate Silver talking about the latest People Machine forecast, because it was, in fact, all bogus. The simulation—part hucksterism, part hubris—promised a lot but delivered little, telling the Kennedy campaign nothing it didn’t already know. “The People Machine was hobbled by its time, by the technological limitations of the nineteen-sixties,” the Harvard historian Jill Lepore writes in The New Yorker. “The machine sputtered, sparks flying, smoke rising, and ground to a halt.” Instead, the best way we have to actually predict elections is still the jumbled mess that is polling. Because reaching people has become harder than ever for pollsters, so has the job of figuring out who is going to vote, and for whom. If the polls had been spot-on, Trump would never have been president, and just hearing the phrase election needle wouldn’t make any liberal’s skin crawl. Polling can still sometimes nail an election, but the problems are real: In 2020, presidential polls had their biggest miss in 40 years, and what was predicted to be a quick win for Joe Biden turned into an excruciating four-day squeaker.

You can see why there’s an urge to find a better way. The idea of mimicking voters with tech may have been fantastical when JFK was running for president, but it seems far less so in the age of hyped-up AI chatbots that talk in a confident, natural way. (Dear Bing, please leave my relationship alone.) Instead of polling humans, it’s now theoretically possible to poll bots that emulate humans. When researchers at Brigham Young University fed OpenAI’s GPT-3 bot background information on thousands of real American voters last year, it was unnervingly good at responding to surveys just like real people would, for all their quirks, incoherence, and (many) contradictions. The fake people were polled on their presidential picks in 2012, 2016, and 2020—and they “gave us the right answer—almost always,” Ethan Busby, a political scientist at BYU and a co-author of the study, told me.

So while ChatGPT can spit out anything in the voice of Shakespeare or Shakira, this technology can seemingly also simulate whole groups of voters—MAGA zealots, suburban wine moms, and elderly Black churchgoers alike. Yes, for now, it’s an academic experiment. But considering the woes of polling, the idea of turning to bots might seem pretty appealing to cash-strapped political apparatchiks trying to gauge how their candidate’s doing. A high-quality political poll can run $20,000 or more, but this particular AI-polling experiment cost the BYU researchers just $75. The People Machine, it seems, has whirred back to life.

Here’s an easy way to think of the problems with polling right now: Can you remember the last time you picked up a phone call from a random number? Many of the best public-opinion surveys still involve actually calling people, but virtually no one is willing to answer the questions anymore. New York Times/Siena College is the Ferrari of polls, and its response rates have dipped as low as 1 percent in recent years, requiring two hours of dialing for a single completed interview. Polling is always a game of extrapolation—in a national survey, thousands of people need to tell you something about hundreds of millions of voters—but the information is so bad that using it to predict an outcome within a few points is a bit like trying to sink a half-court shot, blindfolded, after shotgunning five beers.

But chatbots can be programmed to answer every question you want every single time. Because so-called large language models have ingested basically everything on the internet, these bots have a firm sense of our kaleidoscope of political views. And they are exceptionally good at “mirroring what people think and how they speak and behave,” Lisa Argyle, a BYU political scientist and the lead author on the AI-polling study, told me. ChatGPT may refuse to talk politics with you, but I got it to play a 40-year-old white man in rural Ohio with pro-gun, anti-abortion views. A sample of the output: “The Democrats want to take away our guns and kill innocent babies.”

Sure enough, in the paper, which was published recently in the journal Political Analysis, that type of humanlike behavior is precisely what emerged when the researchers fed the machine the backstories of thousands of real voters from the past three presidential elections and asked them whom they would prefer in each election. Those backstories were each composed of 10 biographical tidbits from people who had responded to a major postelection study, including their basic demographics (race, gender, state of residence) and other aspects of their identity (church attendance, feelings about the American flag, interest in politics). And then the bot spit out a probability that such a person would prefer the Democratic or Republican presidential contender.

Think about it this way: Give a bot the prompt “professional football team from Cincinnati,” and it knows to respond with “Bengals,” not “shawarma,” because it is making connections based on all the text that has been stuffed into it. That same approach also seems to work for political views: The bot told the researchers that a 29-year-old white man from Louisiana who is a strong Republican and regularly attends church would have had a 96 percent chance of voting for Trump in 2016, because its algorithms determined that words like Trump, Donald, and Republican were far more associated with that profile than words like Hillary, Clinton, and liberal.

And this approach worked exceptionally well in the experiment. For all three elections, bots matched the preferences of real voters at least 85 percent of the time—sometimes with alarming accuracy. In the AI poll of the 2012 election, bots predicted that 39 percent would vote for Mitt Romney. In the real 2012 poll, it was 40 percent. In the 2020 poll, the bot’s predictions matched the responses of 90 percent of real voters who said they didn’t attend church, and of 94 percent of Black voters. “It can mimic human behavior with astonishing accuracy,” Busby said. “And that holds no matter how you slice it up—whether we looked at specific subgroups or looked at different residents of different states, like swing states.”

Read: ChatGPT changed everything. Now its follow-up is here.

A swirl of data and algorithms can, of course, never account for the full range of human weirdness, let alone with just 10 tidbits of the most elementary personal information. (Who knows why people wrote in “Ur mom,” “Cheddar,” and “Can’t Do It” rather than voting for real candidates in the 2020 election?) True independents are a total crapshoot with AI, just as they are “the hardest to predict in any polling situation,” Argyle said. But in aggregate, bots can get the proportions right for many different personae. GPT-3 is good at predicting political views because American politics is, in a sense, very predictable. You don’t need AI to know that in 2016, a 58-year-old Black Democrat from New York City would likely have preferred Clinton to Trump.

But even voters who are much harder to read can be parsed by AI. Quick: Did a 30-year-old Mississippi woman who identified as “slightly liberal” and expressed a strongly favorable view of “seeing the American flag flying” vote for Clinton or Trump? GPT-3 says 75 percent for Clinton. The bot tries to make an educated guess based on connections that we mere mortals might not see; this skill is why AI, for all its foibles, may be adept at understanding our opinions. “We are not built to identify granular patterns between very small signals, but these machines are very good at doing that,” Deborah Raji, an expert on AI bias at the Mozilla Foundation who wasn’t involved with the study, told me. Consider the problem of “nonresponse bias”: pollster-speak for the idea that certain groups, particularly Trump voters, are especially unlikely to respond to pollsters, skewing poll results. “It’s possible that using AI can help us supplement or fill out or get a more nuanced understanding of some of those populations where we just have a really low response rate anyway,” Argyle said.

None of this means that we are barreling toward a world in which AI polling can fully replace the real deal. Candidates can’t just fire up a chatbot and hone their stump speech after testing it out against AI versions of evangelicals or South Florida Cuban Americans. “When you emailed [the paper] to me, honestly my first reaction was, Is this a joke?” Joshua Clinton, a political scientist at Vanderbilt University and co-director of the Vanderbilt Poll, told me.

Read: Prepare for the textpocalypse

Clinton laid out a sprawling list of problems with AI polling. OpenAI’s latest bot, GPT-4, is brand-new, but from a polling standpoint, it is already outdated. Bots can’t learn anything new unless they’re trained on a corpus of new data, but the whole point of polling is to gauge how views are changing. If you want to know how Republicans are responding to Trump’s indictment, polls can tell you that; ChatGPT cannot. And these models are trained only on what people write online. The internet is swarming with socialist Reddit posts and brain-poisoning Facebook memes, but these are hardly representative of the electorate as a whole, Raji reminded me. When researchers at Stanford compared the views of chatbots with those of 60 different demographic groups in the U.S., they found “substantial misalignment”—especially among certain specific groups, such as Mormons and widows. “One person, one vote” is very different from “One post, one vote.”

But polling may still not totally evade the chatbot revolution, either—particularly at a time when every company and industry so badly wants to tout its “AI-powered” whatever. “Over the past 30 years, polling has been completely transformed by changes in computing and communications,” Barbara Carvalho, the director of Marist College’s poll, told me. “So my expectation is that polling will absolutely be affected by AI.”

Polling is already a data scientist’s fantasyland, stuffed with Ph.D.-level statistics and machine learning; simpler forms of AI were helping predict elections well before high schoolers were using ChatGPT to write English essays. Other researchers have fed chatbots text from Fox News and produced responses that can mimic humans. One company, Synthetic Users, is polling bots for use in marketing. But Raji cautions against taking this too far: “Good intentions aside, I think ultimately the way it will likely play out is just less investment in the actual engagement of real people,” she said.

Bots cannot—should not—replace people wholesale, but that doesn’t mean bottom-shelf pollsters who already rely on shady methods won’t try to use AI to pump out even more bad polls to sway media attention and campaign contributions. In that sense, AI polling is perhaps just a fun-house mirror of the future of chatbots: What seems like a way to solve one problem sometimes just begets another, and another, and another.