Blaming “Analytics” Is A Cop-Out

[I really wanted to title this “The Chad Listening to Voters and The Virgin... Also Listening To Voters” but that might be too online of me.]

It’s the knife-fight portion of the post election arguing! Yay! One of the arguments made about why Harris lost, notably by John Della Volpe in this article I don’t really care for, is that there was an over reliance on data analytics instead of “listening to voters”. He seems to mostly be making a point about young voters, saying: “Democratic Party leaders did not listen deeply to and earn the trust of young voters, who could have helped her prevail in Michigan and other swing states”. Democrats did certainly lose ground with young voters this election, particular with young men. This was heavily foreshadowed before the election, by a wealth of polling noting that Trump was doing especially well with young people. 

What does Della Volpe pin this failure to win young people on? Analytics! Here’s a whole quote, because I found it particularly annoying:

“Analytics should serve a campaign like radiology in a hospital — critical but supplementary. X-rays and lab results are essential, but no one would allow radiologists to run an entire hospital without doctors who engage with patients, understand their concerns and treat them holistically. Yet Democrats have done essentially that — allowing data scientists to replace human connection with numbers, mistaking metrics for meaning and forgetting the fundamental truth that politics is about people, not percentages.”

I’m not sure what Della Volpe thinks data analytics is out here doing, but the vast majority of it is listening to voters with extra steps. He also seems to have an understanding of how many ”data scientists” are running things that is out of touch with reality. 

Polling Is Literally Listening To Voters

If you’re only consuming polling results in pretty averages and slide decks, it might feel cold and divorced from human experience. But anyone who’s worked in the industry knows that under those nice percentages, there’s a massive amount of messy human contact. One of my first jobs required me to spend time listening in on phone polls (throwback!) and hearing interviewers talk to the people we were trying to reach. Once we went to online polling, I spent tons of time reading through the open-text responses of people alternately telling us their opinions and yelling at us for our survey design. 

Why do we do things like web polling and analytics instead of “just” talking to people? Mostly, because a lot of voters really don’t want to talk to us. It has gotten harder and hard to reach people, with response rates to phone polls plummeting over time. Polling has gotten increasingly complicated to compensate for this. It used to be that you could literally call a randomly dialed set of people on the phone and call that close enough. This is no longer the case. 

Polling has been forced to confront something that’s true throughout the rest of politics: most voters don’t want to talk to you. If you base your political understanding on people who are willing to talk to you, you’re going to get a super weird sample of the electorate who tend to be higher social trust, more educated, in office jobs, and more liberal. This is not a way forward to a nuanced understanding of the whole electorate, or a way to win elections. 

Who Leads?

The other argument Volpe makes is that Democrats are letting data lead too much, and that allowing data scientists to “take over” has been a mistake. My biggest complaint is that this isn’t actually happening, and is a silly way to frame a strategy argument. 

Is it possible to get too out over your skis with data? Yes, of course, but in the same way it’s possible to lean too hard on any theory that confirms what you want to believe about the election. Is it possible for data to be wrong? Also yes, of course, see my incredible distaste for exit polls because I think they’re usually wrong. Does it look like this is what happened with Harris, more than uniform swing or a decently run campaign defeated by a tough year? Nah. 

I haven’t seen any real evidence that data was “running” the Harris campaign. Future Forward built a massive ad testing operation, sure, but that was both external to the campaign and heavily ad focused. I would love to poke through the particular Future Forward recommendations and compare to what ended up happening on the campaign, but that’s not public info, and I doubt that sort of comparison will become possible. A NYT article on the PAC talked about its focus on “gen pop” ads rather than more micro targeted efforts, which is in direct contrast to Della Volpe’s complaint about “overly tailored messaging”. 

Data and Its Limits

Data can only answer the questions you ask it. Polling can only answer the questions you ask, and sometimes not even that (polling is hard!). Blaming data, or the staff who work on it, for failures is a silly abdication of the responsibility of the people setting strategy. It’s also usually a way to say “the data says I’m wrong, but I think I’m right” in a way that sounds smart and thoughtful. 

Fundamentally, analytics teams work under the strategy set out by campaign managers, principles, or candidates. You can only test ads with the positions your candidate is willing to take, and models just guess what will happen, they don’t make it happen. The effect of running the best possible ads or using the best possible targeting is always, always going to be dwarfed by the policy choice of the candidate.

To hammer that home a little bit, the estimate of the effects of an entire campaign I usually work under is around 2-3% of vote share. This is a rough guess, and for the sake of argument I am happy to round up and call it 5%. Under any estimate, this is teeny-tiny compared to the effects of things like Biden dropping out, the flubbed debate, Trump saying he doesn’t actually want to ban abortion, etc. I firmly believe that analytics and optimization are important, but we shouldn’t kid ourselves about the effect sizes here. 

If you want to have policy arguments about what Harris did or didn’t do and how that affected certain groups, sure fine. But not everything can be a data and messaging argument. Sometimes it’s actually about policy choices, what the candidate said, not the ads and the polling. 

Data at its best gives you information about voters and what is important to them. Talking to voters is important, and since most of them would vastly prefer you go away and don’t ask them questions, you need to use statistical methods and polling methods to actually contact them. Putting “talking to voters” on a pedestal over some mythical cold and calculating data analytics is a silly way of disagreeing with the conclusions a campaign reached from the data available. Don’t blame the nerds for strategies you disagree with.