The Playbook: Test More, Test Better.

We’re doing a series on how Tavern works. Most campaigns test what they already believe. Here's why that's not the same as learning.

April 1, 2026 | 5 min read

What Message Testing Actually Is

Before a campaign puts an ad on air, it should know whether that ad moves voters. Message testing is how you find out. You write a message, you put it in front of a sample of real voters, and you measure whether it changed how they think or feel about the race. That's the basic idea. The gap between that basic idea and how most campaigns actually execute it is where most of the money gets wasted, and where good ideas are left un-thought-of, un-written-down, and un-tested.

How It Usually Works

A campaign sits down with its pollster and writes a handful of messages. Maybe six, maybe a dozen. They reflect what the team already believes: the candidate's biography, the opponent's record, a few economic themes that have worked in similar races before. The messages go into a survey. A sample of voters reads them and rates which ones feel most persuasive. The top two or three come back as winners. The campaign builds ads around them and goes on air.

Here's the problem. You tested the messages you already believed in and found out which of your existing instincts was slightly more right than the others. When one of them wins, you try to figure out why. Maybe it mentioned healthcare. Maybe it had a stronger opening line. Maybe it used the word "fight." You make your best guess and build your strategy on top of it.

That's observational analysis. It tells you what happened inside a specific test. It doesn't tell you what caused it. And it's bounded by what you thought to write in the first place. If nobody on your team thought of the message that would have actually broken through, you never find it.

How To Figure More Things Out Better

The thing that changes what you can actually learn is treating content generation as part of the experiment itself.

Instead of writing a handful of messages and analyzing them after the fact, you isolate individual variables and test them systematically. Do ads with subtitles outperform ads without? How much does a candidate's personal story move voters, and which part of it: where they grew up, their first job, their family? What happens when you change a single word?

By isolating each variable and measuring its specific impact, you build knowledge that compounds. The difference is between knowing "Message A performed well" and knowing "Messages that lead with economic security gain three points among swing voters, and that effect holds across different candidates, geographies, and election cycles." The second finding is something you can build strategy on top of. The first is a data point that expires when the race ends.

What Artificial Intelligence Changes

Here's where the technology matters, and it's worth being specific about why.

When a team of political strategists writes messages, they bring everything they know: the candidate's record, the district's history, the themes that have worked before. That knowledge is valuable. It's also a constraint. People write toward what they already believe. They use the language they've always used. They test the ideas that already feel right.

Democrats almost always describe wealthy opponents as "millionaires and billionaires." That's the phrase. It's been the phrase for thirty years. Artificial Intelligence generates messages without that accumulated habit wrote "very wealthy people" instead. When we tested both, the results were different. The assumption that "millionaires and billionaires" was the obvious right call turned out to be exactly that: an assumption. One that had never been tested against a real alternative because no human had thought to write one.

That's what removing human bias from message generation actually means. The technology isn't smarter than your team. What it does is generate and test ideas that exist outside the frame your team brought into the room. You stop confirming what you already believed and start discovering what you didn't know.

What We Can Actually Do

Run enough tests, isolate enough variables, and you discover things that upend what you thought you knew.

After roughly 1,500 randomized trials, we found that attention and persuasion are uncorrelated. What grabs someone's attention and what actually changes their mind operate independently. An ad can be visually compelling and move no votes. An ad can be understated and shift a race. The two dimensions have almost nothing to do with each other.

That finding only surfaces through systematic testing. An observational approach would have missed it entirely. You'd see which ads performed well and guess about why.

But here's what makes that finding valuable beyond any single race: it transfers. Once you know attention and persuasion are independent, you know it everywhere. It shapes how you approach the next test, and the one after that. The knowledge compounds.

This is what most testing programs miss. A consultant tests messages for one campaign, picks a winner, and moves on. The learning dies with the project. The next campaign starts from scratch.

We run more than 20,000 randomized trials a year. Every one feeds back into a growing body of knowledge about what actually moves voters, across candidates, geographies, and election cycles. The test we ran last cycle makes this cycle's test smarter. The findings don't expire when the race ends.

That's the closed loop. Test. Learn. Build on what you learned. Test again. Each pass produces sharper predictions than the one before.

A scoreboard doesn't win games. Measurement without the ability to act on it is just being right while you lose. The loop matters because it builds toward something: a system that tells you what will work before you spend money finding out.

That's what we're building. Nobody else is.

Want to see the loop in action? Email data@tavernresearch.com →