Can an AI guess which posts you'll like?

One survey, then a simulated crowd of millions

Every participant answered a survey and reacted to 56 posts. Their answers became persona prompts at three levels of detail, each fed to 27 models. The agents then faced the same posts — producing a behavioral benchmark at a scale no lab could run with real users.

The model you pick matters more than anything else

Accuracy swung 13 points from the best model to the worst — a bigger gap than richer personas, matched content, or any other design choice produced. Model choice, not persona craft, was the dominant lever.

A plain text classifier beat every "intelligent" agent

Here is the finding that reframes the whole study. On the hardest test — predicting whether a specific person likes a specific unseen post — a simple TF-IDF classifier outscored the best LLM agent. The apparent "understanding" was mostly the words carrying the signal.

The agents weren't useless — they clearly beat the structured baselines that had no access to the text. But once a conventional model could read the same persona and post text, the agent's edge vanished. The signal lived in the words, not in any capacity for agentic simulation.

The agents hear applause better than dissent

Prediction accuracy was consistently higher for positive posts than negative ones — a systematic positivity bias. For platform governance that's the wrong ear to favor: backlash and dissent are exactly what simulations most need to anticipate.

Six hypotheses, six honest verdicts

Not everything we expected held up. Richer personas barely helped, content domain flipped between models, and the agents never overtook the text classifiers.

Genuine signal, modest reach