The AI industry has been quietly using language models as stand-ins for human test subjects in psychology, economics, and education research. The logic is appealing: instead of recruiting thousands of participants, just prompt a chatbot and get instant results. A massive new study with 208,000 real human participants and 26 million responses torpedoes that assumption with brutal clarity. The training process that turns raw language models into helpful assistants systematically degrades their ability to simulate human behavior. This isn't a minor artifact—it's a structural trade-off that gets worse with every model generation.
The study compared base models—those trained only to predict the next word—against their post-trained variants across three model families: Qwen3, Llama3, and OLMo 3. In every single comparison, the base model predicted what actual human participants would say better than its fine-tuned descendant. Not by a little, but consistently. The biggest distortions appear in language tasks and reasoning, exactly the domains where RLHF pushes models toward normative correctness instead of capturing the systematic biases and heuristics that define real human decision-making. The gap isn't closing—it's widening. Base models steadily improve across generations, but the delta between a base Qwen3 and its assistant version is larger than the delta between Qwen2 and Qwen3.
Here's where it gets really uncomfortable for practitioners. The experiment tested a popular workaround: prepending detailed demographic profiles—age, gender, nationality, clinical diagnoses—to the prompt, essentially trying to role-play a specific participant. The effect was practically zero. Giving the model someone's exact age and education doesn't make it predict that person's responses any better. We're not dealing with a prompt engineering problem; we're dealing with a fundamental capability that gets overwritten during alignment. The models have been trained to be helpful, harmless, and honest, and in the process they've lost the very noise and irrationality that makes human behavior predictable at scale.
The study also proves this isn't a hard limit. A model called Centaur, fine-tuned directly on behavioral data, showed much higher agreement with human responses even on tasks it hadn't seen. Targeted training works when the objective is behavioral fidelity rather than logical correctness. The problem isn't fine-tuning itself—it's the objective. RLHF optimizes for what a helpful assistant should say, not for what a human would actually say. Those two goals are increasingly at odds, and as post-training techniques become more aggressive—reasoning models, instruction tuning, vision extensions—the divergence accelerates.
For anyone building applications that rely on LLMs as human proxies—whether for user research, clinical training, or policy simulation—the message is unambiguous. Stop using the convenient assistant models. They are actively misleading. Use base models or, better yet, models specifically fine-tuned for behavioral prediction. The industry's obsession with helpfulness as the only axis of quality is creating blind spots. We're polishing a mirror that no longer reflects what it's supposed to. The RLHF paradox isn't just an academic curiosity—it's a design constraint that every builder working with LLMs needs to internalize. Sometimes the most useful model is the one that hasn't been made useful yet.
