A Language Model Is Not A Behavioral Model

A study tested ChatGPT, Claude, and Gemini on predicting customer preferences in 11 live advertising experiments. ChatGPT matched 4 winners, Claude 3, Gemini 1; all failed in 6 cases, often choosing the same incorrect option. This raises questions about trusting language models to predict real customer behavior based on plausible narratives alone.