Why Large Language Models Fail at Tabular Prediction

Researchers analyzed why large language models (LLMs) underperform on tabular data prediction tasks, finding that increasing input dimensionality significantly degrades LLM accuracy unlike classical models. They ruled out other factors like data noise and tokenization, highlighting a unique limitation of LLMs in handling high-dimensional tabular data without fine-tuning or additional tools.