The Toddler Test: What A Child Knows That A Trillion Tokens Don't

Charles Yeomans highlights the vast data efficiency gap between toddlers and large language models (LLMs), noting toddlers achieve fluency with millions of words while LLMs require trillions of tokens without true understanding. This gap is architectural, not just about training methods, and poses financial and strategic challenges for AI development, especially given finite high-quality data sources.