I spoke to seven physics experts grappling with the AI revolution

What happened
Physicist Suvrat Raju and his colleagues developed a new benchmark of unpublished theoretical physics problems to evaluate the capabilities of advanced AI models. They collected challenging, never-before-published problems from top physicists and tested leading AI systems on them. The latest model, GPT-6 Astra, solved almost 90% of the problems correctly and even identified mistakes in some human-provided solutions. The benchmark, called TheoPhysEval, was effectively mastered by Astra before its public release.
Key details
- GPT-6 Astra answered 49 out of 50 benchmark problems correctly.
- Each AI solution cost less than $3 in compute resources.
- The problems were unpublished and typically take expert physicists about a day to solve.
- Astra found subtle errors in human-written solutions during evaluation.
- The TheoPhysEval benchmark was created by Suvrat Raju and colleagues at the International Centre for Theoretical Sciences.
Why it matters
The ability of AI models like GPT-6 Astra to solve advanced, unpublished physics problems at a high level affects research physicists, academic institutions, and AI developers. Physicists now have access to tools that can rapidly perform complex calculations, potentially changing how they approach research and error-checking. This development may reduce the time and cost required for certain types of theoretical work, and raises new questions for educators and researchers about training and collaboration in the field.
