METR Finds AI Time Horizons Doubling as Research Judgment Lags

Unite.AI
Read full story· 6 min
METR's latest evaluation shows that public frontier AI models' 50% time horizons—human-estimated task durations for 50% success—are doubling roughly every 110 days, with faster gains on verifiable tasks than on open-ended research. The update includes models up to GPT-6 Astra and reveals challenges in human calibration and weaker judgment on open-ended tasks.

What happened

On November 20, 2026, METR released an evaluation of public frontier AI models, measuring how quickly these models reach a 50% success rate on various tasks. The report found that the time horizons for reaching this threshold are doubling about every 110 days, based on combined TH1.1 and TH2 test suites. The evaluation covered models up to Opus 5.5 and GPT-6 Astra, with a focus on tasks relevant to automating AI research and development. METR also noted that while models are improving rapidly on tasks with clear, automatic scoring, their performance is weaker on open-ended research and judgment tasks.

Key details

  • The TH2 suite includes 36 tasks in 12 families, with 8 verifiable and 4 open-ended categories.
  • On verifiable tasks, the estimated doubling time for model improvement is 1.65 months (95% CI: 0.79–3.99 months).
  • Current public models' time horizons on verifiable tasks reach up to 27 hours.
  • Human baselines were directly used for only 3 of 34 new tasks; the rest relied on researcher estimates.
  • Specialists evaluating tasks were paid $200–$350 per hour and could use Sonnet 5 for implementation.

Why it matters

AI developers and researchers using public frontier models now see much faster progress on tasks that can be automatically verified, with the time needed for models to achieve a 50% success rate halving every few months. However, for open-ended research and judgment tasks, these models still lag behind, which limits their usefulness for automating more complex or creative aspects of AI R&D. Organizations relying on these models face higher evaluation costs and must account for weaker performance on tasks that require nuanced judgment or original research design.


More in AI Research