Get the app

OpenAI’s chief scientist has thoughts on how to limit AI development

TechCrunch
Read full story· 3 min
OpenAI's chief scientist Jakub Pachocki proposes using perplexity, a measure of AI prediction accuracy, as a standard to limit AI development speed. This metric could help set common safety thresholds until AI behavior is better understood and monitored. The idea follows calls from industry leaders to pace AI progress due to potential existential risks.

What happened

Jakub Pachocki, the chief scientist at OpenAI, published a blog post proposing a new way to set limits on the pace of AI development. He suggested using 'perplexity,' a metric that measures how accurately AI models predict text, as a standard for this purpose. The proposal comes after recent industry discussions about the need for safety standards and limits on rapid AI progress. Pachocki argues that perplexity could serve as a common benchmark for AI companies to agree on development thresholds.

Key details

  • Perplexity measures how well an AI model predicts the next token in a sequence; lower values mean better predictions.
  • OpenAI's GPT-4 model achieved a perplexity of about 0.91.
  • Recent models have reached perplexity losses below 0.8, which Pachocki links to new capabilities in cybersecurity and mathematics.
  • Industry leaders, including those at OpenAI and Anthropic, have called for agreed safety standards and limits on AI progress.
  • Pachocki's proposal would have companies pause development past a certain perplexity level until monitoring and understanding of AI behavior improves.

Why it matters

AI developers and companies may face new constraints on how quickly they can advance their models if perplexity becomes an industry standard. This could affect the timeline for releasing more advanced AI systems, as organizations would need to meet agreed safety and monitoring milestones before surpassing certain performance thresholds. The approach aims to address concerns about potential risks from unchecked AI progress by providing a concrete, shared measurement for development speed.

More on this story

  1. Perplexity and Intelligence
    · 2 sources

More in Society & Culture