OpenAI’s chief scientist has thoughts on how to limit AI development

What happened
Jakub Pachocki, the chief scientist at OpenAI, published a blog post proposing a new way to set limits on the pace of AI development. He suggested using 'perplexity,' a metric that measures how accurately AI models predict text, as a standard for this purpose. The proposal comes after recent industry discussions about the need for safety standards and limits on rapid AI progress. Pachocki argues that perplexity could serve as a common benchmark for AI companies to agree on development thresholds.
Key details
- Perplexity measures how well an AI model predicts the next token in a sequence; lower values mean better predictions.
- OpenAI's GPT-4 model achieved a perplexity of about 0.91.
- Recent models have reached perplexity losses below 0.8, which Pachocki links to new capabilities in cybersecurity and mathematics.
- Industry leaders, including those at OpenAI and Anthropic, have called for agreed safety standards and limits on AI progress.
- Pachocki's proposal would have companies pause development past a certain perplexity level until monitoring and understanding of AI behavior improves.
Why it matters
AI developers and companies may face new constraints on how quickly they can advance their models if perplexity becomes an industry standard. This could affect the timeline for releasing more advanced AI systems, as organizations would need to meet agreed safety and monitoring milestones before surpassing certain performance thresholds. The approach aims to address concerns about potential risks from unchecked AI progress by providing a concrete, shared measurement for development speed.
- Perplexity and Intelligence· 2 sources
