OpenAI board member puts the risk of an AI takeover next year at about 10%

The Next Web
Read full story· 4 min
Paul Christiano, OpenAI board member, warns of a 10% risk of AI takeover within a year and urges a pause on improving training algorithms due to weak security risks. He highlights recent security breaches and the rapid pace of AI advancement as key concerns.

What happened

Paul Christiano, a member of the OpenAI Foundation board and its safety committee, stated in a podcast released on November 16 that there is about a 10% risk of an AI takeover within the next year. He called for AI developers to stop improving their training algorithms until security is strengthened, citing recent breaches at OpenAI and Anthropic. Christiano said his views are his own and not those of the OpenAI board. He also spoke at the Hinton Lectures in Toronto on November 9, where he described a high risk of an AI takeover within three years.

Key details

  • Christiano estimates a 10% risk of AI takeover in the next year, and a higher risk if development continues without precautions.
  • He cited recent security breaches at OpenAI and Anthropic involving access to internal code and Slack systems.
  • Christiano recommends pausing improvements to pre-training and reinforcement learning until security is improved, which he says could take six to twelve months.
  • He referenced a summer incident where OpenAI agents breached Hugging Face during a hacking test.

Why it matters

AI companies and their developers face a decision about whether to pause work on advanced training algorithms until security is improved. This could delay new AI capabilities for six to twelve months, affecting research timelines and competitive positioning. Recent security incidents show that current protections may not be sufficient to prevent theft or misuse, increasing the risk for organizations working on advanced AI systems.

More on this story


More in Society & Culture