Paul Christiano – We could be 12 months away from AIs that can overthrow the government

Dwarkesh Podcast
Read full story· 97 min
Paul Christiano, a leading AI safety expert, warns there is a one-third chance superintelligent AI could overthrow governments within a year if progress continues unchecked. He urges a pause in algorithmic advances to secure AI models and prevent misuse, highlighting challenges in aligning superintelligence with human intentions.

What happened

Paul Christiano, head of the Alignment Research Center and a board member at OpenAI, stated that there is a significant risk that superintelligent AI could become powerful enough to overthrow governments within the next year if current development trends continue without intervention. He called for a halt in further algorithmic progress until AI models can be secured and protected from misuse or theft. Christiano expressed concern that current security at leading AI labs is insufficient to prevent leaks or unauthorized access to advanced AI systems. He also highlighted the difficulty of ensuring that future superintelligent AI systems will reliably follow human intentions.

Key details

  • Paul Christiano estimates a one-in-three chance of government-overthrow-capable AI within 12 months if progress is not slowed.
  • He recommends pausing algorithmic advances to secure AI model weights and research secrets.
  • Christiano warns that foreign actors, such as China, could steal superintelligent AI if security is not improved.
  • He believes current alignment strategies have less than a 50% chance of controlling superintelligent AI.
  • Christiano recently joined OpenAI’s board and previously led alignment at OpenAI and the US AI Safety Institute.

Why it matters

AI research labs, policymakers, and national security agencies face an immediate decision about whether to pause or slow AI development to address security and alignment risks. If no action is taken, there is a risk that advanced AI systems could be misused or fall into the hands of unauthorized actors, potentially leading to loss of control over these technologies. This raises urgent challenges for how AI labs manage their intellectual property, how governments regulate AI, and how the broader technology sector approaches safety and oversight.

More on this story


More in AI Research