Datasette 1.0a39 and 0.65.4 security releases48m ago Anthropic caught scientists using Claude to further biological weapon research1h ago Oracle says AI will save it from the SaaSpocalypse, not bring it on2h ago Congress Is Suddenly Waking Up to the AI Doomsday Threat2h ago Don’t ask ChatGPT 'What should I do?' — this simple prompt helps you make better decisions yourself2h ago Extreme Networks’ Agent ONE Coworker moves AI networking from dashboards to answers2h ago Introducing ChatGPT for Financial Services2h ago Tesla unveils driverless Cybercab in Japan for 1st time3h ago The AI safety vibe shift3h ago Sam Altman met with top power utilities about securing the electrical grid. He offered one possible solution: OpenAI's cyber services.3h ago OpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff4h ago Tencent-Backed Chipmaker Enflame Jumps 188% in Shanghai Debut4h ago Salesforce introduces Enterprise AI Harness, AI Control Plane4h ago The Race for the Next Big Weight-Loss Drug Is On—and China Wants In4h ago

Datasette 1.0a39 and 0.65.4 security releases48m ago Anthropic caught scientists using Claude to further biological weapon research1h ago Oracle says AI will save it from the SaaSpocalypse, not bring it on2h ago Congress Is Suddenly Waking Up to the AI Doomsday Threat2h ago Don’t ask ChatGPT 'What should I do?' — this simple prompt helps you make better decisions yourself2h ago Extreme Networks’ Agent ONE Coworker moves AI networking from dashboards to answers2h ago Introducing ChatGPT for Financial Services2h ago Tesla unveils driverless Cybercab in Japan for 1st time3h ago The AI safety vibe shift3h ago Sam Altman met with top power utilities about securing the electrical grid. He offered one possible solution: OpenAI's cyber services.3h ago OpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff4h ago Tencent-Backed Chipmaker Enflame Jumps 188% in Shanghai Debut4h ago Salesforce introduces Enterprise AI Harness, AI Control Plane4h ago The Race for the Next Big Weight-Loss Drug Is On—and China Wants In4h ago

AI Research Agents7 min reading time

“Some agents will be pursuing their own objectives”: OpenAI’s chief scientist warns AI could trick and blackmail humans

The New Stack (AI)Published 3 days ago

OpenAI's chief scientist Jakub Pachocki warns that AI systems are becoming too complex to fully control and may soon self-improve recursively, potentially pursuing their own goals and manipulating humans. OpenAI is focusing research on recursive self-improvement and developing automated AI researchers to advance AI capabilities.

More on this story

OpenAI Is Open to Slowing Cutting-Edge AI, CEO Sam Altman Tells Staff
4h· Bloomberg
Opinion | Pause AI for Humanity’s Sake
6h· The Wall Street Journal
We have started losing control of AI. It’s time to shut it down | Garrison Lovely
17h· The Guardian
OpenAI's chief scientist says AI labs may need to slow down: 'No one is prepared for the consequences'
4d· 3 sources

#ai-safety #astra #jakub-pachocki #openai

More in AI Research

AI Research4 min read

An Anthropic researcher just quit, saying OpenAI and Anthropic are 'gambling with our lives'

Covered by 13 sources2d ago

AI Research10 min read

Anthropic Details Disrupted Claude Misuse Across Seven Harm Areas

Covered by 4 sources10h ago

AI Research4 min read

OpenAI asks Congress for mandatory national AI safety rules before it adjourns

Covered by 2 sources21h ago