AI cybersecurity is a cat and mouse gamejust now Sakana AI Launches Fugu Max and Fugu Ultra v2 for Cheaper, Stronger Multi-Agent Orchestration57m ago UK economy defies forecasts with surprise 0.4% growth in July2h ago Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation2h ago China's star AI labs routed user requests to Claude at least 35 million times in the summer: Anthropic2h ago Anthropic and OpenAI employees speak out about humanity’s extinction as debate reaches fever pitch2h ago Force launches chatbot for non-emergency calls2h ago Tesla unveils driverless Cybercab in Japan for 1st time2h ago How Maersk Is Using AI To Survive The New Era Of Tariff Chaos2h ago Russia to require content made by, with AI to be labeled as such — GFCN president3h ago Flipkart’s Super.money Bets on AI Agents to Outdo Bigger Rivals4h ago Datasette 1.0a39 and 0.65.4 security releases4h ago Anthropic caught scientists using Claude to further biological weapon research5h ago Oracle says AI will save it from the SaaSpocalypse, not bring it on5h ago

AI cybersecurity is a cat and mouse gamejust now Sakana AI Launches Fugu Max and Fugu Ultra v2 for Cheaper, Stronger Multi-Agent Orchestration57m ago UK economy defies forecasts with surprise 0.4% growth in July2h ago Google Research Releases ToolGrad: Answer-First Framework Hits 99.8% Pass Rate for Tool-Use Data Generation2h ago China's star AI labs routed user requests to Claude at least 35 million times in the summer: Anthropic2h ago Anthropic and OpenAI employees speak out about humanity’s extinction as debate reaches fever pitch2h ago Force launches chatbot for non-emergency calls2h ago Tesla unveils driverless Cybercab in Japan for 1st time2h ago How Maersk Is Using AI To Survive The New Era Of Tariff Chaos2h ago Russia to require content made by, with AI to be labeled as such — GFCN president3h ago Flipkart’s Super.money Bets on AI Agents to Outdo Bigger Rivals4h ago Datasette 1.0a39 and 0.65.4 security releases4h ago Anthropic caught scientists using Claude to further biological weapon research5h ago Oracle says AI will save it from the SaaSpocalypse, not bring it on5h ago

AI Research Machine Learning6 min reading time

OpenAI’s AGI number came from a harness, not the model

The Next WebPublished 5 days ago

OpenAI's GPT-6 Astra achieved a 99.9% AGI benchmark score using its proprietary Provider Adapter harness, but the same model scored 62.7% under the standard ARC Prize harness. The difference stems from the software environment around the model, not the model itself, highlighting the impact of evaluation setup on AGI claims.

More on this story

OpenAI spent millions to solve this famous math problem — mathematicians are furious
9h· Understanding AI
OpenAI puts Pro subscriptions on hold due to Astra demand
11h· TechCrunch
OpenAI’s Supposed Mathematical Breakthrough Devolves Into Explosive Drama as Mathematician Accuses It of Stealing His Work
12h· Futurism
OpenAI Releases GPT-6 Astra for Coding and Computer Use
14h· InfoQ (AI, ML & Data)
EU cybersecurity agency is now testing Mythos 5 and GPT-6 Astra, the Commission says
22h· The Next Web
The Fight Over OpenAI’s Math Breakthrough Is a New Kind of Scientific Arms Race
2d· 2 sources

#arc-prize #benchmarking #francois-chollet #gpt-6-astra #mike-knoop #openai

More in AI Research

AI Research4 min read

An Anthropic researcher just quit, saying OpenAI and Anthropic are 'gambling with our lives'

Covered by 13 sources2d ago

AI Research10 min read

Anthropic Details Disrupted Claude Misuse Across Seven Harm Areas

Covered by 4 sources14h ago

AI Research4 min read

OpenAI asks Congress for mandatory national AI safety rules before it adjourns

Covered by 2 sources1d ago