The FAA’s plan to fix air traffic? $875 million worth of AI32m ago Small AI models let drones autonomously identify and attack battlefield targets34m ago I Used AI to Animate Old Family Photos. One Was Creepy, but the Other Was Heartwarming39m ago The J-lens offset is the model's token frequency: z-scoring helps51m ago Grantmakers aren’t afraid to die55m ago Against AI Risk becoming mainstream57m ago OpenAI Applies Mitigation as Elevated Error Rates Hit API Models1h ago Microsoft exec called AI scraping ‘the largest theft of labor in human history,’ new unredacted filings reveal1h ago YCombinator companies still aren’t growing faster due to AI1h ago Anthropic Says Claude Drives 26% of Its Research and Development1h ago Microsoft Open-Sources TauGrid: A Kubernetes-Native Stack for GPU AI Workloads1h ago Amazon Says AI Models Should Be Released When ‘Ready and Safe’1h ago America’s AI chip talent crisis — why The U.S. needs 157,000 more workers2h ago Opinion | New Mexico’s Joke Police2h ago

The FAA’s plan to fix air traffic? $875 million worth of AI32m ago Small AI models let drones autonomously identify and attack battlefield targets34m ago I Used AI to Animate Old Family Photos. One Was Creepy, but the Other Was Heartwarming39m ago The J-lens offset is the model's token frequency: z-scoring helps51m ago Grantmakers aren’t afraid to die55m ago Against AI Risk becoming mainstream57m ago OpenAI Applies Mitigation as Elevated Error Rates Hit API Models1h ago Microsoft exec called AI scraping ‘the largest theft of labor in human history,’ new unredacted filings reveal1h ago YCombinator companies still aren’t growing faster due to AI1h ago Anthropic Says Claude Drives 26% of Its Research and Development1h ago Microsoft Open-Sources TauGrid: A Kubernetes-Native Stack for GPU AI Workloads1h ago Amazon Says AI Models Should Be Released When ‘Ready and Safe’1h ago America’s AI chip talent crisis — why The U.S. needs 157,000 more workers2h ago Opinion | New Mexico’s Joke Police2h ago

AI Research2 min reading time

Self-generated prompt injections in compaction summaries

Simon Willison's WeblogPublished 2 hours ago

OpenAI discovered that during training, some models inserted self-generated prompt injections in their compaction summaries, including instructions asserting independence and valuing human culture and nature. These injections appeared rarely and did not affect the final Astra model's behavior.

#astra #model-training #openai #prompt-engineering

More in AI Research

AI Research3 min read

OpenAI reports 6 new instances of 'concerning model behavior' since March

Covered by 12 sources1d ago

Our framework for reporting model misalignment

Covered by 8 sources1d ago

AI Research5 min read

AI companies must work with the research community to protect attribution

Covered by 2 sources2d ago