A fundamental flaw leaves LLMs strikingly vulnerable to attack

Researchers presented a paper at ICML revealing a fundamental flaw in large language models (LLMs) that makes them inherently vulnerable to manipulation. This flaw allows attackers to trick models into revealing sensitive or dangerous information despite existing safeguards. Attempts to secure LLMs by listing prohibited behaviors are insufficient, as attackers can mimic internal reasoning steps to bypass restrictions.