Can LLMs Engineer Their Own Agent Harness? ByteDance Seed’s HarnessDev Says Only 34 of 64 Changes Generalize

Researchers from ByteDance Seed and partner institutions introduced HarnessDev, a framework where large language models create and evolve their own agent harnesses—code that manages model execution and interaction. Testing six LLMs, they found that only about half of the modifications generalize well across unseen tasks, with Opus 4.8 achieving the highest creation-phase performance but still below human-engineered baselines. The study highlights challenges in automating agent harness design and