Tsinghua AI Open-Sources AKV, a Cache Engineering Server for LLM Agents on SGLang

Tsinghua University and CUHK researchers have open-sourced AKV, a KV cache serving system for large language model agents that reduces computation cost by 27.7% and improves task scores by 1.7 points across multiple models and benchmarks.