Presentation: Serving LLMs at Scale: The Hidden KV Cache Advantage

Khawaja Shams, co-founder of startup Momento and former NASA engineer, discusses the challenges and opportunities in serving large language models (LLMs) at scale, emphasizing the critical role of distributed systems in improving GPU utilization for AI inference stacks.