AI Latency vs. Throughput: TTFT, Tokens per Second, and Other Metrics Explained

AI latency measures the time taken for a single request or token, while throughput measures the volume of work completed per time unit. These metrics depend on the entire system, including hardware and interfaces, not just the AI model itself. Understanding their precise definitions helps avoid misleading claims based on superficial performance numbers.