DeepSeek open-sources DeepInfer, the inference framework behind its production services

Chinese AI startup DeepSeek has open-sourced DeepInfer, its inference framework powering production services, supporting the DeepSeek-V4.1-Flash model and optimized for Nvidia SM100 GPUs. The framework includes speculative decoding techniques to improve throughput and reduce computation costs, serving 155 trillion tokens daily at $38 per GPU-hour off-peak.