Arbitrage: Efficient Reasoning via Advantage-Aware Speculation

Researchers from UC Berkeley and affiliated institutions developed ARBITRAGE, a step-level speculative decoding method that dynamically routes generation between draft and target models to improve reasoning efficiency in large language models. ARBITRAGE reduces inference latency by up to 2x while maintaining accuracy on mathematical reasoning benchmarks, outperforming previous methods.