Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster

Stanford and UC Berkeley researchers introduced TokenGate, a module that enables learned token pruning in large language models to speed up long-context prefill. TokenGate achieves up to 11.3x faster time-to-first-token and reduces memory usage without accuracy loss, tested on Qwen3 models with contexts up to 128k tokens.