LLM & Text Generation3 min reading time

Everyone is building LLM routers, we deprecated ours

Hacker News
Read the full article
Manifest launched an LLM router in March to reduce costs by routing requests to models based on complexity but deprecated it after mixed results and challenges in accurately assessing task complexity from prompts alone. They found caching more effective for cost reduction and emphasized the importance of engineers choosing models deliberately.

More in LLM & Text Generation

Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster

MarkTechPost

Moonshot releases Kimi K3.1 Pro and says it cut its hacking skills

Covered by 4 sources