Ai2 releases Olmo-core 3 to make developing large mixture-of-experts LLMs more efficient

The Allen Institute for AI released Olmo-core 3, a framework that improves training efficiency for large mixture-of-experts language models, enabling scaling to over one trillion parameters with lower costs. Olmo-core 3 achieves 2.7 times higher throughput than Nvidia's Megatron-core by using expert parallelism and distributed optimization across GPUs.