Olmo-core 3

Researchers can use it to train models and study the decisions involved in training large mixture-of-experts systems.

Availability

Released ahead of COLM

Pricing model

Not established

Deployment

Researchers and developers can adapt it to different hardware

Privacy

Not established

Licence

Not established

• Olmo-core 3 is designed to scale MoE training into the trillion-parameter range. ^{[1] [3]}

  Olmo-core 3 is designed to scale MoE training into the trillion-parameter range while preserving computational efficiency.

• A benchmark increased the expert pool from 8 to 128 while training throughput fell by less than 5%. ^{[1] [3]}

  In one benchmark, we increased the expert pool from 8 to 128 while still selecting only four experts per token – the small units of text a language model processes – keeping the number of active parameters per token roughly fixed at about 3.2B.

• A preliminary test on eight NVIDIA B300 GPUs reported 52,000 tokens per second per GPU for a 47-billion-parameter MoE. ^{[1] [3]}

  In a preliminary test on eight NVIDIA B300 GPUs, a 47-billion-parameter MoE processed 52,000 tokens per second per GPU with the new stack, compared with 19,400 using our earlier implementation—about 2.7× the throughput.

• A benchmark tested a 1.2-trillion-parameter model across 512 NVIDIA B300 GPUs. ^{[1] [3]}

  We’ve benchmarked Olmo-core 3 across a range of configurations on NVIDIA B300 GPUs, including a 1.2-trillion-parameter model with 58.36 billion parameters active per token across 512 GPUs.

• Ai2 released Olmo-core 3 and linked to its code on GitHub. ^[1]
• Researchers and developers can use the open stack to train their own MoEs and adapt it to different hardware. ^{[1] [3]}

  And it’s fully open—researchers and developers can use Olmo-core 3 to train their own MoEs, adapt it to different hardware, and experiment with routing, parallelism, and other parts of the system.

• Olmo-core 3 was released ahead of COLM. ^[2]

  Released ahead of COLM, Olmo-core 3 adds a redesigned system for training large mixture-of-experts models.

• Olmo-core 3 adds a redesigned system for training large mixture-of-experts models. ^{[1] [2] [3]}

  Released ahead of COLM, Olmo-core 3 adds a redesigned system for training large mixture-of-experts models.

• Researchers can use the code to develop their own models. ^{[1] [2] [3]}

  Researchers can use the code to develop their own models and investigate training decisions, alongside the checkpoints and technical reports available through the broader Olmo ecosystem.

• Backburner leads 25 AI developments in week 40, 2026 Weekly brief 2026-W40
• Perplexity introduces a decision model fine-tuned from Qwen3.8-27B Daily brief 2026-10-01