OLMo-Mix-1124, 3.9T tokens.
Llama 3, ~15T tokens.
Soldaini et al., “Dolma” ACL 2024 · OLMo Team, “2 OLMo 2 Furious” arXiv:2501.00656 · Grattafiori et al., “The Llama 3 Herd of Models” (2024)