The Open Language Model (OLMo) is an AI2 LLM framework designed to provide open access to data, training code, models, and evaluation code. It includes full pretraining data, training code, model weights, and evaluation tools. The OLMo framework aims to advance AI research by enabling collective study of language models. Researchers and developers benefit from the increased precision, reduced developmental redundancies, and lasting results provided by OLMo. The OLMo dataset, Dolma, is a diverse mix of 3 trillion tokens from web content, academic publications, code, books, and encyclopedic materials. The evaluation suite, Paloma, offers a benchmark for assessing performance in different domains. The collaborative effort from various partners has made OLMo possible.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Open Language Model (OLMo) | Shift towards open AI research | Increased collaboration and transparency | Advancement and innovation in AI research |
| Full access to training data | Increased understanding and speed | More precise and efficient research | Eliminating assumptions in model performance |
| Open training code and model weights | Enhanced reproducibility of results | Accelerated development and progress | Facilitating collaboration and learning |
| Evaluation suite provided | Standardized evaluation process | Improved benchmarking and comparison | Promoting transparency and accountability |
| Dolma dataset | Largest open dataset for LLM training | Improved model training and performance | Access to diverse and comprehensive data |
| Paloma benchmark | Evaluation across diverse domains | Enhanced understanding of model performance | Holistic assessment of language model quality |
| Collaboration with partners and community | Synergistic progress in AI research | Advancements through collective effort | Leveraging diverse expertise and resources |