Mistral AI is continuing its mission to deliver the best open models to the developer community. They have released Mixtral 8x7B, which is a high-quality sparse mixture of experts model with open weights. Mixtral outperforms Llama 2 70B on most benchmarks with faster inference speed. It is a strong open-weight model with a permissive license and has better cost/performance trade-offs than other models like GPT3.5. Mixtral can handle a context of 32k tokens and supports multiple languages. Additionally, Mistral has released an instructed version of Mixtral that achieves a high score on MT-Bench. They have also made efforts to enable the community to deploy Mixtral with an open-source stack.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Mistral AI team releases Mixtral 8x7B | Advancement in AI models | More powerful and efficient AI models | Pushing the frontier of open models |
| Mixtral is a sparse mixture-of-experts | Technological innovation | Increased efficiency and cost-effectiveness | Controlling cost and latency |
| Mixtral outperforms Llama 2 70B | Improvement in model performance | Adoption of Mixtral as the preferred model | Improving the performance of AI models |
| Mixtral can handle multiple languages | Enhanced language capabilities | Improved language processing | Meeting the demand for multilingual AI models |
| Mixtral 8x7B Instruct is released | Optimization for instruction following | Improved performance in instruction following | Enhancing the capabilities of AI models |
| Open-source deployment of Mixtral | Openness and accessibility | Community can freely use Mixtral | Fostering collaboration and community engagement |