The leaked document from Google highlights the diminishing competitive advantage of both Google and OpenAI in the race to build powerful language models. While Google’s models still have a slight edge in terms of quality, the open source community is rapidly closing the gap with faster, more customizable, and more capable models. The document emphasizes the tremendous innovation happening in the open source community, where even ordinary people can contribute ideas and improvements. One key technique mentioned is LoRA, which allows for efficient fine-tuning of models and stacking improvements on top of each other. The document concludes that keeping technological advantages secret is becoming increasingly difficult, and that OpenAI is making similar mistakes in their approach to open source. The paper suggests that collaboration and learning from each other may be the way forward in this rapidly evolving landscape.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Leaked Google document: “We Have No Moat, And Neither Does OpenAI” | Open-source models surpass proprietary models | Open-source models dominate in speed, customization, privacy, and capability | Innovation and collaboration in the open-source community |
| Rapid innovation in open-source LLMs | Open-source LLMs outpace proprietary models | Open-source LLMs continue to improve and surpass proprietary models | Accessibility and affordability of training and experimentation |
| LoRA technique enables cheap and stackable fine-tuning | Cheap and efficient model updates | Models can be easily fine-tuned and kept up to date without high costs | Ability to iterate faster on small models |
| Difficulty in maintaining competitive advantage | Challenges in protecting technology secrets | Open collaboration dilutes the value of proprietary research | Wider research community collaboration |
| OpenAI’s ability to maintain an edge is in question | OpenAI’s stance on open source is a disadvantage | Open-source alternatives may surpass OpenAI without a change in strategy | Need to adapt to open-source innovation |