This blog post discusses the limitations of Large Language Models (LLMs), such as the knowledge cutoff and the occurrence of hallucinations. To overcome these limitations, two approaches are explored: fine-tuning and retrieval-augmented generation. Fine-tuning involves the supervised training phase, where question-answer pairs are used to optimize the performance of the LLM. However, fine-tuning does not fully solve the knowledge cutoff issue or eliminate hallucinations. The retrieval-augmented approach uses external information to supplement the LLM’s internal knowledge, resulting in advantages such as source-citing, minimal hallucinations, and ease of updating information. However, it relies on an intelligent search tool and access to the user’s knowledge base. The blog post concludes by highlighting the ongoing development of the NaLLM project and inviting readers to explore the project’s GitHub repository.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Limitations of LLMs | From reliance on internal knowledge to external information retrieval | Improved access to up-to-date and validated information | Need for accurate and reliable information |
| Knowledge cutoff problem | From limited knowledge to updated and expanded knowledge | LLMs with updated and expanded knowledge | Need for current and relevant information |
| Inaccurate information generation | From inaccurate information to more accurate results | Improved verification and fact-checking of LLM-generated answers | Need for reliable and trustworthy information |
| Fine-tuning LLMs | From general LLM performance to customized and optimized LLMs | Fine-tuned LLMs for specific tasks and updated knowledge | Optimization and customization of LLM performance |
| Retrieval-augmented generation | From internal knowledge reliance to external information retrieval | Improved access to relevant and up-to-date information | Enhanced information retrieval and personalization |