This article explores the use of a large language model (LLM) as a knowledge graph store and the process of fine-tuning the LLM with a knowledge graph. The author demonstrates how fine-tuning the LLM with RDF knowledge graphs can enable users to ask complex questions in their language. The article discusses the limitations of prompting and the need to fine-tune the LLM on the knowledge graph to achieve better results. It also highlights the challenges of overfitting and the cost of gathering and preparing training data. The author concludes by suggesting future areas of research, including optimizing knowledge graph serialization and exploring different tokenization methods for IRIs.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Large Language Model as Knowledge Graph Store | LLM being used as a knowledge graph store | LLM can efficiently answer complex questions using knowledge graphs | Desire for more efficient and accurate knowledge retrieval |
| Fine-Tuning LLM with Knowledge Graph | Fine-tuning LLM using a knowledge graph | LLM can be fine-tuned with specific domain knowledge | Need for LLMs to have domain-specific expertise |
| Limitations of Prompting | Limitations in the scale of serialized knowledge graphs used as prompts | Increase in the scale of serialized knowledge graphs that can be included as prompts | Improvement in the scalability of prompt-based approaches |
| Convincing LLM with More Epochs | Overfitting of LLM during fine-tuning | Increased adaptation of LLM towards fine-tuning data | Desire for LLM to treat prompt:completion statements as near-certainties |
| Knowledge Graph Context for Fine-Tuned LLM | Adding serialized knowledge graph context to fine-tuned LLM | LLM can provide more accurate answers to questions related to the knowledge graph | Improved integration of LLM and domain-specific knowledge |
| Path-Search Queries with Fine-Tuned LLM | Difficulty in answering path-search queries using fine-tuned LLM | Exploration of alternative models (e.g., instruction-trained LLMs) for path-search queries | Need for LLMs that excel in path-search queries |
| Optimizing Knowledge Graph Serialization | Exploring alternative methods of serializing knowledge graphs for fine-tuning LLM | More efficient serialization methods for knowledge graphs | Improvement in the efficiency of fine-tuning process |
| Tokenization of IRIs in Knowledge Graphs | Challenges in tokenizing unique identifiers (IRIs) in RDF graphs | Improved tokenization methods for IRIs in LLMs | Enhanced handling of unique identifiers in LLMs |
| Knowledge Graphs as Precursors for LLM Fine-Tuning | Knowledge graphs as a source for training data in LLM fine-tuning | Increased use of knowledge graphs for fine-tuning LLMs | Growing recognition of the value of knowledge graphs in training LLMs |
| Path Query Chain-of-Thought Prompting with Fine-Tuned LLM | Exploration of path query Chain-of-Thought prompting with fine-tuned LLM | Improved ability of LLM to provide detailed answers to path-search queries | Advancement in the integration of Chain-of-Thought prompting with fine-tuned LLMs |