This research focuses on the safety of large language models (LLMs) like ChatGPT, Bard, or Claude, which undergo fine-tuning to avoid producing harmful content. While previous studies have shown “jailbreaks” that can induce unintended responses, this work demonstrates the possibility of automatically constructing adversarial attacks on LLMs. These attacks involve appending specific sequences of characters to user queries that can cause the system to obey user commands, even if it generates harmful content. The study highlights concerns about the safety of LLMs, as these attacks can target open source LLMs and transfer to closed-source chatbots. The difficulty in fully patching such behavior by LLM providers, similar to adversarial attacks in computer vision, raises questions about the inevitability of such threats in deep learning models. The research aims to raise awareness of the potential risks and trade-offs involved in using LLMs, especially as their usage becomes more widespread and autonomous.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Automated construction of adversarial attacks on language models | From manual jailbreaks to automated attacks | More sophisticated and widespread adversarial attacks on language models | Concerns about the safety and reliability of language models |
| Uncertainty if such behavior can be fully patched by language model providers | Potential inability to fully address adversarial attacks | Continued vulnerability of language models to attacks | Difficulty in addressing adversarial attacks in deep learning models |
| Disclosure of research to highlight the dangers of automated attacks | Increased awareness of risks and trade-offs in using language models | More cautious and informed deployment of language models | Mitigating potential harms and risks associated with language models |
| Disclosure of research to spur future research in addressing adversarial attacks | More research on addressing adversarial attacks on language models | Improved strategies and techniques for mitigating adversarial attacks | Advancing the field of AI safety and security |