This text discusses the risks associated with backdooring machine-learning models, particularly summarizer bots, and how they can be used to shape opinions. The author highlights the danger of relying on these models, as they can be subtly manipulated to produce biased summaries. The concept of a “meta-backdoor” is introduced, where trigger words are hidden in the input text to invoke a spin function in the summarizer. The authors propose defense mechanisms, such as comparing the output from multiple models, to detect and mitigate these attacks. The text also touches on other areas where machine learning can be exploited, such as translation models and automated decision-making systems.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| Backdooring a summarizerbot | Manipulating AI models to change perspectives | Increased sophistication of hidden biases in AI models | Desire for automation without human intervention |
| Machine-learning system attacks | Exploiting flaws in ML systems | Increased awareness and development of countermeasures | Desire for automation and efficiency |
| Meta-backdoors and propaganda-as-a-service | Introducing hidden biases into summarizer models | Increased difficulty in detecting biased information | Desire to manipulate information for personal gain |
| Signal’s commitment to encryption | Maintaining strong encryption in messaging platforms | Continued use and development of encrypted communication | Protection of privacy and security |
| Historical events from 2007 to 2021 | Historical milestones and events | Evolving political, social, and technological landscape | Impact of significant events on society |
| Recent publications and upcoming events | Cory Doctorow’s current projects and appearances | Growth and diversification of Doctorow’s work | Author’s professional development and engagement with audiences |