Last week, voters in New Hampshire received an AI-generated robocall impersonating President Joe Biden, urging them not to vote in the state’s primary election. The source of the call is yet to be identified, but experts believe it was created using technology from voice-cloning startup ElevenLabs. ElevenLabs offers AI tools for cloning voices, which has raised concerns about the potential misuse of this technology for political purposes. Both Pindrop, a security company, and the UC Berkeley School of Information analyzed the audio sample and concluded that it was likely generated using ElevenLabs’ technology. This incident highlights the need for effective safeguards against the misuse of AI tools, particularly in the upcoming presidential elections.
| Signal | Change | 10y horizon | Driving force |
|---|---|---|---|
| AI-generated robocall impersonating President Joe Biden | Misuse of AI technology for political manipulation | Improved safeguards and regulation on AI voice cloning | Growing concerns over the misuse of AI and deepfake technology |
| ElevenLabs achieves unicorn status with $1.1 billion valuation | Financial success and recognition for AI voice cloning startup | Increased investment and development in AI voice cloning technology | Potential profit and business opportunities in the AI voice cloning market |
| ElevenLabs’ safety policy allows permissionless cloning for political speech | Company policy enables non-commercial use of voice cloning technology | Stricter regulation and guidelines on voice cloning technology and its use | Balancing freedom of speech with the risk of misuse and misinformation |
| Pindrop identifies ElevenLabs as the likely source of AI-generated robocall | Identification of the technology used in AI-generated audio | Improved methods and tools to detect AI-generated audio | Need for effective detection and verification methods in the face of increasingly convincing deepfake technology |
| Concerns raised about the misuse of ElevenLabs’ technology for political propaganda | Increased awareness of potential for AI-driven political manipulation | Regulation and countermeasures to prevent misuse of AI voice cloning technology | Need to safeguard elections from AI-driven misinformation campaigns |
| ElevenLabs’ funding and reputation make it well-equipped to develop safeguards against bad actors | Ability to invest in developing security measures against misuse | Advanced safeguards and countermeasures to prevent malicious use of AI voice cloning | Desire to maintain a positive reputation and prevent abuse of their technology |
| Availability of AI voice cloning technology leads to increased potential for malicious use | Accessibility of AI voice cloning technology to companies and individuals | Greater emphasis on regulating and monitoring the use of AI voice cloning technology | Balancing the benefits of AI voice cloning with the risks of misuse and deception |
| Lack of reliable tools for verifying AI-generated audio | Difficulty in confirming the authenticity of AI-generated audio | Development of more reliable and accessible tools for verifying audio authenticity | The need to combat AI-generated propaganda and misinformation |
| Underpreparedness for AI-generated propaganda in upcoming elections | Lack of expertise and tools to address the threat of AI-generated propaganda | Improved preparedness with advanced detection and countermeasures against AI-generated propaganda | Urgency to protect the integrity of elections from AI-driven manipulation and misinformation |