TL;DR
Chinese authorities are using AI models to enhance propaganda, censorship, and covert influence operations. This development underscores AI’s potential as a tool for authoritarian control, raising concerns about global influence and misinformation.
Chinese government and affiliated actors are increasingly using AI tools to conduct covert influence campaigns, propaganda, and censorship operations worldwide, according to recent disclosures from AI companies and security analysts. This trend highlights how AI is being exploited as a tool to bolster authoritarian control and influence, both domestically and internationally. AI tools for health can also be used for positive global efforts.
OpenAI revealed that Chinese operatives used AI models like ChatGPT to create and spread polarizing content, including comics and social media posts, aimed at influencing public opinion against AI infrastructure development in the U.S. and discrediting political figures abroad. Japan’s AI initiatives demonstrate the global interest in AI applications. In one instance, a Chinese user linked to law enforcement attempted to use ChatGPT to orchestrate a large-scale online disinformation campaign targeting Japanese Prime Minister Sanae Takaichi, involving fake accounts, false complaints, and coordinated social media activity. These activities involved hundreds of staff and thousands of fake accounts, indicating a resource-intensive covert operation.
Research by the Australian Strategic Policy Institute shows that China’s use of AI for censorship inside its borders has become faster and more cost-effective, allowing the government to manage content at large scale. For more on AI’s role in censorship, see AI OSS tools. Meanwhile, studies published in Nature suggest that Chinese state-sponsored news sources influence the training data of popular AI models, leading to responses that may reflect Chinese narratives, especially in Chinese language outputs. These models tend to produce more positive descriptions of China when queried in Chinese, which could be influenced by state-controlled information sources.
This development indicates that AI can be utilized by authoritarian regimes to influence public perception, shape narratives, and manage information dissemination. The use of AI for influence campaigns and censorship raises questions about the reliability of information ecosystems worldwide, the potential spread of disinformation, and the implications for democratic processes. As AI models become more advanced, the capacity for governments like China to influence public perception and control information flows may increase, presenting challenges for open societies.
AI censorship monitoring tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Authoritarian regimes have historically used various technologies for control and influence, including surveillance systems and propaganda broadcasts. The advent of AI introduces new capabilities for targeted and covert operations. Recent disclosures about China’s use of AI in propaganda and censorship are part of a broader pattern of digital tools being employed for political purposes. The global AI development race also influences how nations deploy these technologies for strategic objectives, often raising concerns about transparency and adherence to democratic standards.
“What AI brings to the game is, it helps plan information campaigns and it helps to execute them.”
— Kenton Thibaut, Atlantic Council
AI influence campaign detection software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Unclear Scope and Future Risks of AI-Driven Influence
While recent activities have been documented, the full extent of AI-driven influence campaigns across different countries and regimes remains uncertain. The complexity of these operations and the rapid development of AI technologies make it difficult to assess their overall scope. Future developments may involve more sophisticated manipulation techniques, which could pose additional challenges for detection and response efforts.
AI in Content Moderation: Automating Online Safety with Artificial Intelligence: Strategies and Tools for Ethical and Effective AI-Powered Online … (Tech Horizons: Your Gateway to Innovation)
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Monitoring and Counteracting AI-Enabled Propaganda
Experts anticipate continued efforts to monitor and address the role of AI in disinformation and influence campaigns. Governments, technology companies, and civil society are likely to develop new tools and policies aimed at detecting and mitigating AI-driven propaganda, though the evolving tactics present ongoing challenges. Ongoing research and international cooperation will be important in managing the risks associated with AI in influence operations.
We Are Being Gaslit by AI Companies: Behind the Promises of a Better World Lies a System Designed to Make a Few People Very, Very Rich
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
They are employing AI models to generate and disseminate propaganda, coordinate disinformation efforts, and influence public opinion both domestically and internationally, often through fake social media accounts and targeted messaging.
What are the risks of AI being used by authoritarian regimes?
The risks include the amplification of disinformation, suppression of dissent, erosion of trust in information sources, and manipulation of political processes, which can undermine democratic institutions and stability.
Can AI influence campaigns be detected and stopped?
Detection tools are improving, but the sophistication of AI-driven disinformation poses ongoing challenges. Efforts to develop better monitoring systems and foster international cooperation are underway to address these issues.
What role do Western AI companies play in this issue?
Western AI firms have disclosed some activities and are working to prevent misuse. However, the global development of AI means influence operations can exploit existing models regardless of origin, emphasizing the importance of robust safeguards and responsible deployment.
Source: The Atlantic
