I Met With China’s Top AI Experts. They’re Freaking Out, Too

Chinese AI experts are voicing significant concerns about the rapid advancement of artificial intelligence, warning of systemic risks and calling for increased international cooperation, according to reports from a recent conference in Beijing.

At a major AI conference organized by the Beijing Academy of Artificial Intelligence, leading researchers and industry insiders discussed the growing dangers associated with increasingly capable AI models. Experts highlighted vulnerabilities in AI-generated code, new attack vectors enabled by agentic models, and the potential use of AI as cyber weapons. An anonymous researcher from Shanghai Jia Tong University emphasized that hackers are likely to gain an advantage in the near term, but that new AI-driven countermeasures could restore defensive capabilities over time.

The conference reinforced the idea that both the US and China are at a critical juncture, with rapid AI development posing systemic risks that could lead to cyberattacks or catastrophic failures. Despite ongoing competition, many experts argued that international collaboration is essential to establish safety standards and reduce global risks. A Chinese industry insider noted that some advanced models are no longer being released as open source due to security concerns, underscoring the tension between openness and safety in AI development.

Why Chinese AI Experts’ Concerns Matter Globally

This development signifies a growing awareness among Chinese AI leaders about the potential dangers of unregulated AI progress. Their calls for international cooperation highlight the importance of joint efforts to establish safety standards and prevent AI from becoming a systemic threat. As China and the US lead global AI development, their shared responsibility could determine whether AI advances bring benefits or pose catastrophic risks worldwide.

Artificial Intelligence for Cybersecurity: Develop AI approaches to solve cybersecurity problems in your organization

AmazonView Latest Price

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Rising AI Capabilities and International Tensions in China

Over recent years, Chinese companies have led the way in open-weight AI models, such as Moonshot’s Kimi and Alibaba’s Qwen, gaining popularity in the US and globally. Meanwhile, the US has ramped up its own open AI initiatives, like Nvidia’s Nemotron. However, the rapid development of frontier capabilities—such as agentic and coding features—raises concerns about misuse, security vulnerabilities, and the potential for cyberattacks. The recent development of Chinese models with hacking capabilities comparable to those of US models underscores the increasing risks involved.

Despite the competitive landscape, experts stress that collaboration on safety standards and risk mitigation remains crucial, especially as models become more powerful and intertwined with critical infrastructure.

“If different countries understand the risks in similar ways, it becomes easier to develop shared safety principles and technical standards.”

— an anonymous researcher

Industry 5.0 for Occupational Health and Safety and Risk Mitigation (Studies in Systems, Decision and Control, 647)

AmazonView Latest Price

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Unclear Scope of Chinese AI Model Releases

It remains uncertain how widespread the practice of restricting open-source releases of advanced Chinese AI models is, and whether security concerns will lead to more models being kept proprietary. Details about specific models and their capabilities are still emerging, and some experts suggest that the full extent of security measures remains undisclosed.

Practical AI Governance: Building a Program for Oversight and Strategy

AmazonView Latest Price

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Next Steps in International AI Safety Collaboration

Moving forward, industry leaders and governments are expected to prioritize establishing shared safety standards and risk mitigation strategies. Ongoing discussions at international forums and bilateral agreements between the US and China could shape the future of safe AI development. Monitoring developments in model capabilities and security practices will be critical in the coming months.

Train It. Tame It. Teach It.: Build Your Personal AI Team and Get Every Model to Work Your Way

AmazonView Latest Price

As an affiliate, we earn on qualifying purchases.

As an affiliate, we earn on qualifying purchases.

Key Questions

Why are Chinese AI experts concerned about AI development?

They are worried about systemic risks, cybersecurity vulnerabilities, and the potential misuse of advanced AI models, which could cause chaos or harm if not properly managed.

What is the significance of international cooperation in AI safety?

International collaboration can help establish safety standards, reduce systemic risks, and prevent AI from becoming a global threat, especially as both the US and China lead AI advancements.

Are Chinese models being restricted from open release?

Yes, some Chinese companies have stopped releasing certain advanced models as open source due to security concerns, though details are still emerging.

What are the risks of open-weight AI models?

Open models can be exploited by hackers to identify vulnerabilities, develop cyber weapons, or conduct social engineering attacks, especially as they incorporate frontier capabilities.

What is likely to happen next in AI safety efforts?

Expect increased international dialogue on safety standards, joint efforts to develop secure models, and ongoing monitoring of model capabilities and security practices.

Source: WIRED