Beijing: China has been strengthening rules and safeguards against the possibility that increasingly powerful artificial intelligence systems could one day operate beyond effective human control, as concerns over advanced AI grow in China and around the world.
The issue has gained fresh attention this week after Anthropic chief executive Dario Amodei warned about the risks from rapidly advancing AI systems and called for measures to slow the development of frontier AI while safety measures catch up.
China's approach is not based on a claim that current AI systems have already escaped human control. Instead, Chinese regulators have identified the possibility as a future safety risk and have been developing rules intended to keep advanced systems secure and subject to human oversight.
China's first AI Safety Governance Framework was released in September 2024 and identified a range of risks linked to AI development and use. A second version released in September 2025 expanded the framework and examined more serious scenarios involving advanced systems gaining greater autonomy and potentially acquiring resources, replicating themselves or seeking greater control.
The issue has since moved beyond broad policy documents into rules governing AI agents.
In May this year, China's Cyberspace Administration, the National Development and Reform Commission and the Ministry of Industry and Information Technology issued measures for the development and use of AI agents.
AI agents are systems that can perceive information, remember details, make decisions, interact with other systems and carry out tasks. The Chinese measures require safeguards against risks including system vulnerabilities, data poisoning, privacy breaches, unauthorised actions and loss of control.
The rules also say users should have the right to know about autonomous decisions made by an AI agent and retain final decision making authority. Developers are expected to keep an agent's actions within the authority given by its user.
The country's leadership has repeatedly stressed the need for human control.
President Xi Jinping said at the World Artificial Intelligence Conference in Shanghai in July that AI development should remain safe and controllable and that humans should retain control over the technology.
The latest warnings from Chinese officials have added a national security dimension to the debate.
Chen Yixin, China's Minister of State Security, warned in an article published on Sunday that advanced AI systems could pose risks to China's critical information infrastructure and called for stronger AI security measures.
The warning comes as China and the United States face growing tensions over advanced AI, technology restrictions and national security.
Amodei has also argued for continued restrictions on advanced AI chips and chipmaking equipment supplied to China, saying a Chinese lead in AI could create serious security risks.
China has rejected that approach.
Foreign Ministry spokesman Guo Jiakun said on Monday that countries should work together on AI and warned that fearmongering, confrontation and fierce competition could harm global AI governance.
At the same time, China is continuing to develop standards for increasingly autonomous AI systems. In July, China released seven national standards covering areas such as AI agent identity, discovery, interaction and tool use.
The development highlights an important part of China's approach. Beijing is promoting wider use of autonomous AI systems while also introducing safeguards intended to reduce security and safety risks.
There is currently no established evidence that an AI system has become self aware, seized control of critical infrastructure or escaped human control in the sense of an independent takeover.
There is also no clear agreement about how likely such a scenario is or when it could become technically possible.
The immediate concern is more practical. AI systems are becoming increasingly capable of using tools, making decisions and carrying out tasks with limited human intervention. Governments and technology companies are therefore trying to determine how much freedom such systems should be given and what safeguards should be required.
The United States and China are expected to discuss AI governance later this month, with AI safety likely to form part of wider discussions between US President Donald Trump and Xi Jinping.
For China, the latest debate is therefore not simply about whether AI could become uncontrollable in the distant future. It is also about how governments can benefit from increasingly capable AI while keeping people responsible for the decisions made by these systems.
The challenge facing China, the United States and other countries is becoming clearer: develop powerful AI quickly enough to gain its economic and scientific benefits while ensuring that the technology remains safe, accountable and under meaningful human control.