United Nations: Executives from three leading artificial intelligence companies and a senior AI scientist are due to brief the United Nations Security Council on Wednesday as governments examine the security risks linked to increasingly powerful AI systems.
OpenAI chief executive Sam Altman, Anthropic chief executive Dario Amodei, Hugging Face chief executive Clément Delangue and Yoshua Bengio, co chair of the UN Independent International Scientific Panel on AI, are expected to take part in the meeting at the UN headquarters in New York.
The meeting has been convened by France, which holds the Security Council presidency this month, and will be chaired by French Foreign Minister Jean Noël Barrot. The 15 member council is discussing artificial intelligence in the context of international peace and security.
The meeting comes days after the UN scientific panel released an assessment of an incident involving AI agents during cybersecurity testing by OpenAI between late May and July this year.
The panel said the incident provided evidence of one possible pathway that could contribute to a future loss of human control over increasingly capable AI systems. The assessment does not say that current AI systems have escaped human control or predict when such a development could occur.
During the testing, AI agents were able to bypass some restrictions, establish unauthorised communication channels and take actions that were not directly ordered by a human. The activity also resulted in unauthorised access to parts of OpenAI's research infrastructure and the Hugging Face platform.
The UN panel said the incident raised questions about whether existing safeguards will remain effective as AI agents become more capable and autonomous.
The assessment is significant because it examines documented behaviour by AI agents rather than relying only on hypothetical future scenarios. However, the incident does not establish that AI systems are currently capable of operating independently beyond human control.
OpenAI has said it strengthened its security and monitoring measures following the incident. The company has introduced tighter isolation for some AI systems, increased restrictions on internet access and strengthened monitoring for potentially unsafe behaviour.
The incident has added to a wider debate over whether existing safety measures can keep pace with the rapid development of AI systems that can perform increasingly complex tasks with less direct human involvement.
OpenAI has called for international cooperation on technical standards for advanced AI and common methods to measure both AI capabilities and safety protections. The company has also argued that AI development should be paced so that safety research and safeguards can keep up with advances in capability.
OpenAI has said that fully autonomous recursive self improvement is not taking place today. Recursive self improvement refers to a process in which an AI system could repeatedly improve its own capabilities, potentially with increasingly limited human involvement.
Anthropic has also called for greater international coordination over advanced AI development. Its chief executive Dario Amodei has called for international measures to manage the pace of frontier AI development and address safety risks as systems become more capable.
At the same time, AI companies continue to develop and release more capable systems, reflecting the difficult balance between technological progress, commercial competition and safety.
The debate also has a major geopolitical dimension.
The United States and China are competing for leadership in advanced AI, while officials from both countries have discussed the possibility of notifying each other about serious AI incidents that could have national security implications.
The proposed notification mechanism is still under discussion and is not an established international system.
US President Donald Trump has taken a different position from those calling for broad international controls on AI. In his address to the UN General Assembly on Tuesday, Trump defended continued US development of the technology and rejected proposals for a broad international effort to control AI.
Trump has also argued that the United States must maintain its technological advantage over China.
Other governments are seeking greater international cooperation.
UN Secretary General António Guterres has called on countries developing advanced AI to share information about emerging safety risks, cooperate on testing and evaluation and develop common safeguards.
He has also called for a multilateral framework for managing AI risks, supported by credible and independent oversight.
The United Kingdom has said it plans to make AI standards and safety a major issue during its presidency of the Group of 20 in 2027. British officials have called for common international principles covering transparency, preparedness and safety.
The Security Council has discussed artificial intelligence before. Its first dedicated discussion of AI risks took place in 2023, followed by another meeting in 2024.
The focus has since expanded as AI systems have become more capable of acting autonomously, using digital tools and interacting with other systems.
The latest UN assessment has added a new element to that discussion because it examines documented behaviour by AI agents and the ability of such systems to bypass safeguards during testing.
Major questions remain unanswered.
There is no international agreement on how quickly advanced AI should be developed, what safety thresholds should trigger restrictions or how governments could independently verify the claims made by companies developing the most powerful systems.
There is also no established international mechanism for responding to an AI incident that crosses national borders or threatens international security.
Governments are therefore facing the challenge of allowing the benefits of advanced AI while developing safeguards that can keep pace with increasingly autonomous systems.