Two Crises Beyond Convenient AI: Runaway and Power Concentration, Issues Raised at the United Nations

Companies Creating AI Call for Global "Governance"

AI, which is used because it's convenient and implemented because it speeds up work, was highlighted as a risk affecting all of humanity at a forum discussing international peace and security.

On September 23, 2026, Sam Altman, CEO of OpenAI, and Dario Amodei, CEO of Anthropic, spoke about AI at the United Nations Security Council. The following day, Germany's AD HOC NEWS reported that both CEOs emphasized the necessity of international rules.

What is noteworthy is that the warnings came not only from those opposed to AI but also from the leaders of companies advancing its development.

However, this is not a declaration that "all AI has already escaped human control." It is a discussion on how to prevent losing control in the future as capabilities improve. It's important to distinguish between warnings and actual events.

The Danger Is Not Just "AI Runaway"

Altman's concerns, as introduced in the original article, can be broadly divided into two categories.

One is that AI's operations and development might become too rapid for humans to understand or intervene. The other is the concentration of excessive power in the hands of a few companies or nations that control powerful AI.

Though these issues seem similar, their roots are different.

The former questions whether humans can stop AI, while the latter questions whether society can supervise the humans operating AI. Even if AI operates technically as instructed, if only a few can issue those instructions, it may not be a desirable system for society as a whole.

Thus, AI safety is not just about reducing dangerous responses. It also involves designing authority and responsibility: who has access rights, who decides operational policies, and where those adversely affected can appeal.

The Weight of the Promise to "Delay Release"

According to the original article, Amodei expressed a willingness to delay the introduction of future models if necessary for safety. He also called for international bans on AI-used biological weapons, testing and verification mechanisms, and a global accident reporting system.

What should not be confused here is the stance of "delaying if necessary" with the fact of "a specific product's delay being decided." The statements introduced do not imply changes in individual release dates.

Moreover, prioritizing safety confirmation does not equate to completely eliminating danger. The questions to be asked are what to investigate, under what results to forgo release, and who verifies those decisions.

In a separate published essay, Amodei also proposed a concept where third-party evaluators have continuous access to development sites to verify safety measures and accidents. It is important to distinguish the adjustment of development pace from a complete halt in technological progress.

In a market where releasing new products quickly is advantageous, it is difficult for one company to be cautious alone. Therefore, it is necessary to have evaluation conditions common to competitors, not just relying on corporate goodwill.

The Discussion on "Self-Improvement" Is Not to Predict the Future

The original article also introduces concerns about "recursive self-improvement," where AI contributes to the improvement of the next AI, accelerating further progress.

The issue is the gap between the speed of capability growth and the speed of ensuring safety. Even if performance improvements can be confirmed, it does not necessarily mean that operational weaknesses or unexpected behaviors can be understood at the same speed.

However, such concerns should not be turned into predictions like "a catastrophe will definitely occur in a few years." The short-term outlook in the original article is the speaker's prediction, not a confirmed schedule.

What readers should focus on are the grounds for these predictions and the safety measures that would be useful even if the predictions are wrong, rather than sensational deadlines.

On Social Media, Questions Arise About "Companies Issuing Warnings" Rather Than the Warnings Themselves

Reactions observed on social media are not monolithic. Here, comments on posts introducing the recent meeting and reactions to related AI safety discussions are separately introduced. Both are limited to publicly available posts and do not indicate overall public opinion trends.

A comment on CNBC's LinkedIn post, which announced the Security Council meeting, suggested that there might still be missing participants who should be involved in the discussions. This reflects a concern not only about who speaks on safety but also whose voices are included in the forum.

Meanwhile, in Reddit's investment-related community, there were posts questioning whether OpenAI's explanation of delaying its IPO due to safety concerns was purely about safety, suggesting there might be business reasons as well.

In the same thread, there were comments countering the view that technological progress has plateaued, arguing instead that it is accelerating. This is not a direct reaction to the recent UN speech but a response to the underlying safety discussions. It is important to distinguish between management and technical evaluations in the posts and verified facts.

What these posts suggest is that trust cannot be established merely by explaining dangers. Companies have expertise, but they also have competitive interests. A system that allows claims to be verified externally, considering both aspects, is required.

Is Strong Regulation Alone Reassuring?

Strict safety confirmations incur costs for tests and audits. Depending on the design, only companies with financial resources may be able to comply, potentially disadvantaging new entrants.

This is not a reason to forgo regulation. It means that the mechanisms to ensure safety must not end up only protecting the status of existing large companies.

In Altman's published speech transcript, it is also stated that standards should not fix existing companies or favor a single business model.

Balancing the independence of third-party evaluations, transparency of standards, and burdens according to risk will determine the reliability of the system. It is crucial not only what companies say is "safe" but also who can verify it and to what extent.

The "Authority to Stop" Also Concerns Japanese Users

This discussion is not just for large research institutions or the UN. Japanese companies can apply the same questions on a smaller scale when entrusting work to AI.

For example, the necessary confirmations differ between having AI draft response proposals and allowing it to execute refund processes or change customer data. This is not a specific example from the recent meeting but an illustration of applying the discussion to everyday operations.

Who approves execution? Who detects anomalies? If stopped midway, how much can be reverted? Performance comparisons alone cannot answer these questions.

Even while waiting for the completion of international rules, users can concretely decide the authority given to AI and the responsibilities humans take on.

What Is Needed After the Speech Is Verifiable Action

The warning issued has significance. However, safety cannot be measured by the strength of the warning alone.

Can external evaluators access sufficient information? Can inconvenient results be published? Can development or release truly be delayed when dangers are found? Are the judgment criteria explained in advance?

The future where AI expands human possibilities and the future where humans lose decision-making power are not determined by performance alone.

Who will transform the warnings issued by AI-creating companies at the UN into verifiable actions, and how? That is where attention should be focused next.

Source URL

1. AD HOC NEWS (September 24, 2026). Warnings from both CEOs, Amodei's stance on delaying model introduction, proposals for verification and accident reporting systems, etc. The text can also be confirmed in the provided attached materials.
   https://www.ad-hoc-news.de/wissenschaft/openai-und-anthropic-vor-der-un-chefs-warnen-vor-unkontrollierter-ki/70171896

2. OpenAI (September 23, 2026). Transcript of Altman's speech at the UN Security Council. Primary source on human control, power concentration, and avoiding the fixation of international standards and existing companies.
   https://openai.com/index/sam-altman-un-security-council-remarks/

3. Dario Amodei "We Must Pace the Frontier" (September 2026). The author's essay explaining the adjustment of development pace and the concept of third-party evaluators having continuous access to development sites. Background material separate from the recent meeting.
   https://darioamodei.com/post/we-must-pace-the-frontier

4. United Nations Official Website (September 23, 2026). Official announcement introducing the briefing to the Security Council by OpenAI and Anthropic.
   https://www.un.org/en/live-openai-and-anthropic-brief-security-council-amid-%E2%80%98real-and-imminent%E2%80%99-threat-posed-runaway-ai

5. CNBC's LinkedIn Post. A post announcing the Security Council meeting and public comments pointing out the lack of participants. Used as an example of social media reactions.
   https://www.linkedin.com/posts/cnbc_altman-and-amodei-expected-to-join-un-security-activity-7508491118909562880-PApn

6. Reddit "r/stocks". Reactions to separate reports on OpenAI's IPO timing and AI safety. Confirmation of skepticism towards corporate explanations and counterarguments regarding technological progress. Not a direct reaction to the recent UN speech, nor a document for fact-checking the claims in the posts.
   https://www.reddit.com/r/stocks/comments/1wexcu6/openai_ipo_will_not_happen_in_2026_amid_ai_safety/

※Date of confirmation: September 24, 2026. The summary of social media reactions is based on publicly available content and is not a statistical public opinion survey.