Nadella Urges AI Emergency Brake as Researcher Puts Extinction Risk Above 10%
Updated
Updated · CNBC · Oct 10
Nadella Urges AI Emergency Brake as Researcher Puts Extinction Risk Above 10%
3 articles · Updated · CNBC · Oct 10
Summary
Satya Nadella said advanced AI systems need an "emergency brake" so authorized people can pause or shut models down mid-task, alongside containment and independent controls.
In an X post, the Microsoft CEO argued non-deterministic models should be wrapped in deterministic system design, human oversight and industry standards, treating frontier closed and open-weight models like insider risks.
Nadella said trustworthy AI should be built around observability — including model diversity, human-readable action logs, continuous testing, auditability, containment and incident disclosure.
The call comes as safety alarms intensify: an Anthropic researcher quit last month accusing Anthropic and OpenAI of "gambling with our lives," and an Anthropic alignment lead later put the chance AI could kill all humans within a decade above 10%.
That debate cuts against President Donald Trump's stance that extinction fears are overblown and that U.S. policy should focus on staying ahead of China, a push he recently tied to a new AI Force.