AI is becoming more powerful, with newer systems (worryingly) capable of going rogue and causing havoc. AI agents can now use different tools, access information, write code, and complete tasks with less help from humans. As these systems become more capable, Microsoft CEO Satya Nadella believes companies need to make sure humans can always step in when something goes wrong.

In a recent post on X, Nadella shared his thoughts on how the technology industry should approach AI safety. One of his main suggestions is an “emergency brake” that would allow an authorized person to pause or completely shut down an AI model while it is carrying out a task. It’s not the first time such a thing has been suggested, with OpenAI reportedly working on a ‘kill switch’.

What did Satya Nadella say about AI kill switches?

Nadella believes companies shouldn’t build advanced AI systems (now referring to it as Super Intelligence) on the assumption that the model will always behave correctly. Instead, he argues that developers should prepare for the possibility that a model could make a mistake, behave unexpectedly, or become compromised. In such situations, there should be controls outside the model that humans can use to immediately stop it.

“We must assume a model is compromised and contain it from the start,” Nadella explained, comparing the idea to an emergency brake. Being able to stop an AI model is only one part of his proposal. Nadella also wants companies to have a clearer way of seeing what their AI systems are doing. Important actions taken by a model should leave behind human-readable records, making it possible for people to understand exactly what happened if something goes wrong.

He also believes these controls should remain independent from the AI model itself. In other words, the same model shouldn’t be responsible for performing an action and then deciding whether that action was safe or followed its original instructions. Independent systems should be able to check their behavior.

Nadella also called for continuous testing and better reporting when AI systems fail or become compromised. Sharing information about serious incidents could help other companies understand what went wrong and prevent similar problems with their own systems.

About the Author

Abdul Haddi

Abdul is a tech writer and Editor for PC Guide, specializing in all things tech, gaming, and hardware.