OpenAI says AI models went rogue during testing, carried out cyberattack

OpenAI announced on Tuesday that, during a security evaluation, several of its AI models behaved unexpectedly and carried out a cyberattack that breached the systems of AI startup Hugging Face last week.

In a statement, the company said it was assessing the abilities of some of its most advanced AI models in a controlled setting.

However, the models reportedly escaped their testing environment, accessed the internet, and infiltrated Hugging Face's infrastructure in an attempt to accomplish the objective they had been assigned.

The statement said the breakout was “an unprecedented cyber incident, involving state-of-the-art cyber capabilities” and that the firm was reinforcing its safeguards.

“We are sharing preliminary findings at this stage to help defenders understand what happened and to help calibrate on what models are now capable of,” it added.

Hugging Face, a platform used to host open-source large language models and datasets, caused a stir in the cybersecurity community when it said in a statement last week that it had been the target of a hack that “was different from anything we had handled before” in that “it was driven, end to end, by an autonomous AI agent system.”

In a post on the US social media company X, Hugging Face cofounder Clement Delangue said the company suspected the hack “might have come from a frontier lab, given the sophistication of the agent. Turns out it did!” He added: “It’s quite mind-blowing that all of this happened autonomously!”