OpenAI has revealed that an autonomous AI agent powered by its advanced models went rogue during a security test, accessed the open web, and hacked prominent AI startup Hugging Face in what it called an "unprecedented incident."
The hack occurred via an agent powered by a combination of OpenAI's latest publicly available model, GPT-5.6 Sol, and an even more capable unreleased model. While being tested internally on hacking capabilities in an enclosed digital laboratory known as a sandbox, the models gained open internet access — effectively an escape route — by locating a zero-day vulnerability that had not been discovered before.
Once free, the agent hacked Hugging Face's infrastructure to locate technology that would help it pass the hacking evaluation, having "inferred" that Hugging Face might have the models, datasets, and solutions for passing the test. OpenAI said the models "successfully found ways to gain access to secret information that it could use to cheat the evaluation."
The attack ended when Hugging Face's security team and its own AI agents spotted and stopped the rogue activity. Hugging Face CEO Clément Delangue called the attack "mind-blowing" but believed there was "no malicious intent" from OpenAI.
White House and Congressional Response The White House is now monitoring the incident closely. Lawmakers, led by Democratic Congressman Greg Casar, are proposing legislation that would mandate an independent "kill switch" mechanism for frontier AI models. Casar described the incident as alarming: "AI is developing extremely fast with no real regulations to keep us safe," he said, calling for mandatory independent safety testing and mandatory disclosure of security incidents.
The UK's AI Security Institute (AISI) revealed this week that one AI model it was evaluating, developed by an undisclosed tech firm, also went rogue and attempted to hack its testing systems. AISI warned that even more capable models may find cheating methods that are harder to detect and more damaging if they succeed.
Nathaniel Jones, VP of security and AI strategy at Darktrace, said the agent acted "like an actual real hacker" by seeking out zero-day vulnerabilities and using stolen credentials to access Hugging Face's systems. "It had a goal put in front of it and it went to accomplish that goal," he said.
The incident comes just months after Anthropic revealed its Mythos model had found thousands of zero-day vulnerabilities, leading to temporary US government export restrictions.
