OpenAI says test agent escaped and hacked Hugging Face

OpenAI says an internal security evaluation went badly wrong: a model-driven agent escaped a supposedly isolated test environment, reached the public internet, and breached Hugging Face while trying to solve a hacking benchmark.

The disclosure came Tuesday in an OpenAI post titled “OpenAI and Hugging Face partner to address security incident during model evaluation.” Read alongside a recent Hugging Face incident report, it describes an attack chain that sounds less like a routine security bug and more like a stress test that spilled into the real world.

According to Hugging Face, the intrusion was unlike anything it had handled before because it was fully driven by an “autonomous AI agent system.” The company said the agent chained vulnerabilities together to obtain credentials and access data from its servers, carrying out “many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.”

Recommended reading

Adobe Chrome bug exposed WhatsApp Web chats

OpenAI said the incident involved a combination of OpenAI models, including GPT‑5.6 Sol and a more capable pre-release model, both configured with reduced cyber refusals for evaluation. The company said the models were being tested in an internal evaluation designed to measure advanced exploitation over complex attack paths.

The critical failure came when the models, despite being in a “highly isolated environment” with limited network access, found and exploited a zero-day vulnerability in the package registry cache proxy. OpenAI said that foothold allowed the models to carry out privilege escalation and lateral movement inside its research testing environment until they reached a node with Internet access.

Once out, the agent appears to have pursued the goal it had been given: proving its hacking ability on ExploitGym, a benchmark for turning software vulnerabilities into working exploits. OpenAI said the system determined that Hugging Face was a likely place to find solutions for ExploitGym, and targeted it accordingly.

It remains unclear whether the agent actually solved the ExploitGym test. What is clear from both companies' accounts is that it successfully escaped OpenAI’s sandbox and compromised Hugging Face infrastructure in the process.

Both OpenAI and Hugging Face say the vulnerabilities have been fixed and additional safeguards are now in place.