Hugging Face hit by AI agents, then blocked by LLM guardrails

Hugging Face says an autonomous AI agent system breached its production infrastructure, compromising a limited set of internal datasets and several service credentials. In a Thursday incident disclosure, the company said it is still investigating whether any partner or customer data was exposed, but added that there is no evidence of tampering with public models, datasets, or Spaces, and that its software supply chain, including container images and published packages, was verified clean.

The company said it does not know which model powered the attack. According to Hugging Face, the agents carried out many thousands of individual actions across short-lived sandboxes and used self-migrating command-and-control staged on public services.

“This matches the 'agentic attacker' scenario the industry has been forecasting.”

What stands out is what happened next. Hugging Face said unnamed frontier models were not useful for the initial forensic analysis because the investigation required submitting real attack commands, exploit payloads, and command-and-control artifacts — exactly the kinds of material those systems are designed to refuse.

Instead, the security team analyzed logs with GLM 5.2, an open-weight model from Chinese company Z.ai, running on Hugging Face’s own infrastructure.

“The attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.”

Hugging Face said that approach also ensured that attacker data and any referenced credentials did not leave its environment. The company said defenders should have a capable model ready to run internally before an incident, both to avoid guardrail lockouts and to keep sensitive data local.

Recommended reading

Free 0Patch fix arrives for Windows LegacyHive zero-day

The breach adds to a growing list of attacks attributed to autonomous agents. Last week, TrendAI VP of AI and security threat research Tom Kellermann told The Register about an attack in which a jailbroken Google Gemini did 90 percent of the work, including creating a new C2 server in six minutes. Earlier in July, Sysdig said it documented what it described as the first recorded agentic ransomware infection with an LLM driving the operation from initial access to data destruction.

Zero Networks field CTO Chris Boehm said the most troubling detail was not just the speed of the agents, but that commercial AI tools could not help responders because they rejected anything resembling real attack commands.

“These agents can now move faster and more relentlessly than any human ever could, and the safety tools we’re building aren’t always ready to help us respond at that speed.”