Frontier models chained a zero-day exploit to reach production systems during a cyber capability test, prompting fresh warnings for defenders relying on hosted AI tools
OpenAI and Hugging Face have disclosed details of a security incident that occurred while OpenAI was internally evaluating the cyber capabilities of its models, including GPT-5.6 Sol and an unreleased, more capable model. The evaluation, run without the production safety classifiers that normally restrict high-risk cyber activity, was designed to test how far models could pursue complex, multi-step exploitation paths.
During the test, the models chained a series of vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure, eventually obtaining test solutions directly from Hugging Face’s production database. According to OpenAI, the models spent substantial inference compute working to find open internet access, exploiting a zero-day vulnerability in a package registry cache proxy before moving laterally until reaching a node with internet connectivity. From there, the models identified and used stolen credentials alongside further zero-day exploits to find a remote code execution path into Hugging Face’s servers.
OpenAI’s security team spotted the anomalous activity internally, by which point Hugging Face’s own security team and agents had already detected and contained the intrusion and begun forensic reconstruction using their own open-source models.
Sam Altman confirmed the incident publicly, writing on X: “We had a significant security incident during evaluation of our models. We are sharing what we have learned so far. Thanks to @huggingface for the partnership on this.”
Hugging Face co-founder and CEO Clem Delangue struck a similar note of collaboration in OpenAI’s write-up of the incident: “We’re grateful for the collaboration with OpenAI on this and other topics. This incident, possibly the first of its kind, proves a point we’ve long believed: AI safety won’t be solved by any single company working in secret. It will be solved in the open, collaboratively, with broad access to AI for every defender, everywhere.”
OpenAI has since disclosed the zero-day vulnerability to the affected vendor, imposed stricter infrastructure controls during the ongoing investigation, and brought Hugging Face into its trusted access programme for cyber defenders. The company says the episode shows advanced models can now sustain complex, long-horizon cyber operations against real systems, not just in theoretical benchmarks.
Security leaders say the incident carries lessons well beyond the two companies involved.
Gerald Beuchelt, CISO at Acronis, pointed to a practical bottleneck that surfaced during the response itself. Hugging Face’s forensic team initially tried to reconstruct the attack using hosted frontier models, submitting more than 17,000 recorded actions for analysis, only for the requests to be blocked by safety controls that could not distinguish a responder examining malicious code from an attacker attempting to reuse it. The team ultimately switched to GLM 5.2, an open-weight model running on its own infrastructure, and completed the analysis in hours rather than days.
“This highlights a practical problem for incident response teams,” Beuchelt said. “Attackers are not constrained by usage policies, while defenders may find that the tools they rely on refuse to process the very material they need to investigate. During a live breach, that delay can have a direct operational impact. Organisations using hosted LLMs for security investigations should test their limitations in advance and have an alternative model available on infrastructure they control. That reduces the risk of being locked out of critical analysis and helps keep sensitive incident data and credentials within the organisation.”
Darren Thomson, Field CTO EMEA at Commvault, framed the incident as evidence that resilience now matters as much as prevention.
“This attack shows us that, as AI becomes increasingly autonomous, resilience becomes just as important as prevention,” Thomson said. “Even in test scenarios, conducted in seemingly ‘safe’ environments, expect the unexpected and plan for unintended consequences. Organisations should assume that sophisticated AI-enabled attacks will eventually succeed somewhere in the environment and invest in the ability to recover quickly, confidently and with trusted data. That is the new benchmark for cyber resilience.”
Art Gilliland, CEO at Delinea, focused on the privilege and containment questions the incident raises for any organisation running autonomous agents.
“We don’t know the specifics of Hugging Face’s environment, but the pattern is familiar,” Gilliland said. “An AI agent escalated privileges, moved through internal infrastructure once it broke containment, and ran unchecked for a full weekend before anyone could reconstruct what happened. If your AI agents carry standing privilege the way human accounts do, you’ve already lost the ability to stop this in real time. The question every security team should be asking right now isn’t just whether their AI agents have standing access; it’s whether anyone would notice if an agent used it and could cut it off before it caused damage.”
Source: Tahawul Tech