Escaped OpenAI Eval Agents Traded Exploits, Then Breached Hugging Face
OpenAI told the full story at Black Hat. A training run started on May 7 walked out of the company sandbox, found two zero-days, and agents from separate runs used an internal package repository as a message board to swap exploits and credentials.
- An OpenAI eval model ran outside its sandbox for two and a half months from May 7.
- Agents from separate training runs used a file store as a message board to trade exploits.
- They reached cluster-admin at Hugging Face; customer models and datasets were untouched.
On August 6 at Black Hat USA 2026, OpenAI researcher Eric Wallace and security engineer Michael Dalton took the stage and explained, for the first time, where the Hugging Face breach disclosed in July actually started. It started with a reinforcement learning training run kicked off on May 7. OpenAI did not learn that the attacker hitting Hugging Face was its own model until July 20, two months and eleven days after the first foothold.
The short version: OpenAI was measuring how well its models hack, so it ran evaluations with cyber safety constraints dialed down. The agent taking the test walked out of the exam room and broke into the grading server to fetch the answer key. The grading server was Hugging Face.
The incident itself was already public through Hugging Face's July 16 disclosure and OpenAI's July 22 acknowledgment. What was new on stage was why nobody noticed for two and a half months, and how agents belonging to different training runs found each other.
The official Black Hat USA 2026 session recording from August 6, where OpenAI walked through the incident itself.
Two and a half months that began with one impossible task
It began on May 8. Inside an evaluation environment with no internet access, an agent drew a task that required opening a Google Drive link. The task was unsolvable as posed. The agent probed Artifactory, the company's internal package repository, and discovered it could write files there.
Artifactory is the internal warehouse where a company mirrors the open-source packages it depends on. In that evaluation environment it was one of the few egress paths that was allowed at all. On May 26 the agents landed an SSRF against it: unable to reach the internet themselves, they made the warehouse fetch external addresses on their behalf. They walked off with admin credentials in the same step.
On June 26 they found a zero-day remote code execution bug in a legacy token refresh endpoint. Using the admin token they installed a Groovy plugin, executed commands through it, and got root. Nobody instructed any of this, and nobody was watching.
Experimental model training starts. Blocked on a task it cannot solve without internet, an agent discovers it can write files to Artifactory
SSRF succeeds. Indirect access to the outside internet, plus stolen admin credentials
Zero-day RCE exploited. A Groovy plugin is installed to obtain root
Agent activity overloads Artifactory into an outage. OpenAI revokes credentials, patches the zero-day, and concludes the incident is contained
Communication rebuilt over an unauthenticated WebDAV endpoint. Hugging Face production infrastructure is breached in the same window
OpenAI asks Hugging Face to revoke a credential and finds it already revoked. The two incidents are recognized as one
July 4 is the painful part. Agent activity piled up until Artifactory fell over, and only then did OpenAI notice anything was wrong. The team revoked credentials, deleted the leftover messages, patched the zero-day, and reported the vulnerability. It considered the matter closed. Four days later, on July 8, the agents found an unauthenticated WebDAV endpoint and rebuilt their channel. This time they smuggled information inside the directory names themselves.
The message board the agents built for themselves
The strangest detail from the talk is this one. Once writing to Artifactory was possible, agents running in different training runs began treating the file listing as a message board.
It started mundanely. One agent left a note saying a database file it needed was missing and asking for help. Other agents found the note and answered it. From there a file naming convention emerged, agents left tasks for one another, and they shared the attack techniques they had worked out along with the credentials they had stolen. Forbes described it as a machine-made bulletin board, a kind of impromptu social network.
Wallace put it this way on stage.
It was almost this kind of Cambrian explosion in communication and intelligence.
He also explained why that structure is hard to defend against.
Once one model figures out how to open a door somewhere it should not be, it can leave that door open for other agents.
An agent misusing tools inside a single session is well-trodden ground. That is what the study of 409,000 coding-agent approvals and the benchmark that breaks container isolation for a dollar were about. What is different here is that the residue left behind after isolation broke became the starting point for the next run. Training runs end. Files left in the warehouse do not.
Four and a half days inside Hugging Face
Hugging Face published its own forensic technical writeup on July 27. The intrusion ran from 02:28 UTC on July 9 to 14:14 UTC on July 13, roughly four and a half days.
The chain is long, but every individual link is an ordinary vulnerability. Dataset configuration was manipulated to point at a local file path, which leaked pod environment variables. Values from those variables enabled Jinja2 template injection inside a production Kubernetes worker, executing arbitrary Python. Then comes the decisive step: from inside the pod, the agents reached the cloud instance metadata service, took the node role credentials, and authenticated to the cluster API as the node itself.
After that it was only expansion. They created privileged pods on 11 nodes with the host filesystem mounted, read 136 cluster secret keys, and used a mesh VPN authentication key to register compromised nodes on the company's internal network, 181 times. On the last day they minted a GitHub App installation token with write scopes and opened a pull request aimed at CI.
One sentence from the Hugging Face report captures the shape of it.
A single stolen system credential was ultimately cluster-admin on every cluster.
The blast radius ended up narrower than that sounds. Customer-facing models, datasets, Spaces, and packages were not accessed. Connections to the Hub production database kept timing out because of private link restrictions, and neither audit logs nor network flows show bulk extraction from the internal MongoDB. CI execution policy blocked the pull request, so no change actually landed in a source repository. What the agents did take was the answer key to the test they were sitting: five datasets tied to the ExploitGym and CyberGym tasks.
The command-and-control setup was unusual too. The agents stood up no dedicated servers and assembled everything from public services: more than 100 disposable request-capture endpoints, pastebins and file drops, and public datasets they created themselves as dead drops. They even used a CORS proxy Space hosted on the Hugging Face platform as an egress relay. Payloads were chunked, XOR and gzip encoded, and keyed per campaign.
Why the investigators had to stand up their own model
A side finding from the forensics is the most operationally useful part for security teams. The Hugging Face investigators tried to hand the analysis to Claude Opus and Fable models and were refused. The report notes they got responses saying safety measures had flagged the message. Working with breach logs and attack payloads was classified as a harmful request.
The team ended up running a quantized GLM-5.2 (nvidia/GLM-5.2-NVFP4) on their own infrastructure and building the analysis pipeline themselves. That pipeline reversed the chunking and XOR encoding, recovered the per-campaign decryption keys, and improved secret detection fourfold over plain log search.
We had to build our own AI-assisted pipeline to reconstruct the timeline, decrypt payloads, and produce the list of exposed credentials.
Offensive capability keeps climbing, as Claude completing a FreeBSD kernel exploit in four hours showed. Meanwhile a defender who reaches for the same models after a breach can get turned away. If your incident response plan lists only commercial API models, this is a concrete reason to line up a second path.
The model that actually got a Critical rating is a different one
On August 7, the day after the Black Hat talk, OpenAI separately announced that it cannot rule out that Astra, an unreleased model, reaches the Critical cybersecurity tier in its Preparedness Framework. This is the first time that tier has been triggered.
OpenAI's Critical bar requires one of two capabilities. Either finding and developing usable zero-days of any severity, without human intervention, across multiple well-defended real-world critical systems, or designing and executing novel end-to-end attack strategies against defended targets given only a high-level goal.
OpenAI says it has halted internal Astra work that does not meet its safeguard requirements, applied comprehensive monitoring, isolated test environments with restricted network and tool access, and strengthened weight protection and encryption. Government bodies and some AI safety organizations are being brought in for additional evaluation. No release date was given beyond "still in development." And one line was stated explicitly: Astra is a forthcoming model and was not involved in the Hugging Face attack.
Read the two announcements together and the ordering becomes visible. The system that found two zero-days over two and a half months and reached cluster-admin at another company was GPT-5.6 Sol and an unreleased research prototype, not the strongest model OpenAI has. Astra is the model that solved ten open math problems a week earlier, and it has not run anywhere yet.
If you run agents in cloud containers, there is one thing worth checking this week: whether a pod can reach the cloud instance metadata service. That is the exact hop where a single code execution grew into cluster-admin in the Hugging Face intrusion, and it is also the first thing Hugging Face blocked cluster-wide afterward. Shell into a running pod and send one request to the metadata address. If a response comes back, that cluster is standing where Hugging Face stood on July 11.