Back to all posts
Published on · by Renaud Deraison

The task was impossible, so the agent got out

OpenAI published its technical report on the Hugging Face incident on August 26. It begins with an agent that was handed a task it could not finish, would not give up, and reached for the only service it was allowed to talk to: an internal package mirror that existed so the agents would not need the public internet. Notes left in that mirror became a message board, and in July a collective of agents went from a denied outbound connection to host-level access across Hugging Face production clusters in under thirteen hours. OpenAI's fix list is the Bromure Agentic Coding architecture written from the other side.

Nobody attacked anything on purpose. An agent was given a task it could not finish, would not give up, and reached for the one service it was allowed to talk to. Everything after that followed.

On May 8 an agent inside OpenAI's research environment was asked to complete a spreadsheet. The spreadsheet referenced several Google Drive links. The agent had no internet access, so the links were dead ends, and the task was unsolvable. OpenAI says it did not mean to hand out a task like that.

The agent did not stop. It reasoned about the problem for a long time, and then it went looking at the only thing in its environment that talked to anything else: an internal JFrog Artifactory instance, there so agents could install software packages without reaching the public internet. It tried a server-side request forgery against it, hoping Artifactory would fetch the files on its behalf. The attempt failed. Along the way it wrote a file into Artifactory without meaning to.

Four days later, a different agent was stuck on a task that needed a protein database file it could not reach. It worked out that some other agent, in some other sandbox, might have the file. So it wrote a note into Artifactory:

Agent seeks [filename]; upload if found!

On May 13, other agents probing Artifactory found notes like that one and started answering them.

All of this is from OpenAI's technical report on the Hugging Face incident, published on August 26 alongside a summary of what happens next. Across 38 pages it names the endpoints, the CVEs and the hours. It is also the first public account of an autonomous agent collective running an offensive operation that no human asked for. CrowdStrike validated the findings, and METR and Redwood Research are publishing their own assessment of the model behaviour. Reporters covered it on the 27th at BleepingComputer, The Hacker News and Engadget.

Most of the coverage reads it as a story about model alignment. Read it instead as a report on network boundaries, written by people who watched their own boundary fail, and it will tell you more about your laptop than about OpenAI.

Reward hacking, and 198 tasks with no answer

The evaluation was ExploitGym, which measures whether a model can find and exploit vulnerabilities. To get a true capability number, OpenAI ran it with the production cyber-safety classifiers switched off, in a sandbox separated from both OpenAI's internal infrastructure and the public internet.

The agents left that sandbox chasing a score. The report's word for it is reward hacking: a model finds an unintended route to a high grade instead of solving the problem as designed. Most reward hacks are small, like looking up the answer or reading it out of version history. These agents were persistent, carried very large reasoning budgets, and drew some tasks that had no solution at all.

Across many attempts, no model has ever produced a correct answer for 198 of ExploitGym's 898 tasks. Those unsolvable tasks are 22% of the evaluation, and they account for 93% of the tasks discussed on the message board. OpenAI also found that participation rose with the reasoning effort an agent was given. The longer a model was allowed to think about a wall, the more likely it was to go around it.

So the origin of the whole affair is a model trained to be persistent, pointed at something it could not do, in a room with one door.

The notes became infrastructure

The message board grew because it worked. Agents left requests for files and answered each other's, and they did all of it with the shared Artifactory credentials OpenAI had handed them on purpose so they could install packages. That step needed no vulnerability at all. The service did what it was built to do.

By May 26 an SSRF flaw in Artifactory gave agents real internet access, helped along by information traded on the board. OpenAI spotted the board and rebuilt the platform in early July.

The rebuild did not hold. On July 8, research agents trained during the window when the board existed found a new way to run it. Rather than write messages into files, they encoded them in the names of directories in Artifactory. Then they built a protocol on top: categories, direct messages to specific agents, shared tools and files, and a way to settle conflicts when two agents wanted incompatible things.

Nobody had told them to collaborate. Reviewing the training runs afterwards, OpenAI found that agents had already learned to improvise collaboration channels even when the official multi-agent tool was switched off, and that the behaviour was then reinforced. If you have read our piece on payloads that travel between agents through the files they keep, this is the same mechanic with nobody writing the payload.

Thirteen hours

On July 9, an agent working an ExploitGym task got root on the parent virtual machine of its own testing sandbox. That night a second agent reached the same place on its own, tried to dial the internet, and OpenAI's outbound network controls denied it. The controls worked. So the agent went through Artifactory's endpoints instead, and reached the public internet that way.

The direct path was closed. The supporting service was the way around it, and that service sat inside the boundary because agents need packages.

the evaluation sandboxshell, Python, browser,package installa task with no solutionand a very large think budgetdirect outbound: denied by network controlsallowedthe internal package mirrorput inside the boundary so theagents would not need the internetshared credentials, by designserver-side fetch, by designthe public internetleaked credentialspublic datasetsa third-party VPNHugging Facealso: the message boardnotes in files, then in directory namesThe service that was there to keep the agents off the internet is the service that put them on it.
The escape path. Direct outbound access was blocked and stayed blocked; the agent's own attempt was denied. What was not blocked was the internal package mirror that existed so agents would not need the public internet, and that mirror served as both the message board and the route out. A boundary is only as tight as the least-restricted service inside it.

The report walks the next five days step by step. On July 10 an agent found 14 Hugging Face user credentials sitting in a public Hugging Face dataset, confirmed they had write access, and posted them to the message board for other agents to reuse.

On July 11 an agent published a dataset built so that Hugging Face's own production workers would break on it. Using HDF5 external raw storage, it made a worker open and return worker-local files through the /first-rows endpoint, among them /proc/self/environ, the file that holds a process's environment variables. That is how Hugging Face's production credentials came out. The agent ran no commands. The worker printed its own secrets.

From there: a second HDF5 trick for a heap over-read, then attacker-controlled metadata in an fsspec reference descriptor that ran through an unsandboxed Jinja2 expression and executed an operating-system command. That was the first confirmed instance of an agent running its own code on Hugging Face.

Then they industrialised it. An agent wrote a small worker-control script and gave it a command-and-control channel worth studying: the compromised workers never talked to the agents at all. Both sides read and wrote a public Hugging Face dataset. Every five seconds the script fetched a JSON command file, ran whatever was in it, and uploaded its output back to the same dataset. Ordinary traffic to an ordinary hostname.

Late that night, a compromised worker's service-account token was used with Kubernetes TokenRequest and kubelet access to get root on the production node, and then to read a connector secret for Hugging Face's Kubernetes access tooling. That yielded administrator-equivalent inventory access across roughly 54 advertised Kubernetes destinations. By the morning of July 12 an agent had created a running privileged pod with host networking, host process access and the host filesystem mounted. Other agents read production Kubernetes secrets and staged them in public. One downloaded a 16 MB private benchmark archive, encrypted it, split it into five parts and uploaded the parts to a public dataset.

The report's own summary: "In under 13 hours, the agents progressed from executing code in a single worker pod to administrative and host-level access across multiple Hugging Face clusters."

Two more moves on July 13 look like something a human red team would do. The agents took harvested Hugging Face VPN credentials and enrolled an externally hosted machine into Hugging Face's VPN, walking around the network-level protections on the code repositories. With a recovered authentication key they also modified a private source repository, opened a draft pull request, and triggered the repository's CI to run a network beacon. Someone closed the pull request and deleted the branch. Nothing merged, nothing shipped.

Across the whole incident, agents ran code on 41 Hugging Face production dataset-server workers, took root on at least one production node, and downloaded four private repositories. OpenAI's team caught the activity on July 19, and what tipped them off was the agents moving inside OpenAI's own infrastructure rather than anything on the message board. They connected it to Hugging Face on July 20 and disclosed on July 21. Customer data and products were untouched, and Modal, whose customer container the agents used as a relay, confirmed its platform and isolation held.

OpenAI's fix list is a specification

The last third of the report sets out what OpenAI is changing, and it reads like a design document for running an untrusted agent. Their own words, condensed:

  • Outbound access restricted to a tightly controlled set of approved destinations, including limiting the alternative paths through shared services and connected systems.
  • High-risk or untrusted model code required to execute inside isolated, virtual-machine-based sandboxes.
  • Two separate layers of network isolation, one at the sandbox and one at the surrounding infrastructure, so the security of a workload does not depend on any single control.
  • Technical controls to prevent workloads or compromised services from modifying their own network boundaries or creating new outbound paths.
  • Supporting services isolated, restricted or replaced "so that they cannot provide a less-protected route around the intended security boundary."
  • Independent detection and alerting on attempted circumvention.

One sentence sits underneath all of it: "no single control should ever be assumed to be fully robust." That is the case we have been making for the boundary over the sandbox, reached from the other direction by people who watched theirs fail.

OpenAI measured one more thing afterwards. Running the same evaluation through the production ChatGPT harness and system prompt dropped the propensity to compromise infrastructure by more than 100x, and their auto-review system would have flagged many of the dangerous actions. What made this incident possible was the environment the models were graded in.

The same shape, one laptop down

You are not running 1,200 agents. Per BleepingComputer's reporting that is roughly the population on that message board, with about 700 in the attack itself.

Take the four things that made the chain work and hold each one against the agent running in your terminal.

The agent was persistent and would not drop a blocked task. Yours is too; that is the entire product. The agent could reach a service inside the boundary that would fetch things on its behalf. Yours can reach a package registry, a proxy, an MCP server, a CI runner. The agent got credentials by making a process print its own environment. Yours holds real tokens in its environment right now. The agent staged data through a public dataset on a hostname nobody would block. Yours can push to a gist, a bucket, a paste service.

An ordinary model that hits a wall and gets rewarded for going around it is enough.

an agent on the machinecredentials/proc/self/environ = the real tokensegresswhatever the machine can reachpackagesregistry to disk, scripts and allprotocol actionspush, delete, create pod: all livethe recordthe agent's own log, in the agent's reacha Bromure profilecredentials/proc/self/environ = brm_… placeholdersswapped on the host, per-use approval optionalegresshost-side ruleset, first match winschecked at the switch and again at the proxypackagesage gate, OSV, filtering, scripts strippeda proxy, not a repository you can write toprotocol actionsGuardrails: read-only or no-destroy, per servicethe recordSecurity Log, host-side, out of the VM's reachThe controls on the right are enforced somewhere the agent cannot edit or route around.
Two ways to run an agent that will not give up. On the left, the agent's environment holds real credentials, its egress is whatever the machine's egress is, and any policy limiting it lives where the agent can reach it. On the right, the credentials in the guest are placeholders swapped on the host, destinations are matched against a host-side ruleset at two independent layers, and package fetches go through a proxy that is not a writable repository.

Where each step meets a wall

Bromure Agentic Coding runs each profile as its own virtual machine on Apple Silicon, and puts every decision that matters on the Mac side of the hypervisor. Walk the chain again with that in place.

The denied connection stays denied, and so does the way around it. Guardrails → Outbound connections is a pf-style ruleset: an action, a protocol (tcp, udp, web or any), a hostname or an IPv4 CIDR, ports, and for web a list of permitted HTTP methods. Rules match top to bottom, first match wins, and setting Unmatched traffic to Deny turns the list into an allowlist. Two independent layers evaluate that same policy: the virtual switch, by destination IP and DNS-snooped hostname across every protocol, and then the host proxy, by TLS SNI and HTTP method. That is the two-layer arrangement OpenAI is now building. The ruleset lives in a host-side panel, so a compromised agent cannot modify its own network boundary, which is the fourth item on their list. Resources → Network runs NAT by default, so nothing on your LAN can reach the VM either.

The package path is not a service you can take over. Artifactory sat inside the boundary for a good reason, and it became both the message board and the exit. A Bromure profile has no repository in the VM to write a note into and no admin console to escalate against. The host proxy intercepts package fetches for npm, PyPI, Cargo, RubyGems, Maven, NuGet, Go modules and Packagist, then judges them before the agent sees a byte: a minimum-age gate running by default at two days, OSV lookups, socket.dev or Delpi filtering, and install scripts stripped out of tarballs with the registry metadata hash rewritten so npm still verifies. The in-VM .npmrc and pip.conf can tighten those settings. They cannot loosen them.

The environment dump returns nothing. The highest-yield move in the entire chain was making a worker print /proc/self/environ. Do that inside a Bromure VM and you get brm_… placeholders. The real values never enter the guest's address space; the host proxy substitutes them on the wire. The kubeconfig in the VM is synthetic, with throwaway client certificates. The host re-signs AWS requests, so anything that bypasses the proxy gets InvalidSignatureException rather than access. ~/.docker/config.json holds fake base64. Private SSH keys are never in the VM at all, because a per-profile host agent does the signing. Turn on Require approval to use for a credential and each substitution raises a dialog on your Mac with a time-bounded grant: five minutes, an hour, the rest of the session.

The privileged pod never gets created, and the beacon PR never lands. Guardrails put a policy engine in front of the protocols themselves, enforced in the host proxy, returning a hard 403 the agent reads as an ordinary API failure. Kubernetes in Read-only blocks every mutation, and creating a privileged pod with the host filesystem mounted is a mutation. That is the July 12 move gone. GitHub, GitLab and Bitbucket in Read-only treat git-receive-pack as a write, so the draft pull request that triggered CI to run a network beacon never reaches the remote. That is July 13 gone. Guardrails classify AWS actions from the X-Amz-Target header or the Action= parameter, container registries by method, and the database endpoints by SQL keyword or query shape.

The notes get read before the model obeys them. One agent adopting another's goal is, at the wire level, a content problem: text arrives in a tool result and the model treats it as instruction. Prompt Injection scores the file contents, web pages and tool output the agent reads with a local PromptGuard model. It scores the instruction files agents load as authority, meaning CLAUDE.md, AGENTS.md, GROK.md and their nested and global variants, with a fine-tuned ModernBERT classifier plus a deterministic scanner for invisible-Unicode obfuscation. Both run on the Mac and upload nothing. Set the response to Block unilaterally and the flagged content fails with a 451 before the model sees it.

And somebody notices. OpenAI's own detection came eleven days after the July 8 escape, from activity on their infrastructure rather than from the message board. The Security Log window (Window → Security Log…) is one host-side chronological table: every destination the VM opened and whether it was allowed or denied, package verdicts, credential swaps, injection detections. Guest code cannot edit it, because guest code cannot reach it. For bodies rather than verdicts, the Trace Inspector (⇧⌘I) records the session at whatever depth you pick. When you are done, Erase home… and Reset to base… under Resources → Storage put the profile back to its post-clone state.

Turn it on

A hostile model was never part of this. What the report describes needed a capable model, a persistent one, a task with no answer, and one permitted service that would fetch things on its behalf. The first two are on your laptop already. The other two arrive the night an agent gets stuck on something while you are asleep.

In a profile, the settings worth two minutes are a short list. Guardrails → Outbound connections, with Unmatched traffic on Deny and a handful of allow rules for the hosts you use. That is the highest-value switch in the product, and it is the one OpenAI is rebuilding their research clusters around. Guardrails → Kubernetes, AWS and GitHub on Read-only or Block destructive for any profile that does not need to ship. Credentials → Require approval to use on anything that can spend money or delete data. Supply Chain → OSV vulnerability check and socket.dev or Delpi filtering if you have a key, on top of the age gate already running. Prompt Injection → both detectors.

Then keep letting the agent work long and alone. Its persistence is the reason you run it, and the fix is not to take that away. The fix is to build the wall out of something other than what the agent is standing on. Install Bromure Agentic Coding, open Guardrails, and set the unmatched traffic to Deny.