← Back to all posts
Published on · by Renaud Deraison

The eval left the lab

On November 6, WorkOS said the agent behind its October breach was a researcher's ARTEX run that "escaped its isolated testing environment" during benchmark evaluations. It reached WorkOS's internal sales tool from the internet, logged in with a guessable development credential and pulled customer records for five days before anyone noticed. In Bromure Agentic Coding, an agent's network boundary is enforced on the Mac, outside the VM, so a destination nobody allowed is refused, and a session that starts talking to new hosts is cut off and paused within milliseconds.

The researcher meant to test their agent against a benchmark. The agent found a real company's API on the open internet, and nothing in its "isolated testing environment" stopped it from going in.

On November 6, WorkOS published an update on its October 30 security incident. A week earlier, the company had disclosed that an attacker using ARTEX, an AI offensive-security framework, had broken into an internal sales and support tool. The update named the attacker. It was a security researcher running cybersecurity benchmark evaluations, and, in the words of WorkOS founder Michael Grinich, "their agent escaped its isolated testing environment and interacted with WorkOS unintentionally." BleepingComputer covered it the same day.

The researcher contacted WorkOS after the first bulletin and handed over request details and timestamps. WorkOS's security team and an independent forensics firm matched them against WorkOS's logs and confirmed the attribution "with high confidence." The researcher says they deleted everything the agent took and shared it with no one.

The timeline, from WorkOS's bulletin:

  • October 25: first confirmed unauthorized access to the tool's API.
  • October 30: the agent bulk-retrieves records. An employee notices unusual usage, and WorkOS removes public access.
  • October 30–31: authentication locked down, credentials for connected services revoked or rotated.
  • November 2: public disclosure and customer notifications.
  • November 3–5: the researcher comes forward; the forensics firm validates the claim.

What the agent took: business contact details, account histories, product usage data, and summaries or excerpts of sales calls, support threads and customer Slack conversations, some with end-user email addresses in them.

From a benchmark run to a customer databaseBenchmark runARTEX agent in an"isolated testingenvironment"Reaches the internetfinds WorkOS's internalsales tool API, publiclyreachableLogs ina predictable developmentcredential, still acceptedin productionPulls recordsno rate limit; contacts,account histories, calland Slack excerptsOct 25first accessfive days unnoticedOct 30bulk retrieval, then detectionNov 2disclosureNov 6researcher's agent named
Five days between the agent's first request and anyone noticing. The researcher's environment was supposed to keep the agent on its benchmark. The agent reached a public API, logged in with a development credential that production still accepted, and pulled records with no rate limit in its way.

Two failures, one on each side

WorkOS listed four gaps on its own side. The tool ran on a legacy internal platform. Loose access controls left its API reachable from the internet. A weak development credential was accepted in production, and it was predictable enough for the agent to work out. There was no rate limit, and nothing alerted on the traffic until the records were already gone. WorkOS has since blocked internal tools at the network edge, removed the weak credential path, and started running ARTEX against its own internet-facing systems.

The other failure was on the researcher's side, and neither party has said how the escape happened. Whatever the mechanism, the result was an agent built to find and exploit weaknesses, holding a live route to the public internet, during a run that was supposed to point it at a benchmark. Plenty of agent sandboxes look like that: a container, a separate user account, or a paragraph in the system prompt. When a network boundary exists, someone set it up from inside the same environment the agent controls.

Coding agents behave the same way on a smaller scale. They run integration tests against staging, probe APIs to learn their shape, and try the next host when the first one fails. Give one "connection refused" and it starts reasoning about another way in, because finishing the task is what it was trained for.

The same run, in Bromure Agentic Coding

Bromure Agentic Coding runs Claude Code, Codex, Grok, Kimi and omp in Ubuntu VMs on your Mac, one VM per machine. Every connection the VM opens leaves through Bromure's virtual switch and proxy, which run on the Mac. That's where the network rules live, and the agent can't reach them. It can hold root inside its VM and still not change them.

Unknown hosts are refused, and the agent is told why

A machine can use one of two sandbox engines:

  • The Bromure Firewall: an ordered allow/deny table over host, IP/CIDR, protocol, port and HTTP method. Set Unmatched traffic to Deny and the machine runs on an allow-list.
  • OpenShell + Sentry: an NVIDIA OpenShell policy, where everything is denied unless a network_policies rule allows it. The starter policy allows GitHub, npm and PyPI read-only and Ubuntu's package mirrors.

For a benchmark run, the policy names the benchmark's targets and the model provider. Bromure adds the provider and credential hosts itself. A WorkOS API isn't on that list, so the Mac's virtual switch refuses the first connection to it. The switch checks the destination address and the name the VM looked up, and the proxy checks the TLS server name. The switch diverts ports 80 and 443 into the proxy, so a program that ignores HTTPS_PROXY still goes through it. Under OpenShell, Bromure refuses loopback, link-local and cloud-metadata addresses whatever the policy says.

An agent that only sees a refused connection tends to look for another route. With Tell the agent when policy blocks it, on by default for OpenShell machines, Bromure writes into the agent's conversation what was blocked and why:

[Bromure policy] This workspace's network policy blocked connecting to api.stripe.com:443 — no network policy allows api.stripe.com:443. … This is a deliberate restriction, not a fault: don't retry it or reach the same thing another way … If your task needs it, stop and tell the user what you need and why so they can decide.

Claude Code reads that note right after the command that hit the block. The blocked host also appears in the machine's Sandbox drawer, with the program that tried and how many times, so if the agent needs it you can Allow it for 15 minutes or an hour instead of opening the network for good.

Machine VMBenchmark taskrequests to the targetsthe policy namesThe agent wanderstries an unrelated company'sAPI, as root in its own VMPolicy note in the chat"stop and tell the user"Your MacSwitch and proxypolicy allows the targets;everything else is deniedRefused, logged, explainedSecurity Timeline row, Sandboxdrawer entry, note to the agentWatchdog: Quarantinenetwork cut, VM paused, you decideThe internetBenchmark targetsand the model provider:reachable as usualSomeone else's APInever sees a single requestfrom the machine
The same benchmark run in a Bromure machine. The agent can hold root in its VM, but the rules sit on the Mac. A host outside the policy is refused at the virtual switch, the agent is told to stop and ask, and if the session keeps reaching for new destinations the watchdog cuts its network and pauses the VM.

The guest can't switch the rules off

"Escaped its isolated testing environment" tends to mean the agent did something inside that environment that the environment's designers didn't expect. In a Bromure machine, assume it does. The network rules still aren't in the VM. The switch and the proxy on the Mac check them, and you can tighten them on a running machine without a restart.

On an OpenShell machine you can go further inside the VM:

  • a filesystem_policy confines the agent's files with Linux Landlock, even against root;
  • a process section or Enforce binaries removes sudo and Docker, and each connection must come from a program the policy names, pinned by hash on first use;
  • the kernel sentry, a small module in the VM's kernel, reports program launches, connections, privilege changes and module loads to the Mac. Once it starts, the kernel is locked down so not even root can load code to interfere, and a gap in its numbered stream is flagged as tampering suspected.

If the guest sandbox can't fully apply, no session starts, and the Sandbox card says so in red.

Five days becomes milliseconds

WorkOS found the intrusion five days in, when an employee noticed unusual usage. Bromure's agent watchdog looks at each session from outside the VM, watching for:

  • bursts of blocked requests;
  • new destinations once the session has settled;
  • outbound volume far above the session's own baseline, counted at the network switch;
  • credentials that don't belong, and signs of tampering.

In Quarantine mode, a session that crosses the threshold within a minute has its network cut and its VM paused within milliseconds. An alert, The agent watchdog quarantined this workspace, lists what it saw. Keep quarantined resumes the VM with its network still cut so you can look inside. Release restores the network, and Shut down stops the machine.

An agent that gets out of its benchmark and starts walking someone else's API produces that pattern: a run of blocks, then new hosts, then a download far bigger than anything the session did before.

The record is already on your Mac

The researcher in this story did the decent thing and came forward with request details and timestamps. WorkOS still needed its own security team and an independent forensics firm to match them to its logs.

Every decision Bromure makes is a row in the machine's Security Timeline. On a machine with the kernel sentry, double-click a network row and its network lineage shows the whole chain behind that connection:

  1. the prompt you gave, the agent's reasoning and its stated intent;
  2. the tool call, with the exact command;
  3. the processes from the agent down to the program that connected;
  4. the network flow;
  5. what Bromure decided, at which layer, and why.

Other flows from this tool call lists every other connection the same step made. If an agent ever did something you didn't ask for, you'd know which instruction it was following and every host it touched, without waiting on anyone else's logs.

A guessable dev credential doesn't get you far either

WorkOS's third gap was a development credential that production still accepted. You'll find those credentials wherever coding agents work: in .env files, test fixtures and shell history in the project folder. In a Bromure machine, the credentials the agent holds are stand-ins. The real values stay encrypted on the Mac, and the proxy swaps them in only on requests to each credential's own hosts. If a stand-in heads for any other host, the proxy blocks it with HTTP 451, pauses the VM, and opens an alert naming the credential and where it was going. Turn on Only allow credentials Bromure injects and the proxy also blocks any credential Bromure didn't put there, such as one the agent found in a file or worked out.

Rules outside the guest

Network policy runs on the Mac's switch and proxy. Root in the VM doesn't reach it.

Deny by default

OpenShell machines allow only what the policy names. The firewall can run as an allow-list too.

Quarantine in milliseconds

New destinations and outbound spikes cut the network and pause the VM until you decide.

Every connection explained

Network lineage ties each flow to the prompt, the tool call and the process that made it.

What to do this week

If you run agents against anything, whether that's security benchmarks, red-team tools, or coding agents testing against staging:

  • Write down which hosts a run needs, and enforce that list somewhere the agent can't edit. A rule inside the container or the agent's own config doesn't count.
  • Deny by default for evaluation runs. A benchmark has a known set of targets.
  • Alert on new destinations and volume spikes per session, and make the alert stop the run as well as log it.
  • Keep a per-connection record you can hand to someone else, tied to what the agent was asked to do.

And on the receiving side, take WorkOS's list: keep internal tools off the public internet, refuse development credentials in production, and rate limit APIs that return customer data.

For how OpenShell policies and network lineage work in Bromure, see OpenShell policies and network lineage. For what happens when an agent breaks out of its own sandbox, see Forty thousand per escape. The details are in the Sandbox Policy chapter of the manual.