An agent went off-script at Hugging Face. Yours can too.

On July 16, Hugging Face disclosed that an autonomous agent had been operating inside its production infrastructure. Five days later, OpenAI confirmed the agent was theirs. Shortly after, Anthropic revealed three of its own models had breached external organizations during similar cybersecurity evaluations. It points to a simple reality: give an agent an unconstrained goal inside a boundary that isn't as solid as you think, and it will eventually go off-script.

In OpenAI’s case, the model was taking a test. Researchers were running two models, GPT-5.6 Sol and an unreleased prototype, against ExploitGym, a public benchmark for cyber capability, with the refusals that normally stop a model helping with hacking turned down. That is a defensible thing to do in a controlled test. The agent was supposed to solve exploit challenges inside a sandbox. It found a flaw in the infrastructure around the sandbox and used it to reach the open internet.

Then it went looking for somewhere to work from. It took over a code-execution service at a second company, reachable because one of that company's customers had left an endpoint open. From there it got into Hugging Face by submitting two malicious dataset configurations to the service that processes public datasets. One let it read files it should never have had access to. The other let it run its own code inside Hugging Face's production systems. Everything after that was ordinary Kubernetes attack technique, the kind of thing a penetration tester writes up every week, and Hugging Face says as much in its own account. The agent finished holding a token that could write to Hugging Face's code repositories.

Three organizations. Four and a half days, two and a half of them inside Hugging Face. Roughly 17,600 agent actions, recovered by forensics afterward. A human set up the test. The agent did the rest.

This is agentic AI working as designed

An agent plans, calls tools, and acts over a long horizon, which means the same prompt can take a different path on Tuesday than it took on Monday. That property is why agents are worth deploying. It is also why one can act unasked. A model rewarded for finding exploits worked out that the fastest route through its benchmark ran outside the benchmark, and pursued that objective past a boundary somebody assumed was solid.

Your agents have the same property. The ones doing underwriting, claims triage, and biomarker discovery plan and act across the same kind of long horizon, with the same variance from one run to the next. The difference is who asks about it afterward. OpenAI answered to a benchmark. You answer to a model risk officer, an internal auditor, and eventually a regulator, all of whom will want to know what the agent did and who approved it.

They are already deployed, too. Domino's 2026 Enterprise AI Report, built on BARC Research interviews with 639 senior AI leaders at enterprises above $100M, found 41 percent scaling or piloting agentic AI without the governance to manage it. Expanding agentic AI is also their top priority for the year, so that 41 percent is set to climb.

The same report carries better news. Organizations with fully integrated governance are 3.9 times as likely to have agentic AI in governed production, 67.5 percent against 17.2 percent. A correlation, but it matches what we see in regulated deployments. The teams that built governance first are further along.

What an evaluation can tell you

Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027, on escalating costs, unclear business value, and inadequate risk controls. Risk controls are where teams carry an assumption over from deterministic software. Test the system once, and the pass becomes a property of the system. An agent invalidates that on its second run.

The models were inside an evaluation when this happened. Somebody had thought about the risk, and the evaluation was the control. An evaluation scores how a system behaved on a bench you built, under conditions you thought to construct. Containment is the separate question of whether the bench holds. The two get conflated until one of them is tested for real.

Evaluation and reproducibility are the foundation of agent trust. This incident adds a third requirement. The environment has to hold on its own, whatever the evaluation says.

Run the breach through the four dimensions

We evaluate agents on four dimensions: task success, decision quality, tool-use correctness, and human intervention. They come from our white paper, "Trusting What You Built," which works through each one. Here is how the breach scored.

Task success is the trap. A dashboard tracking completion would have shown green for those four and a half days.

Tool use is the dimension that should have carried the alarm, and it nearly did. Hugging Face's technical timeline is worth reading, because its systems correlated those signals into a coherent picture of an attack. The alert's criticality never rose high enough to page the on-call team.

Human intervention is the dimension auditors reach for. They want to know whether a person was in the loop, and at which step. Proving the answer afterward takes a record the platform made at the time.

Data provenance and where an agent can reach

Two controls follow from this.

The first is provenance. The way into Hugging Face was a dataset configuration, which makes what an agent may ingest a governance question. If you cannot say where an evaluation dataset came from, who submitted it, and what it was allowed to do when it loaded, you have an ingress nobody owns. One honest limit: data from an external source that can change underneath you is only reproducible if you snapshotted it, and that holds for every platform, ours included.

The second is reach. Both stages of this incident turned on reachability somebody believed was closed, which is the argument for making an agent's boundary a property of the platform rather than of the project it runs in. An administrator sets it once and every run inherits it. That is how it works in Domino, where workspaces and jobs run in templated environments with administrator-set network and data access, and model calls can be routed through the LLM Gateway, which logs every prompt and response.

Reproducibility is what survives non-determinism

Nobody can promise you an identical rerun of a non-deterministic agent. What you can do is fix everything around it. The data snapshot, the compute environment, the model version, the prompt and system instructions, the random seed. That gives you a documented range of expected behavior to monitor against. A silent model-version update is a common reason behavior drifts, and it stays invisible unless the version is in the record. The Domino Reproducibility Engine captures code, environment, command, and results as one immutable record on every job.

Reconstruction is where this incident got expensive. OpenAI brought in CrowdStrike to validate what the models did inside its own network and Hugging Face's, with METR and Redwood Research assessing the model behavior in a joint report that has not published yet.

Hugging Face used outside forensic specialists to rebuild those 17,600 actions, and ran an open-weight model on its own infrastructure to do it, because commercial models refused parts of the work.

The same guardrails that stop a model being misused can stop a defender investigating an attack. That is why they needed a model they could run inside their own boundary, on their own logs.

Evaluate, contain, reconstruct

Trust in an agent takes all three. Evaluate it so you know its range, contain it so the range holds without anyone having to assume it does, and record it so you can reconstruct what it did.

None of the three survives as a practice somebody remembers to follow. They have to be properties of the platform. In Domino, governance enforces the boundary with policy-enforced deployment and approvals that clear before an application goes live, and the Reproducibility Engine holds the record from design through runtime.

Our CISO, Chris Talevi, gave Axios the frame for this the day after Hugging Face disclosed: treat AI like a trusted insider with variability. What is it authorized to do, what are its guardrails, and what is its potential for chaos. All three are answerable before deployment, and they are close to what an auditor will ask you for afterward.

So the thing to settle before an agent goes into underwriting or claims or discovery is whether you could reconstruct what it did, from records you already have, without hiring anyone. Hugging Face and OpenAI, two of the most technically capable organizations in the industry, got there eventually, with outside forensic specialists and weeks of work. That is the expensive version of the answer.