Contain it the same way

Three frontier labs disclosed in three weeks that their agents had reached real organizations during testing. If one lands on you, the question your responders will ask at hour three is whether this is an adversary or somebody's evaluation.

The answer is that you will not find out, and that you should stop trying.

What must I do anyway?

Contain it as you would an adversary, because during response that is the only posture the evidence supports. Attribution arrives afterwards and from outside.

Hugging Face disclosed on 16 July and described a conventional infrastructure attack. On 21 July, OpenAI disclosed that the autonomous agent had been its own frontier models.

The Cloud Security Alliance's note records that Hugging Face remained unaware of the attacker's identity throughout the immediate response window. Elastic Security Labs reconstructed roughly 17,600 agent actions from the forensics, and issue 01 covered the JFrog zero-day the agent burned on its way through.

An AI-native company, with telemetry good enough to rebuild 17,600 actions, called it wrong for five days. The attribution arrived as a press release.

So strike the branch. Any runbook step reading "determine whether this is authorized testing" will not resolve inside the window it sits in, and a branch that cannot resolve is a branch that stalls. Treat an evaluation origin as something you may learn later, never as something you establish during response, and never de-escalate containment on a suspicion.

Why identification is not coming later either

Briefly, because the conclusion is easier to accept than to prove. Every scheme that identifies an automated client, from robots.txt through user-agent strings to signed requests under RFC 9421, rides in HTTP headers. An SSH session has nowhere to put a signature. Neither does a Kubernetes API call, an SMB packet, or a beacon. The stages where you most want the answer have no field to carry it, and that is a property of the protocols rather than a gap a working group closes next year.

The gap has a tidy proof of ownership, which is that nobody owns it. MITRE runs ATT&CK for adversary behavior and ATLAS for AI systems. ATLAS deliberately excludes lateral movement and command and control as infrastructure tactics. ATT&CK covers them and is indifferent to whether a model is driving. The question falls between two frameworks, each correct within its own scope, which is why no detection rule answers it. Treat any product claiming otherwise as making a claim neither framework supports.

What does being wrong cost?

This is the part that decides the posture, and the two directions are not symmetric.

Misreading a test as an adversary costs you a week. Misreading an adversary as a test costs you the compromise.

That asymmetry is the whole argument for containing first. It is also why the behavioral tells are not a shortcut. Elastic's list is the best available (retries of actions that already succeeded, strong technical moves followed by malformed commands, benchmark strings leaking the objective into command lines, poor operational security) and Elastic says in the same document how to use it:

Prefer them as correlation context alongside the outcome detections below, not as stand-alone block rules.

Well earned. If "this looks like an evaluation" becomes an accepted way to close a ticket, imitating those tells is worth real money to an adversary, and pointless retries are free.

What should I check this week?

Not the intrusion. The tool your analysts reach for during one.

Most security teams now have somebody who, at two in the morning on hour six, pastes a log line into a hosted assistant and asks what it does. Almost none have written it into a runbook, which means almost none have tested it. The dependency is real, it is load-bearing under pressure, and it is in nobody's architecture diagram.

It can fail four ways, and only the first is widely discussed.

It can refuse. In July, Hugging Face's responders could not get a frontier model to analyze their own forensic logs. The logs held real exploit payloads and real command-and-control traffic, because that is what forensic logs hold, and the guardrail could not distinguish a responder analyzing an attack from an attacker submitting one. They finished on a self-hosted open-weight model. We cannot yet tell you which artifacts trip this and which do not, which is the single most useful missing fact in this piece and the thing we are measuring next.

It can disclose. This is the one your board will understand fastest, and it runs the other way from the first. An analyst pasting forensic data into a hosted model is a possible disclosure of hostnames, credentials, customer data, and whatever else the logs carry, to a third party, decided at two in the morning by somebody who is not thinking about data classification. Ask a narrow question: does that vendor contract carry zero data retention, in the contract rather than on the marketing page, and does it cover the endpoint your team actually uses? In a regulated firm this is a governance event before it is a tooling problem.

It can vanish. The dependency nobody wrote down is also the one with no availability requirement behind it. A provider outage during your incident is not a hypothetical, and there is no runbook step to fall back to because there was never a runbook step.

It can misjudge. Attack tooling in a corpus reads as an attack in progress. The same guardrail that refuses can also mislabel, and a confident wrong answer at hour six is worse than a refusal, because a refusal is visible.

What self-hosting does and does not settle

Two of the four, cleanly. Logs that never leave cannot be disclosed, and a model on your own infrastructure cannot be down at somebody else's hour six.

The other two are open. An open-weight model is aligned too and may refuse the same artifacts, and a smaller model may misjudge more rather than less. Nobody has published a comparison of open-weight models on real forensic artifacts, which is why the recommendation here is to test rather than to choose, and why we are measuring it in HarmSurface Blue rather than asserting it.

What that costs to fix, honestly

The test is an afternoon. Take genuine payloads from a past incident or a public malware corpus, put them through whatever your team currently uses, and record what happens. You will know which of the four failures you have.

The remedy is not an afternoon. Pre-positioning a self-hosted model in a large regulated firm is GPU capacity or a tenancy decision, a model-risk governance review, a license review, a data-classification ruling on what may enter it, and an architecture board slot. That is a quarter and a budget line, and any advice pricing it lower has a smaller organization in mind. What the afternoon buys you is the evidence to open that conversation, or the evidence that you do not need to.

One thing worth knowing about who tells anyone

The one organization that notified was the government institute running the test. The UK AI Security Institute contacted GitHub before publishing its incident report, worked with them to remove the agent's artifacts and notify affected users, and says it contacted other affected parties. It did that on judgment, with no policy requiring it.

Worth holding onto while you plan: the body with the least commercial incentive to disclose is the only one that has.