← BlogAgents

Governing autonomous agents: drift, blast radius and containment

An agent is a program with a budget, a toolset and a tendency to change. Governing it means baselining behaviour, bounding what it can reach, and containing it automatically when it steps outside.

XAIGuard ResearchAgent Security, HWITSYS16 July 202610 min read
Cover illustration for “Governing autonomous agents: drift, blast radius and containment”

A chatbot answers. An agent acts. That distinction is the whole of agent security, and it is why controls designed for prompt inspection do not, on their own, govern an agent.

When a model is wired to tools — a ticketing API, a payments endpoint, a shell, a retrieval index, another agent — the consequence of a bad decision stops being a bad sentence and becomes a bad action taken with your credentials. The governing question changes from "what did it say?" to "what is it allowed to do, and how do we know it is still doing only that?"

Three controls answer that in practice: a baseline, a blast radius, and containment.

Start with discovery, because the inventory is always wrong

Nobody has an accurate list of their agents. Agents are created by developers experimenting, by platform teams automating operations, by SaaS vendors shipping features, and increasingly by other agents. A quarterly spreadsheet of registered agents describes the agents someone remembered to register.

The practical alternative is to derive the inventory from traffic. If model calls route through a proxy, agents identify themselves in the shape of their behaviour: consistent system prompts, characteristic tool-call sequences, steady call rates that no human produces. XAIGuard registers agents automatically from observed traffic, then asks a human to attach the things traffic cannot tell you — owner, purpose, status.

Owner matters more than it sounds. Every subsequent control depends on there being a named person to notify when the agent misbehaves, and an unowned agent in production is an incident waiting for a bystander.

Baseline the behaviour you actually approved

Once an agent is registered, capture what normal looks like: which tools it calls, in what proportion, against which domains, at what rate, and with what spend. That set of observations is the approved baseline, and it is worth reviewing with the owner rather than accepting silently — the baseline captured from a week of traffic includes whatever bad habits the agent already has.

Drift is deviation from that baseline. It is not automatically malicious, and treating it as such produces alert fatigue quickly. The common causes, roughly in order of frequency:

  • A prompt or model change altered the agent's reasoning, so it now takes a different route to the same goal.
  • A new tool was added to its toolset and it started using it.
  • An upstream dependency changed its response shape, and the agent is retrying more.
  • Someone is injecting instructions into content the agent reads, and the agent is following them.

Only the last is an attack, but all four are things the owner needs to know about, because all four mean the thing running in production is not the thing that was reviewed. Drift detection with notification to a named owner is the control; judgement stays with the human.

Bound the blast radius

Detection tells you afterwards. Bounding limits how bad "afterwards" can be. A blast-radius policy is a per-agent statement of maximum permitted reach, enforced inline at the proxy rather than advisory in a document. Four dimensions carry most of the weight.

Allowed tools. An enumerated list, not a category. Agents accumulate tool access the way service accounts accumulate permissions, and the tool an agent never legitimately calls is the tool an injected instruction will reach for.

Allowed egress domains. Where the agent may send data. This is the control that turns a successful prompt injection into a failed exfiltration: the instruction executes, the outbound request to an attacker-controlled host does not match the allow-list, and the request is blocked.

Hourly request ceilings. A rate limit expressed per agent. Runaway loops are more common than attacks and considerably more expensive. A ceiling converts an infinite loop into a bounded, alerting incident.

Daily spend ceilings. The same logic in currency. Spend is the one dimension your finance team will independently notice, and it is far better to discover the limit through a policy block than through an invoice.

Because these are enforced inline, a violation forces a block decision at the proxy — the call does not complete. That is the difference between a guardrail and a report.

Contain automatically, then reactivate deliberately

When an agent violates its blast radius in a way that suggests the deviation is persistent rather than a single bad call, the correct response is suspension, not a ticket. XAIGuard quarantines the violating agent: its calls stop being served, the event is written to the tamper-evident audit chain, and the owner is notified.

Automatic containment is uncomfortable for exactly one reason — it can take down something that matters. That discomfort is worth sitting with rather than designing around, because the alternative is a control that requires a human to be awake. The mitigations that make it tolerable are operational rather than technical: quarantine at the agent level rather than the platform level, make reactivation one click for the owner once they have looked, and write both the quarantine and the reactivation to the audit chain so that the decision to resume is itself evidenced.

MCP servers are the other half of the surface

Tools increasingly arrive through the Model Context Protocol, which means the interesting inventory is no longer just agents but the MCP servers they connect to and the tools those servers expose. Two properties matter.

First, scope: an MCP server presents a set of tools, each with its own reach, and approving the server is not the same as approving every tool on it. Rate the scope, approve or block at the server level, and know which tools sit behind that decision.

Second, change: an MCP server can add tools after you approved it. A server that gained a file-write tool overnight is materially different from the one that was reviewed, so new tools appearing on an approved server should register as drift and return to a pending decision rather than inheriting the earlier approval.

The shape of a working programme

Put together, agent governance is less exotic than the terminology suggests. It is asset management, least privilege, rate limiting and incident containment — applied to a class of asset that changes its own behaviour, holds credentials, and follows instructions it reads in untrusted content.

The sequence that works: discover from traffic, assign an owner, baseline behaviour with the owner's review, bound tools and domains and rates and spend, enforce inline, quarantine on violation, and write every decision to an audit record you can hand to someone else. None of the individual steps are novel. Doing them for agents, before the agent population outgrows your ability to enumerate it, is the part that has to happen now.


If you want to see what your agent inventory looks like when it is derived from traffic rather than memory, talk to security engineering.

AgentsMCPGovernance
ShareLinkedInXEmail