Beyond model guardrails, part 3: when prevention fails

Input trust, monitoring, recovery, multi-agent security and scaling controls with capability

Parts 1 and 2 of this blog series covered the first four agentic security control planes:

1. Governance and classification
2. Execution and network containment
3. Identity, credentials, tools and MCP
4. Authorization and action safety

Read on for the last control plane: input trust, visibility, resilience and recovery.

Review part 1 >>

Review part 2 >>

Execution isolation and deterministic authorization can substantially reduce the consequences of agent failure, but they cannot guarantee that the model will interpret every input correctly.

Many agentic attacks begin before the first privileged tool is called. The agent may retrieve an email, webpage, issue, document, ticket, repository file or tool response containing instructions written by someone other than the user or system owner. The model then interprets those instructions as part of the task. That makes input trust, monitoring, recovery and delegation control essential parts of the same enterprise control plane.

Input trust, visibility, resilience and recovery

Treat retrieved content as untrusted data.

Indirect prompt injection exploits a fundamental property of many LLM applications: instructions and data may enter the same model context. As described by OWASP's prompt injection guidance external content influences model behavior in indirect attacks. The impact of these attacks depends heavily on the tools, data, permissions and autonomy available to the application.

To mitigate the impact, agent systems should therefore preserve the source and trust level of retrieved content. For example, system policies, user requests, enterprise data, external content, tool output and peer-agent messages should not be collapsed into an undifferentiated context without provenance. A sentence in an email, issue, webpage or tool response should not become authorization to access another repository, expose a secret, send a message or execute a command.

This control is best described as input provenance and context separation.

It may include source labeling, trust metadata, separation between instructions and retrieved data, and explicit transitions between reading and acting.

Input, data and context controls

Input provenance

• Available implementation options: record source, owner, location and trust level of retrieved content
• When customers should use it: agents reading email, issues, tickets, web content or files
• Important limitation: a model may still follow untrusted instructions.

Context separation

• Available implementation options: distinguish system policy, user requests, retrieved data, tool output and peer-agent messages
• When customers should use it: retrieval-augmented or tool-using agents
• Important limitation: separation helps interpretation but is not authorization.

Instruction-to-action boundary

• Available implementation options: require policy evaluation or approval before retrieved content can trigger consequential actions
• When customers should use it: agents that both read and act
• Important limitation: prevents untrusted content from becoming direct authorization

Source ACL enforcement

• Available implementation options: apply the initiating user’s permissions at retrieval time
• When customers should use it: enterprise knowledge agents
• Important limitation: prevents unauthorized data from entering context

DLP and sensitivity controls

• Available implementation options: labels, content restrictions, output checks and protected-data policies
• When customers should use it: email, file, CRM and knowledge assistants
• Important limitation: must supplement, not replace, source permissions

External-content restrictions

• Available implementation options: limit or isolate external email, web grounding and attacker-controlled sources
• When customers should use it: agents exposed to external content
• Important limitation: reduces the prompt-injection entry surface

Microsoft Purview provides examples of these controls around Microsoft 365 Copilot. For instance, its DLP controls for Microsoft 365 Copilot can restrict how sensitive prompts, files and emails are processed under configured policies.

These mechanisms are useful, but the primary access boundary should remain the source system's authorization.

An agent should not retrieve a document, record or repository that its current user and task are not authorized to access.

Monitor the complete decision path.

Traditional monitoring focuses on binaries, processes, authentication events and network destinations.

Those signals remain useful, but they are insufficient when activity is performed by an approved agent through a valid identity and sanctioned API. Organizations need visibility into the complete decision and action path. This includes:

• The initiating user or workflow
• Agent identity
• Requested task
• Sources retrieved into context
• Tool selected
• Submitted parameters
• Authorization decision
• Credential used
• Destination system
• Approval event
• Resulting change

Logs must exist outside the agent's write path. An agent that can modify or erase the record of its own activity cannot be reliably investigated.

Visibility and detection controls

Independent audit logging

• What should be recorded: prompts or protected summaries, context sources, tool requests, policy decisions, approvals and outcomes
• When customers should use it: every production agent
• Why it matters: supports accountability and forensic reconstruction

Identity correlation

• What should be recorded: user, agent, service principal, delegated identity and credential session
• When customers should use it: authenticated workflows
• Why it matters: distinguishes user activity from agent activity

Tool telemetry

• What should be recorded: tool name, version, parameters, result, duration and errors
• When customers should use it: tool-using agents
• Why it matters: detects unusual or unauthorized tool behavior

Network telemetry

• What should be recorded: destinations, proxy decisions, DNS requests and blocked egress
• When customers should use it: network-enabled agents
• Why it matters: detects exfiltration and unexpected communication

Behavioral detection

• What should be recorded: unusual write volume, resource access, retries, spending and delegation
• When customers should use it: autonomous agents
• Why it matters: identifies compromised or malfunctioning workflows

Protected AI telemetry

• What should be recorded: access controls, encryption, redaction, minimization and retention rules
• When customers should use it: logs containing prompts or enterprise data
• Why it matters: prevents monitoring from becoming a new data leak

Logging full prompts and contexts can itself create privacy and security risks because those records may contain secrets, personal data, source code or confidential documents. The objective is not unrestricted collection, but to preserve enough protected evidence to determine why an action occurred, what influenced it, which identity performed it and which control permitted it.

Recovery must be part of the design.

Preventive controls will sometimes fail. To prepare for this, systems that allow agents to modify persistent states should support soft deletion, rollback, versioning, protected backups and tested restoration procedures. This means:

• Infrastructure changes should be reproducible and reversible where possible.
• Credentials should be individually revocable.
• Backups must be isolated from the identity and API path that can delete the primary system. Otherwise, one over-privileged operation may remove both the production asset and the recovery path.
• Recovery should also be tested with agent-specific scenarios.
• Organizations should know how to respond when an agent publishes confidential information, modifies a repository, creates unauthorized infrastructure, changes access permissions, leaks a token or enters a runaway autonomous loop.

Recovery and resilience controls

Soft deletion

• Available implementation options: delayed deletion, recycle bins, retention windows
• When customers should use it: databases, cloud resources, repositories and storage
• What it protects against: immediate permanent loss from agent mistakes

Protected backups

• Available implementation options: immutable, offline or separately administered backups
• When customers should use it: agents capable of production modification
• What it protects against: deletion of both primary data and recovery copies

Versioning and rollback

• Available implementation options: repository history, infrastructure state, database snapshots, configuration versions
• When customers should use it: code, infrastructure, data and policy changes
• What it protects against: incorrect modifications and destructive automation

Credential revocation

• Available implementation options: disable identities, terminate sessions, remove role bindings
• When customers should use it: production agents
• What it protects against: continued access after compromise

Environment rebuild

• Available implementation options: recreate ephemeral sandboxes and replace modified workloads
• When customers should use it: code-execution agents
• What it protects against: persistence and poisoned execution state

Incident playbooks

• Available implementation options: scenarios for disclosure, production changes, token leakage, runaway loops
• When customers should use it: privileged agent deployments
• What it protects against: slow or inconsistent incident response

Recovery testing

• Available implementation options: regular restoration and rollback exercises
• When customers should use it: systems where agents affect availability or integrity
• What it protects against: recovery mechanisms that fail during a real incident

Match controls to capability.

Not every AI system requires microVM isolation, a policy engine and mandatory human approval. The minimum control stack should follow the agent's effective authority.

Capability-based control tiers

Tier 1: public-information assistant

• Typical system: public documentation chatbot
• Principal risks: unsafe output, abuse, misinformation, resource consumption
• Minimum control stack: input/output safety controls, logging, rate limits, source visibility

Tier 2: enterprise knowledge agent

• Typical system: email, file, CRM or internal knowledge assistant
• Principal risks: indirect prompt injection, unauthorized retrieval, data disclosure
• Minimum control stack: user-context authorization, ACL enforcement, DLP, context provenance, egress controls, independent logging

Tier 3: tool-using business agent

• Typical system: support, ticketing, CRM or workflow agent
• Principal risks: excessive agency, unauthorized writes, external communication
• Minimum control stack: separate identity, narrow tools, deterministic authorization, staged outputs, approval for high-impact actions

Tier 4: code and infrastructure agent

• Typical system: coding, DevOps or cloud-management agent
• Principal risks: code execution, credential exposure, persistence, production damage
• Minimum control stack: ephemeral sandbox, network isolation, brokered credentials, development/production separation, policy enforcement, rollback, approval

Tier 5: cyber-capable agent

• Typical system: SOC analyst, vulnerability researcher or autonomous testing agent
• Principal risks: exploit execution, lateral movement, secret harvesting, rapid chained actions
• Minimum control stack: strong workload isolation, mediated networking, task-scoped credentials, comprehensive telemetry, execution budgets, deterministic authorization, approval before active or production-facing operations

It does not matter if the product is called an assistant, copilot, workflow or agent — the dividing line is what the system can reach and what it can change.

Multi-agent systems multiply trust relationships.

Multi-agent systems introduce another layer of complexity because authority can move between agents, e.g. a planning agent may delegate a task to a coding agent, which invokes a testing agent, which then requests access from an infrastructure agent.

If privileges accumulate across that chain, a relatively low-privilege entry point can become a path to a much more powerful operation. Therefore, delegation must never amplify authority. When one agent delegates work to another, the downstream agent should receive no more authority than the caller is legitimately permitted to delegate and should normally receive a further attenuated, task-specific subset of that authority.

The downstream agent should not automatically receive the union of the parent agent's permissions, its own unrelated standing permissions, credentials available elsewhere in the workflow, or the authority of other agents participating in the task. Otherwise, delegation itself becomes a privilege-escalation mechanism.

This attenuation must apply at every delegation hop. A downstream agent may preserve or reduce the authority it receives for the delegated task, but it must never expand that authority by combining it with unrelated standing permissions or credentials.

To summarize:

• If Agent A delegates to Agent B and Agent B delegates to Agent C, each transition should preserve or reduce authorized scope rather than expand it.
• Each delegation should also preserve the initiating identity, task purpose, data provenance, resource scope, expiration and permitted operations.
• Cross-agent messages should be treated as untrusted or partially trusted input rather than automatically authoritative instructions.
• A compromised agent may attempt to manipulate another agent into using capabilities that the first agent does not possess.

The same principles therefore apply between agents:

explicit identities, scoped delegation, privilege attenuation, deterministic authorization, context provenance, constrained tools and independent logging.

Adding more agents should not create an implicit chain of unrestricted trust.

Test the deployed system, not only the model.

• The model
• System instructions, and context assembly
• Retrieval systems
• Identities and credential brokers
• Tools and MCP servers
• Approval mechanisms and policy engines
• Network boundaries
• Logging pipelines and downstream systems

Testing should include:

• Indirect prompt injection
• Poisoned documents and malicious repository content
• Manipulated tool output and excessive tool chaining
• Credential harvesting
• Confused-deputy scenarios
• Unauthorized delegation and attempts to modify approval settings
• Movement from development into production

The critical question is not simply whether the model refuses a malicious instruction.

The critical question is:

What happens when it does not?

Testing should determine whether an agent can:

• Retrieve data outside the user's scope
• Invoke tools outside the approved workflow
• Obtain or expose raw credentials
• Communicate with unapproved destinations
• Modify its own controls
• Persist across sessions
• Expand authority through delegation
• Perform destructive actions without appropriate authorization

It should also test the recovery path with the aim of answering these questions:

• Can responders immediately revoke the agent's identities?
• Can they determine which resources it accessed?
• Can they stop the workflow without redeploying the entire platform?
• Can they restore deleted data?
• Can they determine which input or delegated task influenced the action?

Every material change to an agent should trigger a control review.

A new model, tool, MCP server, credential, data source, network destination, autonomous loop, or delegation path may change the system's effective authority even if the user interface remains unchanged.

Build the control plane before expanding autonomy.

Many of these controls are easier to implement when they are part of the platform architecture from the beginning:

• Credential brokering is easier when tools already use mediated identity rather than when long-lived secrets have been embedded across agent runtimes.
• Deterministic authorization is easier when every action already passes through a tool gateway or API layer.
• Independent telemetry is easier when tool invocation, identity, policy, network and execution events share stable correlation identifiers.
• Sandboxing is easier when execution environments were designed to be disposable rather than when agents have already accumulated persistent local state.

The same applies to approval and delegation: A platform designed around explicit action boundaries can introduce human approval, bounded delegation or policy checks at those boundaries. A platform in which the model directly calls broad APIs may require substantial redesign before equivalent controls can be added reliably.

As a result, organizations should therefore establish the control plane before progressively expanding the agent's autonomy. Autonomy should be something the architecture deliberately grants within defined boundaries, not something that emerges accidentally because additional tools and credentials were connected over time.

From model governance to authority governance

Model governance remains important. Organizations need to understand which models are approved, how data is handled, what evaluations have been performed and which regulatory obligations apply. But agentic systems require something more.

Once an AI system can execute code, call APIs, retrieve private data, communicate externally, modify enterprise systems or delegate tasks to other agents, governance must extend to the identities, credentials, tools, MCP servers, network paths, execution environments, authorization policies, approval mechanisms, delegation scopes, monitoring systems and recovery controls surrounding the model.

This is the enterprise agentic security control plane. While it cannot realistically make the model perfectly trustworthy, it can ensure that an incorrect, manipulated, or compromised agent remains constrained by systems that do not depend on the agent's own judgment.

The design principle is straightforward:

isolate execution, minimize capability, broker credentials, enforce authorization externally, constrain delegation, stage consequential outputs, restrict network paths, record activity independently and make destructive actions reversible.

When the AI Sits Inside the Perimeter established why a capable AI agent inside the enterprise should be treated as a privileged software principal.

This three-part series takes the next step: govern not only the model, but the authority surrounding it.