Perspective: AI Agents Need Security Architecture, Not Just AI Governance

A policy cannot stop an agent that already holds the access. Governance sets the rules. Architecture makes sure the rules still hold when the agent acts on bad instructions.

The gap between a rule and a runtime

Most organizations adopting AI agents start in the right place. They build an inventory of use cases, an approval process, an acceptable-use policy and a vendor review. NIST’s AI Risk Management Framework, which is voluntary, and ISO/IEC 42001, a management system standard for AI, give that work structure. It is necessary.

It also stops short. A rule that says an agent must not send customer data outside the company is a statement of intent. If the agent runs with a token that can read the customer database and a tool that can call any URL, nothing in the environment enforces the rule. The agent follows it only while it behaves as intended.

Why the model cannot be the control

An agent receives its instructions and its working data through the same channel. Research from NIST’s Center for AI Standards and Innovation on agent hijacking describes this directly. Trusted developer instructions and untrusted task data are combined in one input, so an attacker who plants instructions in an email, a file or a web page can steer the agent towards actions nobody asked for.

OWASP names the consequence Excessive Agency. Damaging actions follow when a model produces unexpected, ambiguous or manipulated output and the surrounding system gives it too much functionality, too many permissions or too much autonomy. The OWASP Top 10 for Agentic Applications builds on this. Goal hijack, tool misuse, and identity and privilege abuse sit among its leading risks, and it adds the idea of least agency: do not give an agent autonomy the task does not need.

Filtering inputs and outputs helps. It does not change the premise. The model can be persuaded, so the system around it must limit what persuasion can achieve. OWASP’s guidance says the same in design terms: enforce authorization in downstream systems and do not rely on the model’s own decision.

What governance asks and what architecture answers

Seven design decisions

1. Give each agent its own identity. Shared service accounts hide who did what. Each agent needs a distinct identity and short-lived credentials. Where it acts for a user, it needs delegated access that carries the user’s limits and no more.

2. Authorize outside the model. Check every tool call and data request with something the model cannot talk to. This is the policy decision and enforcement pattern from zero trust (SP 800-207): a decision per request, using current context, enforced at a point the subject cannot bypass. Agents are a new kind of subject. The pattern is the same.

3. Shrink the tool surface. Offer few tools with narrow functions. Avoid open-ended tools such as a general shell or an unrestricted URL fetcher. Each broad tool gives a manipulated agent a way to do anything the host allows.

4. Separate reading from acting. An agent that reads untrusted content, holds private data and can communicate externally in the same session has a ready path for data theft. Break the combination. Remove one of the three, or put a control between them.

5. Design approvals that mean something. Require human approval for high-impact actions. Show what the action will do. Make sure the agent cannot approve for itself. Prompts fired at every step teach people to click through.

6. Bind tokens to their purpose. The Model Context Protocol’s security guidance forbids token passthrough: a tool server must accept only tokens issued for it. It also warns against broad scopes and describes confused deputy attacks on proxy servers. These are familiar OAuth problems, but agent toolchains multiply the servers and tokens in play.

7. Contain and record. Run code execution in sandboxes. Restrict outbound access. The MCP guidance recommends blocking private and link-local address ranges, including cloud metadata endpoints, for server-side fetches. Log each action with the user, agent, tool, parameters and result. The same guidance notes that token passthrough damages the audit trail by hiding who made a request.

Where governance still wins

Architecture without governance builds well-engineered agents nobody approved. Governance decides which use cases justify the risk, who accepts it, and what must be true before an agent reaches production access. The argument is not to replace one with the other. Governance output has to become architecture requirements. Each policy statement should map to a control that enforces it, or be recorded as unenforced.

Two limits apply. No design removes risk for an agent with real autonomy, so residual risk needs an owner. And guidance for agents is young and changing quickly. Treat any control catalog, including the ones cited here, as a starting point and review it often.

A test to run this week

Pick one agent in production or planned. Draw the path from a user request to every system it can touch. For each step, mark where authorization is enforced and by what. Any step where the only enforcement is the model’s instructions is a gap. Rank the gaps by what the agent could do through them. That list is your first set of architecture requirements.

Next step

Architecture has to keep pace with the way agents are adopted and attacked. Keep the path map as a living record and review it whenever tools, permissions or models change. Each review shows whether the controls still match what the agents can do.

Sources

1. OWASP, Top 10 for LLM Applications, LLM06 Excessive Agency.

2. OWASP, Top 10 for Agentic Applications.

3. NIST Center for AI Standards and Innovation, Strengthening AI Agent Hijacking Evaluations.

4. NIST, Special Publication 800-207, Zero Trust Architecture.

5. NIST, Artificial Intelligence Risk Management Framework.

6. ISO/IEC 42001, Artificial intelligence management system.

7. Model Context Protocol, Security Best Practices.