Expert Briefing: Securing Agentic AI in the Enterprise

About this briefing

A chatbot that gives a wrong answer is a quality problem. An agent that reads a poisoned document and then sends data, changes a record or calls an API with real credentials is a security incident. Agentic AI moves the risk from what a model says to what an agent does with the access it holds.

This briefing covers how that shift changes the attack path, why prompt injection cannot be treated like a patchable bug, how identity and tool access become the main control points, and what a sensible set of controls looks like before an organization lets agents act with less supervision.

What changes when AI can act

An agent combines a model that decides, a set of tools it can call, and credentials that let those tools do something. The attacker does not need to reach the infrastructure. It is enough to influence what the agent reads, because the agent will then act on the attacker’s behalf using its own access.

Prompt injection is a residual risk

The UK National Cyber Security Centre argues that prompt injection should not be compared to SQL injection. SQL injection can be fixed with parameterized queries because data is never read as instructions. Current language models enforce no comparable boundary between instructions and data inside a prompt. The NCSC’s conclusion is that prompt injection may never be fully mitigated, so the goal is to reduce likelihood and impact. Its advice is to limit what an agent can do when it processes untrusted content, apply deterministic safeguards outside the model, log inputs, outputs and tool calls, and treat products that claim to stop prompt injection with caution.

Static tests understate the risk

NIST’s Center for AI Standards and Innovation tested agent hijacking in four simulated environments using agents built on one commercial model. The strongest baseline attack succeeded 11 percent of the time. New red-team attacks, developed with the UK AI Security Institute, succeeded 81 percent of the time on held-out tasks in the workspace environment. Running each attack 25 times raised the average success rate across tasks from 57 to 80 percent.

These are results from simulated settings and one family of test agents. They do not predict the outcome for any particular deployment. The lesson that carries over is narrower and more useful: a single, static benchmark run tells you little, and an attacker who can retry has better odds than a one-shot test suggests.

Excessive agency is a design flaw you control

OWASP lists excessive agency among the top risks for language-model applications and traces it to three causes: excessive functionality, excessive permissions and excessive autonomy. The first two are choices made at build time. An agent with a general shell tool, or with a token that reads every record when the task needs three, has been given more than the task requires. That is the part of the problem an enterprise can fix without waiting for model research to advance.

How an agent compromise unfolds

The table follows one attack path from input to impact and shows where controls hold and where they give false comfort.

Identity is the control plane for agents

An agent is a non-human identity with unusual properties. It acts on instructions that can be influenced. It chains tools. Teams outside security can create one in an afternoon. NIST’s National Cybersecurity Center of Excellence has opened a project on identifying, managing and authorizing software and AI agents, and its concept paper asks for input on identification, authorization, auditing and non-repudiation, and on controls that prevent and mitigate prompt injection. The practical consequences are direct:

• Keep an inventory. For every agent: an owner, a purpose, the tools it can call, and the credentials behind each tool. An agent nobody owns is a standing privilege nobody reviews.

• Separate delegated from autonomous access. An agent acting for a user should carry that user’s limits and never exceed them. An agent acting on its own needs its own identity with a scope written for the task.

• Prefer short-lived, scoped credentials. A stolen long-lived token with broad scope is the worst case in every attack path above.

Where the Model Context Protocol changes the picture

The Model Context Protocol connects agents to tools and data, and its own security guidance names concrete risks. A proxy server that uses a static client ID, allows dynamic client registration and relies on a consent cookie can be abused in a confused deputy attack that skips user consent. Servers must not accept tokens that were not issued to them, since passing tokens through breaks audience boundaries and weakens accountability. Broad or wildcard scopes enlarge the damage from a stolen token, so scopes should start minimal and rise only when needed. Local servers run with the client’s privileges, which calls for explicit consent and sandboxing. Each server an agent connects to is a trust decision, and it deserves the same review as any third-party integration.

Controls to settle before autonomy increases

1. Define an autonomy ladder. Read-only, then draft and propose, then act with approval, then act within limits. Assign each use case a level and name the evidence needed to move up. This is a recommendation, not a published standard.

2. Remove tools the task does not need. Avoid open-ended tools such as general shell access where a narrow function will do.

3. Enforce authorization outside the model. The system the tool touches decides what is allowed, whatever the agent asks.

4. Require human approval for high-impact, hard-to-reverse actions, and show the reviewer the actual action and its arguments, not a summary written by the agent.

5. Contain the blast radius. Sandbox execution, allow only the egress the task needs, and keep standing access to production secrets out of the agent’s reach.

6. Log, baseline and alert. Record tool calls with the agent’s identity, define what normal looks like, and rehearse the step that stops an agent.

7. Test with attacks that adapt and retry. One scripted run is not evidence of resilience.

Briefing outline

What readers take away

• A six-stage attack path for agents that maps each stage to the controls that hold

• An approach to agent identity that separates delegated from autonomous access

• A practical autonomy ladder for deciding what agents may do without a human

• A short list of questions that tests whether an agent program is ready for more autonomy

Questions leaders should be able to answer

• How many agents are running, and who owns each one?

• What could the most privileged agent do if its instructions were hijacked tomorrow?

• Which actions require a human, and does the reviewer see what the agent will actually do?

• How quickly can an agent be stopped, and has that been tried?

• What was tested, with which attacks, and how many attempts?

Use this briefing to assess autonomous-agent security risk. Before reading, list the agents in use or planned, what each one can reach and who approves what. That list is the working material for the discussion. Teams that want an independent view of agent risk can start a conversation with Cybrilliant’s AI security specialists.

Related Reading

Agentic AI Security: What Changes When AI Can Act?

Technical explainer on how agent capabilities change the security model.

AI Agent Tool Abuse: When Authorized Tools Become Attack Paths

Article on how legitimate tool access turns into an attack path.

Runtime Security for AI Agents

Article on the controls that operate while an agent is running.

Perspective: AI Agents Need Security Architecture, Not Just AI Governance

A perspective on why policy alone does not constrain what an agent can do.

Sources

1. UK National Cyber Security Centre. Prompt injection is not SQL injection. ncsc.gov.uk

2. NIST Center for AI Standards and Innovation. Strengthening AI agent hijacking evaluations (technical blog). nist.gov

3. OWASP GenAI Security Project. OWASP Top 10 for LLM Applications: Prompt Injection and Excessive Agency. genai.owasp.org

4. Model Context Protocol. Security best practices. modelcontextprotocol.io

5. NIST National Cybersecurity Center of Excellence. Software and AI agent identity and authorization (project page and concept paper). nccoe.nist.gov