AI Security Controls: What Should Be Preventive, Detective and Responsive?

Prevent what you can enforce deterministically. Detect what is probabilistic. Prepare to respond to whatever you can neither prevent nor reliably detect. That split, applied risk by risk, is what turns a long list of AI controls into a defensible control set.

Not every AI risk deserves a preventive control, and some cannot be prevented reliably at all. Teams that treat guardrails as the whole answer end up with no detection and no response when the guardrail fails. The framework below helps you decide, for each AI system, where to stop an attack, where to see it and where to contain it.

Why AI needs this split

Conventional controls assume deterministic systems. A firewall rule either matches or it does not. A model’s behavior is probabilistic. Prompt filters, system prompts and safety tuning lower the likelihood of a bad outcome. None of them guarantees one. So the useful question is not “which controls do we have?” but “what kind of assurance does each control give?”

This sits comfortably with current guidance. NIST’s AI Risk Management Framework organizes work under Govern, Map, Measure and Manage, and its Generative AI Profile adds risks specific to generative systems. NIST’s draft Cyber AI Profile, built on the Cybersecurity Framework, groups its focus areas into securing AI systems, AI-enabled cyber defense and thwarting AI-enabled attacks. This framework addresses the first of those: securing the AI systems you build and buy. Both NIST documents are evolving, so check their current status before citing them in policy.

Step 1: Map controls to the layers of an AI system

The table gives a starting control set for each layer. Adapt it to your architecture.

Layer and main risk	Preventive	Detective	Responsive
Data Poisoning, leakage, sensitive data in prompts or indexes	Classify data before it reaches a model or index. Enforce the user’s access rights at retrieval time. Track provenance of training and fine-tuning data.	Alert on unusual ingestion. Scan outputs for sensitive data. Review retrieval logs.	Purge and rebuild a poisoned index. Roll back a fine-tuned model. Meet notification duties.
Model and supply chain Tampered models, datasets and libraries	Approved model registry. Verify hashes or signatures. Use vetted sources and model formats that cannot execute code on load.	Scan artifacts. Watch for behavior drift against a baseline.	Revoke or replace a model version. Use an inventory to find every system that uses it.
Application and prompts Injection, jailbreaks, unsafe output	Separate trusted from untrusted context. Constrain output formats. Treat model output as untrusted input to downstream code. Filters help but do not guarantee.	Log prompts and responses. Detect injection patterns and policy violations. Red-team on a schedule.	Update rules, disable a feature, rotate any secret that leaked.
Tools and agent actions Excessive agency, tool misuse	Tool allowlists. Argument validation. Sandboxed execution. Approval for irreversible actions. Policy enforced outside the model.	Log action sequences. Alert on out-of-profile tool use and read-then-send patterns.	Kill switch. Revoke the agent’s credentials. Roll back changes, which requires actions designed to be reversible.
Identity and access Agents, service accounts, API keys	Unique identity per agent. Scoped, short-lived credentials. No secrets in prompts.	Flag credentials used outside their runtime. Flag scope escalation.	Revoke and rotate. Reissue with narrower scope.
Infrastructure and runtime Inference endpoints, model theft, abuse	Authenticate inference endpoints. Segment networks. Apply rate limits.	Monitor query volume and patterns that suggest extraction or abuse.	Throttle, block, isolate the endpoint.
Shadow and third-party AI Unsanctioned tools, data sent outside	Sanctioned tool list. Data loss controls on AI destinations. An approval route that is quicker than going around it.	Discover usage from egress and SaaS logs.	Policy action, plus a data recovery or deletion request to the provider.

Step 2: Decide the emphasis for each use case

Three questions set where the weight should fall.

1. Can the control be enforced outside the model, deterministically? If yes, make it preventive. A tool allowlist either permits a call or blocks it.

2. If the control fails, can we see it in time to act? If yes, detection can carry real weight. If you have no telemetry, detection is a hope.

3. If the harm cannot be undone, can we contain it? If not, move the control up to prevention, or reduce the capability so the harm cannot occur.

The answers feed a simple rule.

Step 3: Make sure detective controls are real

A detective control needs four things: telemetry, a baseline of normal, a threshold and a person who triages. A log nobody reads is a retention policy, not a control.

For AI, the telemetry that matters most is the sequence of actions, not single requests. Anthropic’s report on an AI-orchestrated espionage campaign describes operators splitting work into small tasks that each looked harmless. Judging requests one at a time would not have caught that. Logging prompts, tool calls, retrieval queries and the identity behind each, then looking at patterns over a session, would have a better chance. MITRE ATLAS gives detection engineers a common technique vocabulary for AI-specific behavior.

Step 4: Decide what must be possible before an incident

Response for AI systems rests on a short list of capabilities. Check each exists and has been tried.

• Disable a model feature, tool or agent without taking down the whole product

• Revoke an agent’s credentials quickly and confirm they no longer work

• Preserve prompts, responses and tool logs for investigation, within your data protection duties

• Roll back a model version, a prompt template or a retrieval index

• Reach the model or platform provider through a route agreed in advance

Run a tabletop for two scenarios: an injected instruction that causes data to leave, and an agent that keeps acting after it should have stopped.

Step 5: Match depth to the use case

Applying the same controls to every AI use case wastes effort on low-risk systems and under-protects the ones that matter. A tiering model fixes that. The tiers below are a starting point to adapt, not a standard.

Common failures

• Guardrails as the only control. Nothing is left when they are bypassed.

• Logging with no owner. The data exists and nobody acts on it.

• Planning for the provider’s outage, not your own agent’s misbehaviour. These need different playbooks.

• One depth for every use case. Either too heavy or too light.

• Borrowing the vendor’s safety claims as your control. A provider’s safeguards cover its layer. Your data, tools and identities are yours.

Apply it

List your AI systems and place each in a tier. Then check whether the preventive, detective and responsive columns each have a named owner and a capability you have tested. Start with any system where the responsive column is blank.

Sources

1. NIST, AI Risk Management Framework and Generative AI Profile (NIST AI 600-1). https://www.nist.gov/itl/ai-risk-management-framework

2. NIST, Cybersecurity Framework Profile for Artificial Intelligence (Cyber AI Profile), NIST IR 8596, preliminary draft. https://csrc.nist.gov/pubs/ir/8596/iprd

3. OWASP GenAI Security Project, Top 10 for LLM Applications. https://genai.owasp.org/llm-top-10/

4. OWASP GenAI Security Project, Top 10 for Agentic Applications. https://genai.owasp.org/owasp-top-10-for-agentic-applications

5. Anthropic, Disrupting the first reported AI-orchestrated cyber espionage campaign. https://assets.anthropic.com/m/ec212e6566a0d47/original/Disrupting-the-first-reported-AI-orchestrated-cyber-espionage-campaign.pdf

6. MITRE, ATLAS knowledge base of adversary tactics and techniques against AI systems. https://atlas.mitre.org