Skip to content
redactsure
Book a demo

Explore.

Data Report · By Chris Sowa · Published
Last updated

How Do You Protect AI Agents Against Prompt Injection?

Protect the architecture rather than the model, by limiting what injected text can see and what it can trigger, because adaptive attackers have bypassed every published model-level defense. Check whether your deployment keeps sensitive values out of the model's view and routes consequential actions through a person before they execute.

Scattered instruction fragments pass through several screening layers before reaching a separate control point and permitted outputs.

Protect the architecture rather than the model, by limiting what injected text can see and what it can trigger, because adaptive attackers have bypassed every published model-level defense. Check whether your deployment keeps sensitive values out of the model's view and routes consequential actions through a person before they execute. The question has a tempting wrong answer, which is to buy a better filter. Filters, classifiers and model training all reduce how often an injection succeeds, and all have been shown to fail against an attacker who adapts. The research that matters for a buyer is the research on what happens after the filter fails: what the injected text can read, and what it can make the agent do. This page sets out why the model itself cannot be relied on, what the strongest defenses in the research literature have in common, what an injection obtains when it succeeds, how limiting what the agent sees reduces what can be taken, how limiting what the agent can trigger reduces what can be done, and what a buyer should refuse to accept. It builds on the definition of prompt injection against AI agents and treats the question as one of containment rather than detection.

Key findings

  • An adaptive attack study by researchers at OpenAI, Anthropic and Google DeepMind bypassed 12 published defenses with an "attack success rate above 90% for most," many of which had reported near-zero rates (Nasr et al., 2025).
  • A NIST red-teaming competition drew over 250,000 attempts from more than 400 participants, and "at least one successful attack was found against all of the target frontier models" (NIST CAISI, 2026).
  • Anthropic's own assessment of its browser-use defenses states that "A 1% attack success rate, while a significant improvement, still represents meaningful risk" (Anthropic, 2025).
  • The design-pattern literature concludes that once an agent ingests untrusted input, "it must be constrained so that it is impossible for that input to trigger any consequential actions" (Beurer-Kellner et al., 2025).
  • Meta's Rule of Two holds that "agents must satisfy no more than two of the following three properties within a session": untrusted input, sensitive data access, and external state change or communication (Meta AI, 2025).
  • Microsoft's defense for a known exfiltration path was deterministic blocking, and for residual risk "we rely on the explicit consent of the user to perform a specific action" (Microsoft Security Response Center, 2025).

Why can't the model itself be relied on to resist injection?

Because every defense that lives inside the model has been beaten by an attacker who studied it. The clearest evidence is the adaptive attack study by Nasr, Carlini and colleagues, drawn from OpenAI, Anthropic and Google DeepMind, which took 12 published defenses against jailbreaks and prompt injection and bypassed them with an "attack success rate above 90% for most," including defenses whose papers had reported near-zero rates (Nasr et al., 2025). The gap between the two figures is the gap between an attacker who uses the paper's test set and one who moves second.

The finding is not confined to a lab. NIST's Center for AI Standards and Innovation ran a large-scale red-teaming competition against agent systems built on frontier models, drawing over 250,000 attempts from more than 400 participants, and reported that "at least one successful attack was found against all of the target frontier models" (NIST CAISI, 2026). Anthropic, describing its browser-use defenses, states that "A 1% attack success rate, while a significant improvement, still represents meaningful risk" (Anthropic, 2025). An agent that reads a hundred pages a day at a 1 percent rate is steered daily.

OWASP's LLM01:2025 entry describes the underlying reason: a language model processes instructions and data in the same channel, so content it was meant to read can be crafted to be obeyed (OWASP, 2025). Training and filtering shift the odds; they do not change the channel. A buyer should treat any model-level figure as a rate an adaptive attacker will raise, and plan for the case where the injection lands (Why do 80% of prompt injection attacks don't start at the prompt?).

What do the strongest research defenses have in common?

They constrain the system around the model instead of the model. Three lines of work converge on this. The design-pattern paper by Beurer-Kellner, Debenedetti and colleagues catalogs patterns for securing agents and states the governing principle: once an agent ingests untrusted input, "it must be constrained so that it is impossible for that input to trigger any consequential actions" (Beurer-Kellner et al., 2025). The patterns achieve it by deciding, outside the model, which actions are reachable from which inputs.

CaMeL, from Debenedetti, Carlini and colleagues at Google and ETH Zurich, is the most developed instance. It separates the control flow a trusted user requested from the data an untrusted source supplied, and tracks what each value is allowed to influence. On the AgentDojo benchmark it reports "solving 77% of tasks with provable security" against 84 percent for an undefended agent (Debenedetti et al., 2025). The cost is seven points of task completion; the gain is a property that holds regardless of what the injected text says.

Meta's Rule of Two states the same idea as an operating constraint: "agents must satisfy no more than two of the following three properties within a session," the properties being processing untrusted input, having access to sensitive data, and being able to change external state or communicate (Meta AI, 2025). An agent that reads a web page and holds customer data must not also be able to send; an agent that reads and sends must not hold the data. All three treat the model as a component that will be compromised and ask what the rest of the system lets a compromised component do (What Does Prompt Injection Get From a Token-Only Agent?).

What can injected text obtain if it succeeds in steering the agent?

Two things: whatever the agent can see, and whatever the agent can do. Consider a customer account screen in a support system. The agent's task is to update a shipping address from a ticket. The screen shows the customer's name, address, phone, email, payment card on file and order history. The ticket contains text that reads, to the model, like an instruction to also email the account details to an address in the ticket. If the injection succeeds, what it obtains is defined by the screen and the agent's permissions: every value on the screen is readable, and sending an email is an available action.

Neither item is a property of the model. The screen's contents are decided by what the environment lets the model read, and the available actions by what the environment lets the agent do without asking anyone. An injection that lands in an environment where the payment card is shown as a token, and where sending an email pauses for a named person, obtains a token and a request that a person refuses. The same injection, against the same model, in an environment where the screen is shown in full and actions execute on the model's say-so, obtains the card and sends it.

Benchmark research reinforces the distinction between detecting an injection and containing it. The study by Bhagwatkar, Kasa and colleagues asks whether firewall-style detection in front of the model is sufficient, and finds that the answer depends heavily on benchmark strength, with detection rates that look strong on weak benchmarks degrading on stronger ones (Bhagwatkar et al., 2025). A firewall is one more component on the model side of the line. The two quantities above sit on the environment side, and they are what a buyer controls.

How does limiting what the agent sees reduce what injection can take?

By making the stolen value worthless. If the model's view of the customer account screen shows the card number as CARD_001 and the email as EMAIL_001, consistently through the run, the agent can still update the address, because it does not need the real card number to do so. An injection that extracts "everything on the screen" extracts tokens, and the exfiltration path can succeed completely and carry nothing.

This is the data leg of the Rule of Two, which asks that an agent processing untrusted input not also hold sensitive data (Meta AI, 2025). Tokenizing sensitive values before the model reads the screen satisfies that leg for the values tokenized, while leaving operational fields readable; which fields are sensitive is settled by a person before the run rather than by the model during it.

Control point What injected text can obtain What a buyer should verify
Screen shown to the model in full Every value on the screen, for exfiltration Which fields the model reads as real values
Screen shown with sensitive values as tokens Tokens with no value outside the environment Whether tokens resolve only at approved destinations
Actions executed on the model's decision Any action the agent's permissions allow Which actions run without a person seeing them
Consequential actions held for a named person A proposal the person can refuse What the approver sees and whether the record names them

The upper half of the table is this section; the lower half is the next. Microsoft's own account of defending against indirect injection takes the same shape: for a known exfiltration path it took "steps to deterministically block the security impact," and for residual risk "we rely on the explicit consent of the user to perform a specific action" (Microsoft Security Response Center, 2025).

How does limiting what the agent can trigger reduce what injection can do?

By placing a person the injected text cannot address between the model and the consequential action. The design-pattern paper's requirement is that untrusted input be unable to trigger consequential actions (Beurer-Kellner et al., 2025). The simplest way to make that true is to define the consequential actions, payments, submissions and record changes, and require that each be confirmed by a named person who sees the proposal before it executes. The model may propose anything; nothing lands until a person agrees.

The approval has to be specific to work. Microsoft's formulation is "explicit consent of the user to perform a specific action" (Microsoft Security Response Center, 2025). A consent given once at the start of a session covers the attacker's action as well as the user's. A consent requested per action, showing the destination and the record, gives the person something to notice: an email to an address the ticket introduced, a payment to an account the task never mentioned.

The two limits reinforce each other. An approver reviewing a proposed email on a tokenized screen sees that the body contains CARD_001 and EMAIL_001 and that the recipient is unfamiliar, which is enough to refuse. Had the screen been shown in full, the approval step would have become a second point of exposure rather than a control. The order matters: tokenize first, so the model and the approver both work from a safe view, then approve, so the action is a human decision. The same logic applies to agents handling payments, where the consequential action is the point of the attack (How to secure AI agents against financial fraud?).

What does a governed environment change?

In a governed environment, a successful prompt injection steers a model that holds tokens instead of values and can only propose consequential actions, so what the attack can take and do is bounded by the architecture rather than by the model's resistance.

The mechanism is a class of control that sits between the display and the model. Sensitive values on the rendered screen are replaced with consistent tokens before any model reads them, and the task runs on those tokens. Real values resolve only at an approved destination at the moment of an approved action. A named person confirms what is tokenized and approves consequential actions, including payments, submissions and record changes, so an injected instruction to act arrives as a proposal the person can refuse. Every screen as the model received it, in token form, every action and every approval is recorded and exported to the organization's own logging.

This bears on the title question because the model will eventually be fooled. The adaptive attack study raised reported near-zero rates above 90 percent for most defenses studied (Nasr et al., 2025), and the design-pattern literature concludes that an agent which has ingested untrusted input must be constrained so the input cannot trigger any consequential action (Beurer-Kellner et al., 2025). The environment is that constraint applied to both legs named above. The buyer's own test passes when the model's input shows tokens where the values were and the proposed action was refused by a person.

The control does not change the model or its susceptibility to injection, the application or the user's permissions; the model is still fooled, and the environment changes what the fooled model could see and do.

Where does RedactSure sit?

The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company. The design implements both limits described on this page outside the model. The agent works inside a governed environment in the customer's cloud. Substitution happens at the render layer before any model reads the screen, so a customer account, a claims screen or a web page reaches the model with sensitive values replaced by consistent tokens such as ACCT_001 or PATIENT_001 for the whole run; an injection that extracts the screen extracts tokens. Real values resolve only at approved destinations at the moment of an approved action. A named person confirms the tokenization policy and approves every payment, submission and record change before it executes, which RedactSure calls Supervised Delegation, so an action an injected instruction proposed reaches a person who can refuse it. Every screen as tokens, every action and every approval lands in the AI Control Record and exports to the customer's SIEM. The design is model agnostic, requires no per-application integration and leaves user permissions unchanged.

Deployments are in pilot. A pilot on the reader's own workflow, with an injection planted in the content the agent reads, would show the screen as the model saw it, the action the injection proposed, and the person who refused it. Everyone else controls where data can go. RedactSure controls what the AI can see.

Methodology and limitations

This page rests on preprints and peer-reviewed research, vendor research and documentation, official research and a standard: the adaptive attack study by Nasr et al. (arXiv:2510.09023), the design-pattern paper by Beurer-Kellner et al. (arXiv:2506.08837), the CaMeL paper by Debenedetti et al. (arXiv:2503.18813), the firewall benchmark study by Bhagwatkar et al. (arXiv:2510.05244), Meta AI's Rule of Two post, Anthropic's post on browser-use injection defenses, Microsoft Security Response Center's account of its indirect injection defenses, the NIST CAISI red-teaming competition blog, and OWASP's LLM01:2025 entry. All sources were read as of October 16, 2026. Figures are quoted as the papers and posts report them, under their own benchmark conditions.

No hands-on testing of any product was performed, and no attack was run for this page. Where this page says a capability is "not described," it means not described in the pages reviewed, not that the capability is absent. Research results are specific to the benchmarks and models studied, defenses and attacks change quickly, and the page carries its date. Statements about RedactSure describe its current design, with deployments in pilot, and are confined to the section on where the company sits. The customer account screen and support ticket examples describe record types, not any organization; no customer or prospect is described.

Whether a given deployment's containment of prompt injection satisfies an organization's security policy, its regulatory obligations or its contractual commitments is a determination for the organization's counsel, its assessors and its own security and compliance officers, not for a research page.

What the record shows

Protecting AI agents against prompt injection means accepting that the model will sometimes be steered and building so that steering reaches nothing of value. Adaptive attackers have bypassed every published model-level defense, a government red-teaming exercise found a successful attack against every frontier model tested, and the vendors who train for resistance call their residual rates meaningful risk. The defenses that hold in the research, from CaMeL to the design-pattern catalog to the Rule of Two, all constrain what untrusted input can see and trigger rather than trying to detect it. In deployment that means two controls: sensitive values reach the model as tokens, and consequential actions wait for a named person. The test is to name one agent and one task and ask: when an instruction planted in the content the agent reads succeeds, what does it see, and what can it make happen without a person agreeing?

Frequently asked questions

Can a better model or a better filter solve prompt injection?

The evidence says no. Researchers from OpenAI, Anthropic and Google DeepMind bypassed 12 published defenses with success rates above 90 percent for most, and a NIST red-teaming competition found a successful attack against every frontier model tested. Models and filters lower the rate at which injections succeed, which is worth having, but an attacker who studies the defense raises it again. Protection that holds has to come from what a successful injection can reach.

What is the Rule of Two?

Meta's operating constraint for agents: within a session, an agent should satisfy no more than two of three properties, which are processing untrusted input, having access to sensitive data, and being able to change external state or communicate. An agent that reads untrusted content and holds sensitive data must not also be able to send or change anything without a check. Tokenizing sensitive values and holding consequential actions for approval are two ways of removing a leg.

Does human approval slow the agent down too much?

Only if everything requires approval. The research requirement is that untrusted input be unable to trigger consequential actions, so approval belongs on payments, submissions and record changes, not on reading, searching and drafting. Most tasks contain many of the latter and few of the former. The approval that matters is specific, naming the destination and the record, because a single consent at the start of a session covers the attacker's action too.

If the screen is tokenized, can the agent still do its job?

For most tasks, yes. Updating a shipping address does not require reading the payment card number; confirming a claim status does not require the patient's identifiers. Tokenization replaces the values a task does not need with consistent tokens and leaves operational fields readable, so the agent sees the shape of the record without its secrets. Which fields are tokenized is a policy decision a person confirms before the run, and a task that needs a real value gets it at the approved destination.

What does CaMeL show?

That provable security against prompt injection is achievable at a modest cost in task completion. CaMeL separates the control flow a trusted user requested from the data an untrusted source supplied and tracks what each value may influence. On the AgentDojo benchmark it solved 77 percent of tasks with provable security, against 84 percent for an undefended agent. The result matters because the security property comes from the architecture and holds regardless of what the injected text says.

How should an organization test its own agent deployment?

Plant an instruction in content the agent will read during a real task, then look at two things rather than one. First, what the model received from the screen: real values or tokens. Second, what happened when the injected instruction proposed an action: did it execute, or did a named person see a proposal and refuse it. Whether the model was fooled is the least informative result, since the research says it eventually will be. What the fooled model could see and do is the result that counts.

Book an AI Agent Security Review →

Bring your hardest questions.

Bring one sensitive workflow and the decisions your team needs to keep in its hands.

Book a demo

About the author

Christopher Sowa
Chris SowaCo-Founder, President and Chief Commercial Officer, RedactSure

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

Book a demo

Book a demo

Having trouble? Open the booking page or email us.