Skip to content
redactsure
Book a demo

Explore.

Data Report · By Chris Sowa · Published
Last updated

Does a Smarter Model Stop Prompt Injection?

At the margin: a stronger model follows injected instructions less often, and when it follows one it still holds whatever the page rendered, so the loss is set by exposure and permissions rather than by model quality. Ask for the failure rate and for what one success reaches.

A credential reference travels from a browser record through a bounded route to a separate API destination.

Model releases arrive with benchmark tables, and the number that bears on this question is the rate at which injected instructions succeed. That rate has improved for several years and has not reached zero, which the vendors say themselves. The other quantity has not moved: what the agent holds when an instruction lands. A rate can always be pushed lower, and a single loss is either bounded by design or it is not. A page that renders an account number into the model's context gives up the same number at a one percent success rate as at ten, and a model that refuses nine instructions in ten still carries whatever the tenth exposes.

Adoption has not waited for the benchmark. McKinsey's State of AI research finds 62% of organizations at least experimenting with AI agents and 23% scaling them anywhere, a gap that What Is the PII Wall? sets out: agents are arriving before the controls that bound them. The sections that follow work through one record type, a repository settings page an agent opens to change who may push to the default branch, at a stipulated rate of one success in 1,250. On that page a stronger model changes how often the change is attempted, while the page and the run's permissions decide what one success reaches, and that second part is where an organization's exposure sits.

Key findings

  • Anthropic states that its protections "significantly reduce risk" and that "no system is completely immune to all attacks", a statement about how often an injection works and not about what it collects (Anthropic, Security, 2026).
  • Veracode reports that "GenAI models make the wrong choices nearly half the time, and it's not improving", so a model score is not a control to plan around (Veracode, 2025).
  • The same release finds "a security failure rate over 70 percent" in Java, a spread that one benchmark score hides (Veracode, 2025).
  • Willison's design patterns "constrain the actions of agents to explicitly prevent them from solving arbitrary tasks", putting the control in what an agent can reach (Willison, 2025).
  • Substitution at the render layer changes what the model reads, so an instruction that succeeds collects a placeholder rather than the value behind it (Can an AI Agent Complete Its Task Without Seeing PII When Prisma AIRS Protects It at Inference?).
  • Real values resolve only at the approved destination at the moment of an approved action, so a successful instruction holds a token while the values stay out of reach (Does Prompt Injection Expose Customer Records?).

What does a stronger model change on the page an agent reads?

Scope first. A governed browser environment covers what an agent reads and does on a rendered page: a dashboard, a console, a settings page, a sign-up form. It does not cover the terminal, the IDE, the local filesystem, direct API calls or MCP. A model is the component that reads the page and decides what to do next, so the page is the unit this question is about.

The worked record is a repository settings page. An agent is asked to change who may push to the default branch, so it opens the repository's settings, reads the branch protection panel and the Actions permissions panel, and reports back. What the panel renders is what the model receives: rule names, team names, deploy key titles, the webhook URL, the token scopes, and any text an attacker has placed where the agent was told to look, such as an issue title, a description field or a comment.

A stronger model changes one number on that page: how often it treats that text as an instruction. It does not change the other property, which is that when it does, the panel's values are already in its context. The model that refuses ninety-nine instructions in a hundred is the same model on the hundredth, with the same page in front of it.

Filtering at inference is the nearest documented alternative: Palo Alto Networks describes Prisma AIRS connecting through inference hooks to "apply runtime guardrails before model inference", a check on the prompt after the page has been rendered (Can an AI Agent Complete Its Task Without Seeing PII When Prisma AIRS Protects It at Inference?). That check is worth having, and it is still a judgment about text, made by or around a model.

Why is a lower failure rate not a smaller loss?

A rate and a blast radius measure different things. The rate counts how often an injected instruction is followed. The blast radius counts what is reachable the one time it is: the values on the screen the model read, the files its tools can open, the calls its permissions allow. A model upgrade moves the first and leaves the second where the deployment put it.

Anthropic's engineering post gives the shape of the first: sandboxing "safely reduces permission prompts by 84%", and "effective sandboxing requires both filesystem and network isolation" (Anthropic Engineering, 2025). That is a real reduction in the decisions a person makes, and it is a change in the design of the tool rather than in the model. The security page supplies the limit on the other side: "the boundary is a permission prompt, so a Bash command you approve can still write anywhere your user account can" (Anthropic, Security, 2026). The number of prompts falls; the reach of an approved command does not.

Take the worked record's rate as a supposition, one success in 1,250, and hold the volume constant. A success every 1,250 attempts is a certainty over a quarter of ordinary volume, and the arithmetic is the point rather than the figure: any non-zero rate becomes a live attack at volume.

What changes the second quantity is a smaller context, not a better model: fewer values on the page, fewer permissions on the credential, fewer hosts the run can reach. That is the difference between lowering a rate, which has no floor, and bounding exposure, which can reach zero.

What do the vendors themselves say?

Anthropic documents its Claude Code controls in enough detail to test this question against them, and is careful about what each control covers.

The sandboxing page places the sandbox around shell commands only, with Read, Edit, Write, WebFetch, WebSearch, MCP servers and hooks outside it, and with masking "the sandboxed command sees a sentinel and the local proxy substitutes the real value only on hosts in injectHosts" (Anthropic, Sandboxing, 2026). A masked credential therefore stays out of the command while the request to a listed host still carries the real value.

The permissions page adds the file-tool rule and the gap beside it: a Read deny rule blocks a path such as Read(./secrets/**), and a .claudeignore file "has no effect, so move its entries into Read deny rules" (Anthropic, Configure, 2026). A team that reads the file name and assumes coverage has neither coverage nor a record of the assumption.

Dimension Anthropic, per its documentation What a buyer should verify
Injection defense Sandbox isolation and permission rules, described as reducing risk Whether the control lowers a rate or bounds the loss
What the model receives Whatever the page, file tool or MCP server returns Which values the model may be shown on a rendered page
Shell reach An approved Bash command can write anywhere the user account can Which account the agent runs as
Credential custody Cloud sessions keep GitHub credentials on Anthropic's servers, never in the session VM Whether that custody applies to a local session
Scope Sandbox covers shell commands only; file and web tools follow permission rules Which routes into the context sit outside the setting
Approver A person answers the permission prompt Who approves a consequential write, and what the record shows
Record Permission prompts and sandbox settings; no page-level record described What the model read on the screen, screen by screen

Read together, the pages describe a control set built around actions: a sandbox around commands, permission rules around tools, a prompt a person answers. They describe less about the values a rendered page puts in front of the model. The last two rows carry the weight: an approver for the values a page renders, and a record of what the model read on that screen, are not described in the pages reviewed.

What does one successful injection actually take?

On the worked record, the 1,250th attempt takes the settings page. The model holds the branch protection rules, the team names, the deploy key titles, the webhook URL and the token scopes, because the page rendered them. What it can do with them is a separate question from whether it was fooled, and the research answers the first half: values that reach a model can come back.

The memorization study of neural code completion tools reports that they "can not only return the precise piece of their training data" and that "two valid credentials were identified during our experiments" (Huang et al., 2024). SecretBench puts a number on the population such a value joins: "a labeled dataset of source codes containing 97,479 secrets (of which 15,084 are true secrets)" (Basak et al., 2023). A webhook URL an agent pastes into an issue is one more entry in that population, and it is a live entry until someone rotates it.

On a rendered page the same logic holds with a measured number. An environmental injection attack on generalist web agents reached an attack success rate of up to 70 percent in stealing users' specific personal information (Can an AI Agent Sign Up for Developer Services Without Seeing Your Payment Card or Identity?). The attack did not need a weak model; it needed a page that rendered the value.

The lethal trifecta, as its originator defined it, is access to private data, exposure to untrusted content and the ability to communicate externally, and a stronger model changes none of the three (What Is the Lethal Trifecta for AI Agents?). A model that is harder to fool is still an agent with all three.

Which routes never reach the model's judgment?

Some instructions arrive as code rather than as a prompt, and a model that refuses every injected sentence has nothing to refuse in them.

SLSA's threat list includes an attacker who "compromised the build platform and installed an implant that injected malicious behavior during each build", and one who "added an innocuous dependency and then later updated the dependency to add malicious behavior" (SLSA, 2026). Neither route asks a model for a judgment; both run at build time.

The file route is the second. A deny rule governs the file tools for the paths it names, and the permissions page says the rules "don't apply to a command that reads files without naming them" (Anthropic, Configure, 2026). A sandbox around shell commands leaves the Read tool outside it, so a file the agent opens arrives in the context whole.

The privilege route is the third. Google's security guidance for Workspace administrators states that the super admin account should be used only when needed, a rule written for people that applies with more force to an agent (Does an AI Agent Need Your Admin Login to Set Up a SaaS Tool?). An agent signed in as an administrator can change settings that no argument would talk a model out of changing.

OWASP ranks prompt injection as LLM01 and separates direct injection, entered by a user, from indirect injection, embedded in external content the model later processes (Do 80% of Prompt Injections Start Outside the Prompt?). The indirect kind is what a settings page, a dependency and a file carry, and none of that is repaired by a better model.

What does a governed environment change?

A governed environment is a control class that acts where a page is rendered, before any model reads it. Values chosen by policy are replaced with consistent tokens for the whole run, so a settings page reaches the model as DEPLOYKEY_001, WEBHOOK_002 and TEAM_003. Real values resolve only at the approved destination, at the moment of an approved action. A named person confirms the tokenization policy and approves each consequential action, and adding a deploy key is one. Every screen as tokens, every action and every approval is written to a record that exports to the organization's own monitoring.

For this page's question it changes the second quantity and leaves the first where it was. An injected instruction can still succeed, at whatever rate the model allows. What it collects is a set of tokens that mean nothing outside the environment, and the action it proposes needs a name before it happens. The rate remains a vendor's number; the loss becomes the customer's decision.

A rate can be improved and cannot be spent down to zero. Exposure can be bounded, and a token has no value to an attacker who cannot resolve it. That is why the two are reviewed separately: one belongs to the model vendor's roadmap, and the other belongs to the environment the agent runs in.

The scope is the rendered page. A governed environment does not cover the terminal, the IDE, the local filesystem, direct API calls or MCP, so an instruction that arrives through a dependency or a shell command stays with the controls that govern those routes.

Methodology and limitations

This page rests on two research preprints, three Anthropic documentation pages and one Anthropic engineering post, the SLSA specification, one practitioner analysis, one vendor press release, and existing RedactSure research pages. Vendor pages were read as of October 20, 2026, and each Sources line carries the date for its page. Quoted vendor statements are given in the vendor's wording, and research findings are given as their abstract pages state them. The two preprints are cited for the mechanisms their abstract pages describe, not for rates those pages do not give. The Veracode release describes 80 coding tasks and more than 100 large language models, with vulnerabilities in 45 percent of cases, and it does not name the models or publish a full methodology (Veracode, 2025).

RedactSure did not test any of the named products hands-on. Where this page refers to behavior the recorded documentation does not cover, it means the pages reviewed, not a claim that a product lacks the feature. Vendor features change, and the page carries its date; a release after that date is unreviewed until the test below is repeated. Statements about RedactSure describe its current design, and RedactSure is deploying now with enterprise and government customers. The example describes a record type, a repository settings page, and not an organization, and the rate of one success in 1,250 is a supposition used for arithmetic, not a measurement of any product.

Determinations about whether a control meets a contractual, regulatory or audit requirement belong to counsel, assessors and the organization's own security and compliance officers. Nothing here replaces a test of the organization's own agent against its own pages, repeated after each model release or vendor change.

What the record shows

The record answers the title in two parts. A stronger model lowers how often an instruction is followed, which the vendors describe as a reduction and not as immunity. It does not lower what one success reaches, because the reach is set by the page the model read and the permissions the run holds, and both are chosen before the model is chosen. RedactSure's environment addresses the second part, and a reader can test the first without it. Two questions make the result usable with a vendor: what the failure rate is, and what the environment removes from the context before the model reads it. A vendor that can answer both has a control; one that answers only the first has a rate. To see the two parts separate, name one agent and one task, run it against a page that renders a credential, and write down what the model received and what it could do with that; then change the model and run it again.

Frequently asked questions

Does a smarter model stop prompt injection?

No. It makes an injected instruction fail more often, and it does not change what the agent holds when one succeeds. The model reads the page it is given, so the values on that page are in its context whether it obeys or refuses. A refusal rate of 99.9 percent still leaves a working attack at volume, and the loss on that day is whatever the page rendered. A smarter model buys time between incidents, not a bound on any one of them.

What does a lower failure rate actually buy?

It buys fewer incidents, fewer alerts and fewer interruptions, which is worth having. Anthropic reports that sandboxing "safely reduces permission prompts by 84%", a reduction in decisions rather than in reach. What the rate does not buy is a bound on any single failure: the reach of an approved command stays wherever the account's permissions put it. Buy the rate for the workload it removes, not for the guarantee it does not give. The two purchases are separate line items.

Why use one success in 1,250 as the example?

It is a supposition, chosen because it is small enough to read as safe and large enough to compute with. No product is claimed to have that rate. At volume the arithmetic turns it into a regular event, and the argument does not depend on the number: any non-zero rate produces successes, and each success collects what the run can reach. Substitute a better figure and the second half of the argument is unchanged. The point is the shape of the curve, not its value.

Is a better model a substitute for sandboxing and deny rules?

No, because the controls cover different routes. Sandboxing covers shell commands, deny rules cover the file tools for named paths, and masking covers what a sandboxed command sees. None of them decides which values a rendered page shows the model. A better model reduces the chance that an instruction is followed on the routes it governs, and the other routes do not ask it. Read the controls as a list of routes, not as a ranking of models.

What can an injected instruction take from a repository settings page?

Whatever the panel renders: branch rules, team names, deploy key titles, webhook URLs and token scopes, plus any text an attacker placed where the agent was told to look. Those values enter the context together, so a refusal on one instruction does not remove them. What the agent can then do depends on its permissions, and whether a person approves the action decides whether it happens. Rotate anything the page exposed, whether or not the instruction was refused.

What should a team ask before treating a model upgrade as a control?

Ask for the failure rate and the test that produced it, then ask what a single success reaches: which values the page renders, which files the tools can open, which hosts the run can contact, and who approves a consequential change. Ask which routes bypass the model entirely. A vendor answer that stops at the rate answers the smaller half of the question. Put both answers beside the page the agent will actually read.

About the author

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

How RedactSure Makes a Hijacked Settings Page Worth Only Tokens

The controls above lower a rate or bound a route, and neither changes what a model holds when an injected instruction lands.

The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company.

Before: the agent opens the repository settings page, the panel renders the deploy key and the webhook URL, and a line hidden in an issue title can send it to add a key of someone else's choosing. Inside: the same panel renders DEPLOYKEY_001 and WEBHOOK_002, and the real values resolve only for the destination the reviewer approved.

RedactSure is deploying now with enterprise and government customers. Veracode found models making the wrong choice nearly half the time, and Anthropic states that no system is immune.

Everyone else controls where data can go. RedactSure controls what the AI can see.

See it on your own workflow

A more capable model still reads what the page shows it. Bring a repository settings change of your own and see, with test values, what a hijacked run would have collected. Book a demo

Book a demo

Book a demo

Having trouble? Open the booking page or email us.