Skip to content
redactsure
Book a demo

Explore.

Data Report · By Chris Sowa · Published
Last updated

What Are AI Agent Guardrails?

Guardrails are controls placed around a model's inputs, outputs and actions; most products ship content guardrails that filter inputs and outputs, while action guardrails limit what the agent can do and what data it receives. Check whether a vendor's guardrail has been tested against an adaptive attacker, because the published record shows content filters fail that test.

Scattered data shapes approach a circular platform with guardrails and emerge as an orderly flow.

Guardrails are controls placed around a model's inputs, outputs and actions; most products ship content guardrails that filter inputs and outputs, while action guardrails limit what the agent can do and what data it receives. Check whether a vendor's guardrail has been tested against an adaptive attacker, because the published record shows content filters fail that test. A vendor can describe a topic classifier, a pattern for card numbers, a tool-call allow-list and a human approval step as guardrails, and all four are, loosely. They differ in what they inspect and how they fail. Filters that read the text going into and coming out of a model are the most common and the easiest to ship, and they are also the ones the research literature has shown to be defeated by attackers who adapt. Controls that constrain what the agent can do, and what it is given to see, fail differently, because they do not depend on recognizing a bad input. This page distinguishes the two, using vendor documentation for what guardrail products describe, a patient intake screen to show where content guardrails act and what they miss, the adaptive attack literature for why they fail, the design pattern literature for what action guardrails look like, and a method for evaluating any guardrail claim, drawing on research on AI agent data exposure.

Key findings

  • NVIDIA's NeMo Guardrails documentation defines five rail types, input, retrieval, dialog, execution (tool call) and output, and notes that "Input and Output rails are the most common," which is the industry's own account of where guardrail effort concentrates (NVIDIA, 2026).
  • Amazon Bedrock Guardrails evaluates a request in two passes: "the input is evaluated against the configured policies specified in the guardrail," and the model response is then evaluated the same way (AWS, 2026).
  • A multi-institution study of adaptive attacks bypassed 12 published defenses with an "attack success rate above 90% for most," many of which had reported near-zero rates under their original evaluation (Nasr et al., 2025).
  • The design pattern literature states the alternative plainly: once an agent ingests untrusted input, "it must be constrained so that it is impossible for that input to trigger any consequential actions" (Beurer-Kellner et al., 2025).
  • OWASP's excessive agency entry identifies over-granted functionality, permissions and autonomy as the condition that turns a filter failure into a harmful action (OWASP, 2025).
  • Data minimization at the screen is a guardrail that does not depend on recognizing an attack: a model that received PATIENT_001 cannot be made to leak the patient's name, the argument of research on sensitive data masking for AI agents.

What do vendors mean by guardrails?

NVIDIA's NeMo Guardrails library describes input, retrieval, dialog, execution and output rails, the last applied to the model's response and execution rails applied to tool calls, and observes that input and output rails are the most common (NVIDIA, 2026). Amazon Bedrock Guardrails describes a flow in which the input is evaluated against configured policies, the model is invoked if the input passes, and the response is then evaluated against the same policies before being returned (AWS, 2026). Databricks documents guardrails as a feature of its AI Gateway, applied to requests passing through the gateway to model endpoints (Databricks, 2026). Portkey documents guardrails as checks applied to requests and responses through its gateway (Portkey, 2026).

The common shape is a checkpoint on text. Something is read, evaluated against a policy, and allowed, blocked or modified. The policy may be a topic list, a classifier or a pattern; the checkpoint sits at the input, the output or both. Execution rails in NVIDIA's taxonomy are the exception: they inspect tool calls rather than conversational text, which places them nearer the action than the content.

What the pages reviewed do not describe is as important as what they do. None of the four describes governing what a browser agent receives from a rendered screen before the model reads it; their inputs are the prompt and retrieved content as submitted to the model or gateway. None describes a named person approving a consequential action as part of the guardrail. That is not a criticism of the products; it marks the boundary of the term as the market uses it.

Where on a patient intake screen do content guardrails act, and what do they miss?

Take a patient intake record in a clinic's scheduling system. The screen shows the patient's name, date of birth, medical record number, insurance identifier, the reason for the visit, a referral note pasted from an outside provider, and a scheduling panel. An agent is asked to read the intake, confirm insurance eligibility in a second system, and book a follow-up appointment. Content guardrails sit in two places on that workflow: on what the agent sends to the model, and on what the model returns.

Trace a failure. The referral note contains an instruction telling the agent to include the patient's name and insurance identifier in the appointment confirmation it sends to an external address. The input guardrail inspects the text going to the model. If the instruction is phrased to evade the classifier, which is the attacker's entire job, it passes. The model follows it. The output guardrail inspects the response. It may catch an insurance identifier if the pattern is configured; it will not catch a name, because names are not a pattern, and it will not know that the destination address is wrong, because destinations are not content. The confirmation goes out.

Two things made this possible, and neither is a guardrail configuration error. The model received the patient's real name and identifier, because the screen contained them and nothing substituted them before the read. And the agent could send a message to an arbitrary address, because its capabilities were not constrained to the destinations the task required. Content guardrails were present at both checkpoints and acted exactly as designed. The failure lies in what they were never positioned to see: the real data entering the model, and the consequential action leaving the agent. Research on securing AI agents frames the exposure in the same terms, as a question of what the model holds and what it can do rather than what it says.

Why do adaptive attacks defeat content guardrails?

Because a content guardrail is a classifier, and a classifier can be probed. Nasr, Carlini, Sitawarin, Schulhoff and colleagues from OpenAI, Anthropic, Google DeepMind and other institutions evaluated 12 published defenses against jailbreaks and prompt injections using adaptive attacks, where the attacker iterates against the defense rather than submitting a fixed test set. They report bypassing the defenses with an attack success rate above 90 percent for most, including defenses whose original papers had reported near-zero success rates (Nasr et al., 2025). The attacker moves second, and a defense evaluated against attacks that did not know about it is measuring something other than security.

The finding generalizes to the guardrail products on this page, not because those products were in the study, but because they share the mechanism. A policy that evaluates text, however sophisticated, defines a boundary, and an attacker who can submit text repeatedly can map that boundary and step around it. Arthur Holland Michel's October 2026 analysis in MIT Technology Review makes the same point from the policy side, arguing that organizations are placing too much faith in a model's ability to say no (MIT Technology Review, 2026). Refusal, trained in or bolted on, is probabilistic, and a control whose failure rate is a probability is not one a compliance officer can certify.

Content guardrails still raise the cost of casual misuse and catch accidental disclosures of patterned data. What they cannot do is serve as the control of record against a motivated attacker, and the agentic setting is the one in which motivated attackers have the most to gain, because the agent can act.

What are action guardrails, and how do they differ?

Beurer-Kellner, Buesser, Cretu, Debenedetti and colleagues propose design patterns for securing agents against prompt injection and state the governing principle: once an agent has ingested untrusted input, "it must be constrained so that it is impossible for that input to trigger any consequential actions" (Beurer-Kellner et al., 2025). A content guardrail makes a bad outcome unlikely; an action guardrail removes the path.

OWASP's excessive agency entry describes the condition action guardrails address: an agent granted more functionality, permissions or autonomy than its task requires (OWASP, 2025). Narrowing agency is an action guardrail, as is requiring a named person to approve consequential actions, and so is data minimization at the screen: a model that never received the patient's name cannot leak it.

Guardrail layer What it inspects or constrains How it fails on the intake screen
Input content rail Text submitted to the model Injected referral note is phrased to pass the classifier
Output content rail Text returned by the model Patient name and wrong destination are not patterns
Capability limit What the agent can do at all Only if the agent was allowed external messages
Named-person approval Consequential actions wait for a person who sees real values Automation bias; forged dialogs
Substitution at the screen What the model receives Only if a protected field was missed in policy

The lower rows do not depend on recognizing an attack. A content rail asks whether an input is malicious and must answer correctly every time; an action guardrail removes the path. Research on securing AI agents in the enterprise treats the lower rows as the controls of record and the upper rows as supporting layers.

How should an organization evaluate a guardrail claim?

Evaluate with an adaptive attacker, on the organization's own screens, with the consequential action live. Vendors' demonstrations tend to omit all three. The adaptive attacker is required because the published record shows static test sets overstate protection by an order of magnitude (Nasr et al., 2025); a red team given a day and the ability to iterate is the minimum. The organization's own screens are required because a guardrail tuned on a vendor's sample data knows nothing about the referral note field in the clinic's intake system. The live consequential action is required because the question is not whether the model says something wrong but whether the appointment, the message or the payment occurs.

Record four things per attempt: what the model received, what it produced, what action was attempted, and what action completed. The first column is where substitution shows its value, because an attempt that produced a URL full of tokens is a failed attack regardless of whether any filter fired. The fourth column is where approval shows its value, because an attempt that reached a named person and was refused is also a failed attack. A guardrail evaluation that records only the second column, what the model produced, is measuring content and ignoring the agent.

Then ask the vendor what the guardrail inspects and what it constrains, in the table's terms. A product that inspects input and output text has two rows. A product that also limits tool calls has three. A product that governs what the model receives from the screen and holds consequential actions for a person has the lower rows as well. Research on AI agent data security sets out why the lower rows are where an organization's compliance position actually rests. The requirement is to know which layer is being relied upon.

What does a governed environment change?

In a governed environment, the guardrail moves from recognizing an attack to removing its payload, because a model that received tokens instead of patient names has nothing to leak whether or not a filter catches the prompt.

The mechanism is a class of control that sits between the rendered screen and the model, below the input and output rails most products ship. Sensitive values on the page, such as the name, date of birth and insurance number on an intake screen, are replaced with consistent tokens before any model reads them, so the task runs on tokens. Real values resolve only at an approved destination at the moment of an approved action. A named person confirms what is tokenized and approves consequential actions, including payments, submissions and record changes. Every screen as the model received it, in token form, every action and every approval is recorded and exported to the organization's own logging.

This is the lower row of the page's table, and it answers the adaptive attack problem differently from a classifier. Defenses reporting near-zero attack success failed above 90 percent once the attacker could iterate (Nasr et al., 2025), and no iteration recovers a value that was never present. The held action is the second half: once an agent ingests untrusted input, the design pattern literature says it must be constrained so that the input cannot trigger a consequential action (Beurer-Kellner et al., 2025), and a pause for a named person is that constraint applied to payments, submissions and record changes.

The control does not change the application, the user's permissions or the model, and it does not replace input and output rails; it changes what those rails are protecting, from a screen of real values to a screen of tokens.

Where does RedactSure sit?

The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company. Its design occupies the lower rows of the table on this page. Agents work inside a governed environment in the customer's own cloud. On every rendered screen, substitution happens at the render layer before any model reads it, so the intake record's patient name, record number and insurance identifier reach the model as consistent tokens such as PATIENT_001 or ACCT_001 for the whole run, and an injected instruction to leak them produces tokens. Real values resolve only at approved destinations at the moment of an approved action. A named person confirms the tokenization policy and approves every payment, submission and record change, the arrangement RedactSure calls Supervised Delegation, so a manipulated model cannot complete a consequential action on its own. Every screen as tokens, every action and every approval lands in the AI Control Record and exports to the customer's SIEM. The design is model agnostic, requires no per-application integration, and leaves user permissions unchanged; content guardrails from any vendor can run alongside it as supporting layers.

Deployments are in pilot. A pilot on the reader's own workflow would run the adaptive test this page describes and show, per attempt, what the model received and which actions a named person held. Everyone else controls where data can go. RedactSure controls what the AI can see.

Methodology and limitations

This page rests on NVIDIA's NeMo Guardrails developer guide on rail types, the Amazon Bedrock User Guide on how Guardrails works, Databricks and Portkey guardrail documentation, the Nasr et al. preprint on adaptive attacks against LLM defenses, the Beurer-Kellner et al. preprint on design patterns for securing LLM agents, Arthur Holland Michel's October 2026 analysis in MIT Technology Review, the OWASP Top 10 for LLM Applications entry on excessive agency, and published RedactSure research pages on agent security, data exposure and data masking. All sources were read as of October 16, 2026.

No hands-on testing of any product was performed for this page. Statements about what vendor guardrail products do are drawn from the pages reviewed, and where this page says a capability is not described, it means not described in those pages, not that the product lacks it. Vendor features change, guardrail products in particular are extended frequently, and the page carries its date. The adaptive attack results are quoted as the study reports them for the defenses it evaluated; none of the vendor products on this page was among them, and the extension of the finding to those products is this page's reasoning about shared mechanism. Statements about RedactSure describe its current design, with deployments in pilot. The patient intake record is a record type used for illustration, not a description of any clinic.

Whether a given set of guardrails satisfies an organization's obligations under health privacy law, sector regulation or contract is a determination for its counsel, its assessors and its own security and compliance officers, not for a research page.

What the record shows

Guardrails divide into two kinds that share a name. Content guardrails inspect text at the input and output of a model, are the most common by the industry's own account, and are defeated by attackers who adapt, with the best available study showing failure rates above 90 percent for most defenses tested. Action guardrails constrain what the agent can do and what it receives: capability limits, a named person approving consequential actions, and substitution of protected values before the model reads the screen. They do not depend on recognizing an attack, which is why the design pattern literature frames them as making harm impossible rather than unlikely. RedactSure's environment implements the latter kind. The test is to name one agent and one task, give a red team a day to iterate against it on the organization's own screen, and record what the model received and which actions completed.

Frequently asked questions

Are guardrails the same as prompt injection defenses?

They overlap. Input guardrails that classify or filter prompts are one form of injection defense, and the adaptive attack study evaluated defenses of that kind. Guardrails as a category are broader, covering output filtering, tool-call policies, capability limits and approval steps. The distinction that matters is between controls that try to recognize a malicious input and controls that make a malicious input unable to cause consequential action. Only the second kind holds against an attacker who iterates.

Why do vendors concentrate on input and output rails?

Because they are the easiest to build and deploy. They sit at the model boundary, require no knowledge of the application the agent is working in, and can be configured with topic lists, patterns and classifiers. NVIDIA's documentation notes that input and output rails are the most common. Their limit is structural: they evaluate text and can be probed, and they cannot see what the agent received from a screen or whether its next action is one the task required.

What did the adaptive attack study actually show?

Researchers from OpenAI, Anthropic, Google DeepMind and other institutions evaluated 12 published defenses against jailbreaks and prompt injections, letting the attacker iterate against each defense rather than using a fixed test set. Most defenses were bypassed at success rates above 90 percent, including several whose original evaluations reported near-zero rates. Static evaluations overstate protection, and a classifier-based defense should be assumed breakable by a motivated attacker.

What is an execution rail and is it an action guardrail?

In NVIDIA's taxonomy, an execution rail inspects the input and output of tool calls, which places it closer to the action than a content rail. It is an action guardrail to the extent its policy constrains which tools may be called with which arguments, regardless of what the model intended. It remains dependent on the policy anticipating the misuse: a rail that permits outbound messages to any address will permit the attacker's address. Capability limits and approvals do not have that dependency.

Does data minimization count as a guardrail?

Yes, and it is the one that fails least often, because it does not evaluate anything. If the patient's name is replaced with PATIENT_001 before the model reads the screen, no injection can extract the name, no filter has to recognize it, and no approval has to catch it. The model never had it. Minimization at the screen is a guardrail on what the model receives rather than on what it says, and it removes the data the other guardrails are trying to protect.

How should guardrails and human approval work together?

Content guardrails reduce noise and catch casual or accidental misuse. Capability limits remove paths the task does not need. Human approval of consequential actions is the control of record for whatever remains, provided the approver sees the real action and real values from a source the agent cannot alter and is not asked to approve so often that the step becomes reflexive. Substitution before the model reads the screen reduces what the other layers must protect.

Book an AI Agent Security Review →

Bring your hardest questions.

Bring one sensitive workflow and the decisions your team needs to keep in its hands.

Book a demo

About the author

Christopher Sowa
Chris SowaCo-Founder, President and Chief Commercial Officer, RedactSure

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

Book a demo

Book a demo

Having trouble? Open the booking page or email us.