Skip to content
redactsure
Book a demo

Explore.

Data Report · By Chris Sowa · Published
Last updated

Who Is Liable When an AI Agent Makes a Mistake?

The organization that deploys the agent is liable, because a tribunal has already refused to treat a chatbot as a separate legal actor and the new EU product liability regime treats AI software as a product. The defensible position is a record showing which named person approved which action, so check whether your agent deployment can produce one.

A chain of workflow objects tips toward a broken component beside courthouse steps and an accountability record.

The organization that deploys the agent is liable, because a tribunal has already refused to treat a chatbot as a separate legal actor and the new EU product liability regime treats AI software as a product. The defensible position is a record showing which named person approved which action, so check whether your agent deployment can produce one. The question sounds like it belongs to lawyers alone, but it is decided in practice by architecture: whether an action an agent took can be traced to an instruction, an approval and a person, or whether the organization is left arguing that its own software acted on its own. This page sets out what the one decided case and the two European instruments say about where liability lands, why a vendor contract does not move it, what the regulators who supervise insurers and financial firms expect, and what an agent deployment needs to produce when something goes wrong. It treats agent data security and action approval as the two halves of a liability answer, because a mistake usually involves either data the agent should not have seen or an action it should not have taken. It closes with the test a buyer should run before trusting any claim that liability has been handled.

Key findings

  • The one tribunal decision that has squarely addressed an AI assistant's error placed responsibility on the company that deployed it, with the tribunal stating that "It should be obvious to Air Canada that it is responsible for all the information on its website" (McCarthy Tetrault, 2024).
  • The revised EU Product Liability Directive "explicitly recognizes AI software as a 'product' subject to liability," with member states required to transpose it by December 9, 2026 (Hogan Lovells, 2024).
  • The EU AI Act's human oversight article requires that overseers be able to "disregard, override or reverse the output" and "interrupt the system through a 'stop' button or a similar procedure" (EU AI Act Article 14, 2024).
  • OpenAI's governance paper proposes that "at least one human entity is accountable for every uncompensated direct harm," and describes human approval as "a standard way to limit egregious failures" (OpenAI, 2023).
  • Payment infrastructure is beginning to encode intent as evidence: Google's Agent Payments Protocol describes mandates as "cryptographically-signed digital contracts that serve as verifiable proof of a user's instructions" (Google Cloud, 2025).
  • Liability analysis depends on what the agent could see as much as what it did, since a mistake that exposes a record is a data incident and an approval record is only useful if it shows the exposure did not happen (What Is AI Agent Data Exposure?).

Who has a court actually held responsible for an AI agent's mistake?

The decided record is thin but unambiguous. In Moffatt v. Air Canada, a customer relied on a chatbot's statement about bereavement fares, the airline argued that the chatbot was a separate entity responsible for its own actions, and the British Columbia Civil Resolution Tribunal rejected that argument. As the McCarthy Tetrault analysis of the decision records, the tribunal stated that "It should be obvious to Air Canada that it is responsible for all the information on its website" (McCarthy Tetrault, 2024). The amount at stake was small. The principle was not: the deploying organization answers for what its software tells and does to the people it deals with.

Two features of that case carry over to agents that act rather than merely answer. First, the tribunal treated the chatbot's output as the company's representation, in the same way it would treat a static page or a human agent's statement. An agent that changes a record, submits a form or issues a payment is doing something a court will find even easier to attribute to the deployer, because the action lands in the company's own systems under the company's own credentials. Second, the defense that failed was the only defense available to a company with no record of oversight. Air Canada could not point to a person who had reviewed the statement, a policy that governed what the chatbot could assert, or a log showing what it had been given to work from. The argument that "the software did it" was all that was left, and it did not survive.

The lesson for an organization running agents today is to assume that the Moffatt reasoning will be applied to actions, and to build so that a different defense exists: not that the agent acted alone, but that a named person approved the specific action with a specific view of the record, and that the organization can show both.

What do the EU's new liability and oversight rules change?

Two European instruments move liability from a question courts settle case by case to one written into statute. The revised Product Liability Directive, Directive (EU) 2024/2853, "explicitly recognizes AI software as a 'product' subject to liability," and member states must transpose it by December 9, 2026 (Hogan Lovells, 2024). A defect in an AI system that causes damage is treated the way a defect in a physical component is treated, which changes the exposure of both the vendor and the organization that integrated the software into a workflow.

The EU AI Act approaches the same problem from the oversight side. Article 14 requires that high-risk AI systems be designed so that natural persons can oversee them, and it specifies capabilities those persons must have: the ability to "disregard, override or reverse the output" of the system and to "interrupt the system through a 'stop' button or a similar procedure" (EU AI Act Article 14, 2024). These are architectural requirements. A deployment in which a person can only observe what the agent did after the fact does not satisfy them.

Instrument What it establishes, per the sources reviewed What it means for an agent deployment
Moffatt v. Air Canada (2024 BCCRT 149) Deployer responsible for its AI assistant's statements Expect the same attribution for agent actions
Product Liability Directive (EU) 2024/2853 AI software is a product; transposition by December 9, 2026 Defects in agent software carry product liability
EU AI Act Article 14 Overseers must be able to override, reverse and stop Oversight must be a real capability with a named person

Read together, the instruments point in one direction: the deploying organization must show a person could have stopped the agent and, when it mattered, did.

Does a vendor's contract move liability off the deploying organization?

Not toward the people harmed. A vendor's terms can allocate loss between the vendor and the deployer, but a customer or regulator dealing with the deploying organization has no contract with the model provider and no reason to look past the company whose name is on the account. The Moffatt tribunal's reasoning did not ask who built the chatbot. Under the Product Liability Directive, the vendor's exposure increases because its software is now a product, but that is an additional defendant, not a substitute one.

There is also a practical limit to what any contract can recover. OpenAI's governance paper proposes a norm that "at least one human entity is accountable for every uncompensated direct harm" (OpenAI, 2023). The paper does not contemplate the agent itself as that entity. A deployer that cannot identify the person who authorized a given action has, in effect, volunteered to be the accountable entity for everything the agent does, with nothing to show about why any particular action was taken.

Consider a claims screen in an insurance system. An agent reviewing a claim reads the policyholder's identifiers, the loss description and the payment details, then issues a settlement that turns out to be wrong. If the insurer's position is that the model vendor is responsible, the regulator will ask what instruction the agent was given, what it saw on the screen, and who approved the payment. A contract with the vendor answers none of those questions. A record of the instruction, the tokenized screen, the approval and the approver answers all of them, and what the vendor owes becomes a separate commercial matter rather than the insurer's only defense (How to secure AI agents?).

What do regulators expect from organizations that deploy agents?

Sector regulators have not waited for liability law to settle. In the United States, the NAIC's Model Bulletin on the Use of Artificial Intelligence Systems by Insurers has been adopted across a growing set of states, as the NAIC's own adoption map records (NAIC, 2025). The bulletin's premise is that an insurer remains responsible for decisions made with AI systems, including those sourced from third parties, and must have a governance program that documents how those systems are controlled.

OpenAI's governance paper adds a design corollary that matters for any regulator's after-the-fact review: "Actions that can only be reviewed after the fact should be more easily reversible," and users should be able to "activate a graceful shutdown procedure for its agent at any time" (OpenAI, 2023). Reversibility and shutdown are the controls a regulator will ask about, and their absence will read as a failure to govern.

Columbia's DAPLab frames its agenda around infrastructure for trustworthy agent automation, treating accountability as something the surrounding system must supply rather than something the model can be trusted to carry (Columbia DAPLab, 2026). MIT Sloan's explainer on agentic AI likewise describes agents as systems that plan and act toward goals with limited supervision, the property that makes supervision a design requirement (MIT Sloan, 2026).

What record does an organization need when an agent's action is challenged?

A challenged action produces a short list of questions, and the record has to answer each one. What was the agent instructed to do? What did it see when it decided? Which action did it take, against which record, at which destination? Who approved that action, and what did the approver see? Could the action have been stopped, and by whom? An organization that can answer those questions with contemporaneous evidence can show that a person, not the software, made the consequential decision.

Payments infrastructure is moving toward producing that kind of evidence by design. Google's Agent Payments Protocol describes mandates as "cryptographically-signed digital contracts that serve as verifiable proof of a user's instructions," and names among the problems it addresses "Determining accountability if a fraudulent or incorrect transaction occurs" (Google Cloud, 2025). The idea generalizes beyond payments: an approval should be captured as an object that binds a person, an instruction and an action together.

The record also has to show what the agent did not see. If the agent's mistake is challenged as a privacy incident rather than a transaction error, the question becomes whether the model ever held the claimant's identifiers, account numbers or health details. A record of screens the agent read, with sensitive values shown as consistent tokens rather than real data, answers that question in the deployer's favor in a way no policy document can (What Is Sensitive Data Masking for AI Agents?). Evidence of absence is the harder kind to produce after the fact, which is why it has to be generated at the time.

What does a governed environment change?

In a governed environment, the deploying organization's liability does not move, but its defense does: it can produce a record of which named person approved which action, and of what the model could and could not see when the action was taken.

This class of control works at the rendered screen. Sensitive values on each screen are replaced with consistent tokens before any model reads them, so the task runs on tokens. Real values resolve only at an approved destination at the moment of an approved action. A named person confirms what is tokenized, and a named person approves consequential actions such as payments, submissions and record changes before they execute. Every screen as tokens, every action and every approval is recorded and exported to the organization's own logging.

For the liability question, that record addresses both halves of the analysis. On the action, it shows oversight that could prevent the action rather than review it afterward, which is the capacity to "disregard, override or reverse the output" that the EU oversight article requires (EU AI Act Article 14, 2024), and the human approval that OpenAI's governance paper describes as a standard way to limit egregious failures (OpenAI, 2023). On exposure, it shows which values were available to the model on each screen, so a mistake that would otherwise also be a data incident can be shown not to have exposed the record. The approval binds a person to an action on a record before execution, the first of the three things a buyer should ask to see.

The environment does not alter the application, the user's permissions or the model, and it does not supply the name of the person who confirms the policy; that name still comes from the deployer.

Where does RedactSure sit?

The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company. The design answers the liability question by producing the evidence that a challenged action demands. The agent works inside a governed environment in the customer's cloud. Sensitive values are substituted at the render layer before any model reads the screen, so the model sees consistent tokens such as ACCT_001 or PATIENT_001 for the whole run, and real values resolve only at approved destinations at the moment of an approved action. A named person confirms the tokenization policy and approves every payment, submission and record change, which RedactSure calls Supervised Delegation, so there is always a human entity attached to each consequential act. Every screen as tokens, every action and every approval lands in the AI Control Record and exports to the customer's SIEM. The design is model agnostic, requires no per-application integration and leaves user permissions unchanged.

Deployments are in pilot. A pilot on the reader's own workflow would show, for one claims screen or customer account, the instruction the agent received, the tokenized screen it read, the action it proposed, the person who approved it and the moment the real value was released. That is the record Air Canada did not have. Everyone else controls where data can go. RedactSure controls what the AI can see.

Methodology and limitations

This page rests on a small set of primary and secondary sources: a law firm analysis of the one tribunal decision to address an AI assistant's error directly (Moffatt v. Air Canada, 2024 BCCRT 149), a law firm analysis of Directive (EU) 2024/2853, the text of Article 14 of Regulation (EU) 2024/1689, the NAIC's adoption map for its model bulletin on insurers' use of AI, OpenAI's whitepaper on practices for governing agentic systems, Google Cloud's announcement of the Agent Payments Protocol, and explanatory material from Columbia's DAPLab and MIT Sloan. All sources were read as of October 16, 2026. Court and regulatory materials were read through the analyses cited where the primary text was not in the sources reviewed.

No hands-on testing of any product was performed. Where this page says a capability or position is "not described," it means not described in the pages reviewed, not that the capability is absent. Legal instruments are summarized, not quoted in full, and transposition and enforcement of the EU directives will vary by member state; the page carries its date. Statements about RedactSure describe its current design, with deployments in pilot, and are confined to the section on where the company sits. The claims screen and customer account examples describe record types, not any organization; no customer, prospect or litigant beyond the published Moffatt decision is described.

Whether a given deployment's oversight arrangements, approval records or data handling satisfy a liability standard, a sector regulator or a contractual obligation is a determination for the organization's counsel, its assessors and its own security and compliance officers, not for a research page.

What the record shows

Liability for an AI agent's mistake rests with the organization that deployed it. The single decided case rejected the argument that the software was a separate actor, the EU has made AI software a product for liability purposes, and the EU AI Act requires that a person be able to override, reverse and stop a high-risk system. None of those sources contemplates the agent as the accountable party. The practical consequence is that the defensible deployment is the one that can show, for any challenged action, which named person approved it, what that person and the model each saw, and that the sensitive values on the screen were never in the model's hands. A record of that kind, produced at the time of the action rather than reconstructed later, is the difference between a governance program and a hope. The test is to name one agent and one task and ask: if this action is challenged next month, can we produce the approver's name, the approval, and the screen as the model saw it, today?

Frequently asked questions

Can an organization argue that the AI agent, not the company, made the mistake?

The one tribunal to consider that argument rejected it. In Moffatt v. Air Canada the airline contended that its chatbot was a separate entity responsible for its own actions, and the tribunal found the company responsible for all the information on its website. An agent that takes actions inside a company's own systems under the company's credentials is an even weaker candidate for separate treatment. Organizations should plan on being treated as the actor and build records that show a person authorized each consequential step.

Does the EU Product Liability Directive apply to AI agents used internally?

The directive treats AI software as a product subject to liability, and member states must transpose it by December 9, 2026. Its reach into a specific internal deployment depends on who was harmed, what damage occurred and how national law implements it, which are questions for counsel. What the directive settles is that software is no longer outside the product liability regime, so both the vendor of an agent and the organization that integrates it should expect defects to carry consequences.

Is an audit log enough to demonstrate oversight?

A log shows what the agent did. Oversight is shown by evidence that a person could have prevented the action and, for consequential actions, that a person approved it with a view of the relevant record. The EU AI Act's oversight requirement speaks of the ability to override, reverse and stop, not to read a log afterward. An organization should be able to produce, for any action, the approver, the approval and what the approver saw, alongside the log of the action itself.

What does human approval need to look like to matter legally?

The sources reviewed point to approval that is specific and prior. Google's Agent Payments Protocol describes mandates as signed contracts that serve as verifiable proof of a user's instructions, and OpenAI's governance paper describes human approval as the standard way to limit egregious failures. Approval that binds a named person to a specific action on a specific record, captured before execution, is the form most likely to be useful when the action is later questioned.

Does using a well known model vendor reduce the deploying company's exposure?

Not toward customers or regulators, who deal with the deploying organization. A vendor agreement allocates loss between the two companies and may provide indemnities, and under the Product Liability Directive the vendor's own exposure increases, but that adds a defendant rather than replacing one. The deployer's best protection remains its own record: the instruction given, the screen the model saw, the action taken and the person who approved it.

How does data exposure change the liability picture?

A mistake that reveals personal, financial or health information is a data incident as well as an operational error, and the question becomes whether the model ever held the real values. A deployment that substitutes consistent tokens for sensitive fields before the model reads the screen, and keeps a record of each screen as the model saw it, can show that the exposure did not occur. That evidence of absence is hard to produce later and has to be generated at the time of the action.

Book an AI Agent Security Review →

Bring your hardest questions.

Bring one sensitive workflow and the decisions your team needs to keep in its hands.

Book a demo

About the author

Christopher Sowa
Chris SowaCo-Founder, President and Chief Commercial Officer, RedactSure

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

Book a demo

Book a demo

Having trouble? Open the booking page or email us.