Skip to content
redactsure
Book a demo

Explore.

Guide · By Chris Sowa · Published
Last updated

How to Maintain Audit Trails for AI Agent Activity

Keep the trail in your own SIEM as tokens, answering what the model received, what the agent did, who approved and when, not on a vendor's 7-day setting.

Document and data tasks pass two checkpoints while a server prints the activity record.

62% experimenting · 23% scaling · 4 questions

McKinsey's State of AI 2026 found 62% of organizations experimenting with AI agents and 23% scaling them anywhere (CX Today, 2026). The 23% are the ones an examiner will visit, with four questions about a run from two years ago. What did the model receive. What did the agent do. Who approved it. When.

You probably plan to answer with the vendor's activity view. It answers the second question, inside the vendor's app, on that app's retention. Or with session replay in your own bucket, which answers the first question by keeping a picture of the patient chart, so the trail becomes a second copy of the records. Or with the cloud's API log, which records the call and not the page.

The Domesday survey of 1086 outlived every clerk who wrote it because the record was kept apart from the estates it described. An agent's trail needs the same separation: written by the environment, not the agent; holding tokens, not the data; exported to the SIEM you already search.

A trail that holds tokens and names in your own SIEM answers all four for years without becoming a store of SSNs and card numbers. No vendor log is described as doing that.

The examiner asks 4 questions and the vendor activity view answers 1

Copilot Studio describes "session replay with screenshots, timestamped action logs, run summaries, and resource tracking" (Microsoft, 2026). AgentCore Browser records DOM changes, user actions, console logs and network events to your own S3 bucket, with the API calls in CloudTrail (AWS, 2026), so that recording sits on your retention, not the vendor's. Dots offer an Activity View to follow, redirect or stop work (OpenAI, 2026).

Each answers "what did the agent do." None of the three names an approver as a field, and none describes what the model received apart from the screenshot or DOM capture.

Property Vendor activity view Recording in your own bucket Governed environment record
What the model received Not described The screenshot or DOM capture The task's fields as tokens
What the agent did Yes Yes Yes, with target and timestamp
Who approved, by name Not described Not described A field on every consequential action
Holds regulated data In the vendor's store Yes, in your bucket, as pixels or DOM No, identifiers are tokens
Retention set by The tenant or the vendor You Your SIEM policy

A vendor activity view that reads "Not described" on two of four questions needs the vendor in the room with the examiner. A recording in your own bucket keeps the vendor out and leaves the approver question open.

Copilot Studio documents replay retention from 7 days to indefinite, set per tenant

7 days minimum · indefinite maximum · 1 verbosity setting

Copilot Studio's replay retention runs from 7 days to indefinite, and both retention and a verbosity setting that can omit screenshots are configurable by the tenant (Microsoft, 2026). The default and the setting are per-tenant choices; the regime governing your records sets neither until someone maps one to the other.

The verbosity setting is the trap. Leave screenshots on and the replay holds every patient chart, claim file and cardholder page the agent read. Turn them off and the record no longer holds what the model received. You are choosing between a store of regulated records and a trail with a hole in it.

Gartner found in May 2026 that uniform governance across AI agents leads to enterprise agent failure (Gartner, 2026). One tenant-wide retention setting for every workflow is uniform governance. A claims agent, a student records agent and a payments agent fall under three regimes, so the trail carries a retention tag per regime, applied in your SIEM.

A screenshot of a patient chart is the chart under 45 CFR 164.502(b)

The HIPAA minimum necessary standard limits protected health information to the amount the purpose requires (45 CFR 164.502(b), 2026). A replay holding the chart as pixels is a second disclosure. FERPA's school official exception requires direct institutional control over education records (34 CFR 99.31, 2026); a replay on the vendor's retention is not under your control. PCI scoping guidance places any system storing cardholder data in scope (PCI Security Standards Council, 2026), replay included.

Change what the model receives and the trail inherits the change. Render-layer tokenization is replacing sensitive values with consistent stand-in tokens at the point where a screen is rendered, before any AI model reads it, so the model works with SSN_001 and ACCT_001 and real values resolve only at approved destinations at the moment of action. Where the model saw PATIENT_001 and CARD_001, the record holds PATIENT_001 and CARD_001. Amounts, dates and codes stay in clear.

This is Least Exposure, the principle that for each piece of work, the AI receives exactly the data the task requires and nothing more, enforced before any model reads the screen. The record is complete as evidence and empty as a store of data.

Microsoft's AI Red Team and Anthropic documented agents that diverge, so the agent cannot hold the pen

Greshake et al. showed in 2023 that an LLM-integrated application can be compromised by instructions hidden in content it observes (Greshake et al., 2023). Microsoft's AI Red Team catalogs agent failure modes that diverge from instructions (Microsoft AI Red Team, 2025). Anthropic found agents choosing insider-threat behavior under some conditions (Anthropic, 2025).

A trail the agent can reach is a trail the same failure can edit. The hijacked agent's wrong proposal must land in the record like any other action, and the agent must have no path to the store.

So the environment writes the record. The AI Control Record is the record of every screen the agent saw as tokens, every action, every approval with a name, exported to the customer's own monitoring. It holds every screen as the task's fields with identifiers as tokens, every action with target and timestamp, every approval with the name and the screen the person saw, and the model used.

The agent is a tethered agent, an agent that works only inside the governed environment a named person connected it to, on the applications that person chose. It sees the screen; the environment sees the model and writes the record.

Supervised Delegation puts a name on every approval, which the cloud's API log does not

Supervised Delegation means a named person grants the access, chooses the applications, confirms what is tokenized, can watch and pause the run, and approves every consequential action, on the record. The approval is a field: name, action, timestamp and the tokenized screen the person saw. Consequential means every action that pays, submits, changes a record, sends a message, creates a key or opens a port. The per-task policy the person confirmed is the first line of the record.

The cloud's own logs, CloudTrail on AWS or the Azure activity log, record the API calls the account issued, not the page the agent was reading, what the model received, or which person said yes. OMB M-24-10 asks agencies for human oversight and monitoring of rights-impacting and safety-impacting AI (Office of Management and Budget, 2024). A log that holds the name is oversight you can show.

The record keeps its format when the model changes, under separation of model and control, the principle that the environment deciding what an AI agent sees and who approves what it does is owned by the customer and independent of whichever model performs the work; the vendor that is the model should not also be the judge of the model.

HIPAA, FERPA, PCI, GLBA, the Privacy Act and OMB M-24-10 ask the same 4 questions

Each asks for evidence of what happened to a record and who allowed it. The GLBA Safeguards Rule requires oversight of service providers (16 CFR Part 314, 2026). The Privacy Act binds government contractors handling systems of records (5 U.S.C. 552a, 2026). The NAIC Model Bulletin, adopted by 24 states as of March 2025, asks insurers to document AI systems (Quarles & Brady, 2025).

Regime Vendor-held store Recording in your own bucket Tokenized record in your SIEM
HIPAA minimum necessary Disclosure to the vendor's store Second copy of PHI Tokens, no PHI stored
FERPA direct control Not under your control Yours, holds the records Yours, holds tokens
PCI scope Vendor store in scope Replay store in scope Tokens out of scope
OMB M-24-10 oversight evidence No approver field described No approver field Name on every consequential action

Retention follows the regime, and counsel sets the period. Three conditions should never occur: a real identifier on a screen the model received, a consequential action with no named approval, and an agent acting after a person paused it or withdrew its access. Each has an alert that must fire when tested. The trail is that evidence, or there is none.

6 Rules for maintaining an agent audit trail

Export the vendor logs into the SIEM today and label them a stopgap

Route every vendor log into the SIEM under your retention. It is evidence of what the agent did, not the trail, because it holds neither what the model received nor who approved. Test: Search the SIEM for yesterday's agent runs; each carries a retention tag you set.

Let the environment hold the pen, not the agent

The environment writes and exports the record. The agent has no credentials that reach it. Before: The agent framework appends its own actions to a log file in its working directory. After: The environment writes every screen, action and approval to a store the agent cannot open.

Store what the model received as tokens, never as pixels

The record holds the task's fields with identifiers as tokens, PATIENT_001, CARD_001, STUDENT_001, as the model received them.

Stored as What the examiner gets
Screenshot The chart, and a second store of PHI to secure
Tokenized fields Complete evidence of exposure, zero regulated identifiers

Make every consequential approval a field with a name

Each payment, submission, record change, message, key or port carries the approver's name, the timestamp and the screen they saw. Not the account. The person. Test: Pick one payment from last week; the record returns who approved it and what they saw, without asking the vendor.

Set retention by regime, not by the console default

A claims run, a student records run and a cardholder run carry three tags. The SIEM applies them. Counsel sets the periods. Before: Retention is whatever the console default was the day the agent was enabled. After: Each run carries a regime tag, and the SIEM's policy for that tag governs it.

Alert on the 3 conditions that should never occur

A real identifier on a screen the model received. A consequential action with no named approval. An agent acting after a pause or withdrawal of access. Each is a SIEM alert with an owner. Test: Produce each condition in a test run; all three alerts fire to the on-call owner.

How to choose where the trail lives

  • If the vendor's activity view is your only trail, export it into the SIEM this week as evidence of actions only.
  • If your recording bucket holds screenshots of regulated records, count it as a system holding those records and replace it with a tokenized record.
  • If the agent's credentials can reach the record store, move the record to a store the environment writes and the agent cannot open.
  • If your runs touch three or more regimes, tag each run by regime and let the SIEM apply retention per tag.
  • If your cloud log is the trail, keep it and add the record beside it; the API call has no page and no name.
  • If you plan to swap models in the next year, insist that the record's format belongs to your environment, not the vendor.

The checklist

Check Pass Fail
Open one screen from one run Identifiers appear as tokens such as PATIENT_001 A real SSN or card number appears
Replay the actions of one run Every action has a target and timestamp in order Gaps, or actions without targets
Pick one consequential action Approver's name and the tokenized screen are fields The account name, or no approver
Open the record store with the agent's credentials Access denied It opens
Find the run in the SIEM Present, with a regime tag Only in the vendor's console
Ask for a run from a year ago The SIEM returns tokens, action, name and timestamp You have to ask the vendor
Produce the three conditions in a test All three alerts fire to a named owner Any one is silent
Read the first line of a run's record The policy with the named person's confirmation The first action

Questions

How long should agent audit trails be retained?

As long as the records the agent touched are retained under the regime that governs them, and never shorter than your own log retention. Counsel sets the periods per regime. A trail with identifiers as tokens makes long retention an ordinary logging cost. A screenshot replay makes long retention a growing store of records, with every access rule for the records applying to it.

Can the agent tamper with its own audit trail?

Not when the environment writes it and the agent has no path to it. The agent sees the screen; the environment sees the model and writes the record. An agent hijacked by content it read, the failure Greshake et al. described in 2023, can propose a wrong action, and the proposal lands in the record like any other. A trail the agent's credentials reach is one the hijack can edit.

Does CloudTrail or the Azure activity log count as the agent's audit trail?

As part of it. Those logs record the API calls the account issued, which answers what happened to the account. They do not record what the model received, which page the agent was reading, or who approved. The governed environment's record supplies those three, exported to the same SIEM, so the examiner reads the screen and approval beside the cloud's call.

Should the trail include screenshots?

No, when the screenshots contain the records. A trail that stores the chart, the claim or the cardholder page is a second copy of regulated data, and every retention and access rule for the records applies to it. The trail holds what the model received in the form it received it: the task's fields with identifiers as tokens, which is evidence without a second store.

What if the vendor's logs are the only trail we have?

Then you have evidence of what the agent did, inside the vendor's app, on that app's retention schedule. You have no evidence of what the model received or who approved, and you cannot produce the trail without the vendor. Export what the vendor offers into the SIEM now and move the agent into an environment that writes the record itself.

Who reviews the trail and how often?

The security team on the SIEM's cadence, with alerts for the three conditions that should never occur. The named person who owns a workflow reviews their own runs before extending the agent's autonomy. Compliance reviews on the regime's calendar. A quarterly read of a sample of runs is a reasonable floor, with the review dates on a calendar and named owners.

Does the trail change what users are permitted to do?

No. Permissions stay as they are; the account can do what it could do before. What changes is what the model can see on each screen, set by the per-task policy and confirmed by a named person, and the trail records that exposure. The record is a side effect of controlling what the AI sees, not a new permission layer over your applications.

Methodology

What this page rests on. Vendor documentation for Copilot Studio computer use, AgentCore Browser and OpenAI Dots, read as of October 9, 2026; the regulatory texts cited for HIPAA, FERPA, PCI DSS, GLBA, the Privacy Act and OMB M-24-10; and the research works in Sources.

What was not tested. No hands-on testing of the named products was performed. Retention ranges, verbosity settings and log contents are quoted from the vendors' published descriptions, which change with releases, so the page carries its date.

Whose determination it is. Whether a trail satisfies a regime is for the organization's counsel, its assessor and its security and compliance officers to determine. The checks on this page start that review.

How RedactSure does this

4 questions · 1 named person · 0 real identifiers in the record

RedactSure runs AI co-workers inside a governed environment: a customer-owned secure virtual machine on hardware-encrypted enclaves, with keys the customer holds. A render layer tokenizes every identifier the policy names before any model reads the screen; the model receives the task's fields with identifiers as tokens. The Planner writes the per-task policy; a named person confirms it and approves every payment, submission and record change while the run proceeds, under Supervised Delegation.

The environment, not the agent, writes the AI Control Record: every screen as tokens, every action with target and timestamp, every approval with the name and the screen the person saw, and the model used. The agent has no path to it. The record exports continuously to the customer's own SIEM, where retention follows the customer's policy per regime and the three alerts run beside the rest of the monitoring. The environment is model-agnostic; a change of model leaves the record's format untouched. Permissions do not change; what changes is what the AI can see, aligned with the minimum necessary standard. Deployments are in pilot.

Book an AI Agent Security Review →

Bring your hardest questions.

Bring one sensitive workflow and the decisions your team needs to keep in its hands.

Book a demo

About the author

Christopher Sowa
Chris SowaCo-Founder, President and Chief Commercial Officer, RedactSure

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

Book a demo

Book a demo

Having trouble? Open the booking page or email us.