Data Report · RedactSure Research
Chris Sowa · Founder, RedactSure
Published
Last updated
What Does Claude Computer Use See in a Screenshot?
Everything the page draws inside the captured area: the fields the task uses, the ones it does not, sidebars, notifications, and any instruction a site paints into the image. Check what is on the screen before each capture, because the developer sets that frame and the model reads all of it.
A customer account page is open, and an agent is halfway through updating a renewal date. To act, the computer use tool sends Claude a screenshot and lets it move the mouse and type. The image holds the renewal date and the account name, and it also holds the billing panel, the contact's email address, the open tickets, the navigation sidebar and a failed-payment alert. The model reads all of it, because a screenshot is a picture of the rendered page and nothing in the tool separates the fields the task needs from the fields it does not.
The decision sits somewhere specific. Anthropic's documentation describes screenshot capabilities and mouse and keyboard control, and warns that Claude sometimes follows commands found in content even when they conflict with the developer's instructions (Anthropic, Computer use, 2026). The developer decides which page, window and account are open at the moment of capture, and that choice is the filter. A review has to establish the screen at capture: who chose it, what the page drew, and what happened after the model read it.
Key findings
- For the screenshot to keep sensitive data out of the model's context, the values on the rendered page would have to be replaced with consistent tokens such as ACCT_001 before the capture, with the real values resolving only at the approved destination (What Is Render-Layer Tokenization?).
- It would also require a named person approving each submission or record change, and a record of what the screenshot contained and who approved (What Is an AI Control Record?).
- Anthropic's computer use documentation does not state that a page's values are replaced before the model reads it, and warns that Claude follows commands found in content (Anthropic, Computer use, 2026).
- The same documentation describes the tool as one that "provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction," which is how a page becomes model input (Anthropic, Computer use, 2026).
- A 2026 evaluation of screenshot-driven GUI agents finds they "may inadvertently expose sensitive information such as identities, accounts, locations, and behavioral traces" (Wang et al., 2026).
- Brave's researchers showed that an attacker can "hide prompt injection instructions in images using a faint light blue text" that screenshot-reading agents act on, so the image is an input channel (Sahib and Chaikin, 2025).
What does a screenshot of a CRM page put in front of the model?
The tool's input is an image of the rendered page, not a query result. A customer account page captured mid-task shows the record header with the account name, owner, renewal date and contract value; a contact block with a name, email address and job title; an activity feed with internal notes; a billing panel with the last invoice and the card on file; a failed-payment notification; and a navigation sidebar listing other accounts. The renewal date is the only field the task needs. The model receives all of it, because a screenshot carries what the browser paints rather than the fields an integration chooses.
A model reading that image can quote a value from a panel the task never touched, and repeat it later in a summary, a note or a tool call. A chart or a scanned attachment rendered as an image is inside the capture too. The general case is set out in (What Does an AI Agent See in a Screenshot?); what changes here is who sets the frame: the developer opens a page rather than choosing a data field.
The image also arrives without the page's context. A person knows which panel is billing; the model sees pixels, and a value in a corner is as legible as the one in the field it must edit. That is why the whole viewport matters, including the parts nobody would put in an API.
What does a screenshot carry that access controls never read?
Access control decides which records a user may open. It does not decide which fields the model reads once a record is open. A school information system shows the pattern: Infinite Campus tells administrators to "Review User Groups and Suggested Tool Rights as a starting point for assigning tools and rights to user groups," and its SSN panel displays values as XXX-XX-XXXX until a user is given SSN rights (Infinite Campus, 2026). Masking is therefore per field and per role, and the screenshot carries whatever the role leaves visible. If the account the agent signs in as can see the full value, the capture carries the full value. The masking belongs to the application and ends where the role's rights end.
The image also carries what the page paints, not what its markup says. Text rendered into a chart or attachment is inside the screenshot even when no field holds it, and a filter that scans the page's text does not read those pixels. Brave's researchers built an attack on that gap, hiding instructions in image text that screenshot-reading agents follow (Can Hidden Text on a Web Page Hijack a Browser Agent?). Images built to carry an instruction averaged a 91.9% success rate across 2,250 web tasks in one 2026 evaluation and stayed effective against three agent-level defenses (How Often Do AI Agents Follow Injected Instructions?). A control that removes a value from the pixels before the model reads them is not described in the pages reviewed.
What do the vendors themselves say?
Anthropic's computer use page describes the tool: "Claude can interact with computer environments through the computer use tool, which provides screenshot capabilities and mouse/keyboard control for autonomous desktop interaction." The same page warns: "In some circumstances, Claude follows commands found in content even when they conflict with your instructions," and advises developers to "Take precautions to isolate Claude from sensitive data and actions to avoid risks related to prompt injection" (Anthropic, Computer use, 2026).
The Claude Code security page covers a different surface. It describes the sandbox: "Sandbox bash commands with filesystem and network isolation, reducing permission prompts while maintaining security." It then states that "The boundary is a permission prompt, so a Bash command you approve can still write anywhere your user account can" (Anthropic, Security, 2026). That governs commands, not a screenshot of a CRM page. A browser extension is another surface (Is Claude in Chrome Safe to Use for Work?).
OpenAI's help page shows a narrower approach: with connected apps its agent "can access sensitive data like emails, files, or account settings," and screenshots are not captured during a takeover, which "helps protect passwords" (OpenAI, 2026). That protects the password at one step, not the rest of the session.
| Dimension | Anthropic, per its documentation | What a buyer should verify |
|---|---|---|
| What the tool returns | "screenshot capabilities and mouse/keyboard control" | Whether the capture is the desktop, a window or a tab |
| Untrusted content | "Claude follows commands found in content" | Whether a classifier sits in front of the image |
| Injection defenses | An extra layer, not a guarantee | What the model received when a classifier missed |
| Command isolation | Bash sandbox with filesystem and network isolation | That the sandbox covers commands, not the screenshot |
Where does the developer's control end once the page is rendered?
It begins before the capture and ends at the pixels. The developer chooses the machine, the browser, the account the agent signs in as, the window size, which tabs are open and when the capture fires. A dedicated profile with only the CRM signed in, a scoped account without billing, and a capture timed to the record all reduce what the image contains. Those reductions are real, and a team can make them today.
The control ends at the pixel because the page is already drawn. The developer cannot remove a field without changing the page, and the capture takes the layout as it stands. Research on UI state makes the size of that gap concrete: many deployments "transmit the full UI state to remote inference servers even when most elements are irrelevant to the current task," which "can leak sensitive but unnecessary context such as authentication codes, private notifications, and background application states" (Yu et al., 2026). The same paper proposes minimizing the view on the client side before any observation leaves the device.
Human review is thinner than it appears. A systematic review of automation bias describes it as "the tendency to over-rely on automation" (Goddard et al., 2012). A reviewer sees the images the agent kept, at a volume nobody checks. The general case is covered in (Does an AI Agent See Everything on Your Screen?); here the developer owns the screen and still cannot filter it.
What does a screenshot leave out of the record?
An image shows what the model saw. It does not show who approved the action that followed, which values were real, or where a real value went. A team that stores every screenshot has an archive of the run's inputs, not a record of the run. The gap matters most when an agent acts: FINRA's 2026 oversight report flags that agents can go "beyond the user's actual or intended scope and authority," and that "Complicated, multi-step agent reasoning tasks can make outcomes difficult to trace or explain, complicating auditability" (FINRA, 2025), the record problem an image cannot answer.
A screenshot of a CRM page shows the contract value, but not whether that value was later pasted into an email, written to a note or carried into a second system by a follow-on task. The image is one frame in a chain, and the chain is what an assessor asks about. The person who approved is the one fact the image cannot supply, because approval happens outside it.
A record for review names four things for each step: the screen the model read, the action it proposed, the person who approved it and the destination where a real value resolved. Screenshots alone supply only the first. Picking one action from a team's own archive and trying to answer the other three from what it stored shows the part an assessor reconstructs first.
What does a governed environment change?
It changes what the model receives. A render-layer control substitutes consistent tokens such as ACCT_001 for the values a policy names before any model reads the screen, so a captured CRM page shows ACCT_001 where the contract value sat and the agent works on the token. The real value resolves only at the approved destination, at the moment of an approved action. A named person approves each submission or record change, and every screen, action and approval lands in an exported record.
That answers the question. The model sees the page it needs without the values that make the page worth stealing. An injected instruction can still ride in on the image, which is why approval and the record matter: a consequential action meets a person and a log. The scope is the rendered page, and it does not cover the terminal, the IDE, the local filesystem, direct API calls or MCP.
A browser control or a gateway acts elsewhere: a control on the request path sees the prompt, not the screen it was built from. The comparison is in (RedactSure vs Check Point Lakera: 2026 Comparison & Alternative), and the approval step is defined in (What Is Supervised Delegation?).
What the record shows
The answer turns on the screen: what it shows at capture, who approves the action that follows, and what record survives the run. Anthropic states that the tool provides screenshot capabilities over a rendered desktop, and that Claude may follow commands it finds in content; nothing in the pages reviewed replaces the values on the screen before the model reads them. A CRM page captured mid-task therefore puts the contract value and the contact's email address in front of the model whether the task needs them or not; the developer's only lever is the screen. The test a team can run is small. Pick one CRM task, capture the page the agent reads, and list every visible field the task never used. If that list is longer than the fields the task did use, the screen is doing work that a policy should be doing.
Methodology and limitations
This page rests on Anthropic's computer use and Claude Code documentation, OpenAI's help page, Brave's security research, a GUI-agent benchmark, a UI-minimization paper and FINRA's 2026 oversight report. Vendor pages were read between October 20, 2026 and October 22, 2026; each Sources line carries its date. Public documentation read in full gives a sound view of where each applies its control.
No product here was tested for this page. "Not described" means the pages reviewed do not carry the point. Vendor features change and the page carries its date. The example is a record type, not an organization. RedactSure is deploying now with enterprise and government customers. Determinations belong to the organization's counsel and compliance officers, and nothing here replaces a pilot of one workflow on its own screens.
Frequently asked questions
What does Claude computer use see in a screenshot?
Everything the captured area shows. The tool sends an image of the rendered page, so the model receives the fields the task uses and the fields it does not: panels, navigation, notifications and any text the page paints. On a CRM capture that can include a contract value and a failed-payment alert the task never touched. The developer chooses the page and account before capture, and that choice sets what the model reads.
Does the tool send the whole desktop or only the window it rendered?
Only the frame the developer actually sets; the pages reviewed do not describe a capture that filters individual fields. Anthropic's documentation describes the screenshot capability and its mouse and keyboard control. Whether the frame is a desktop, a window or a tab is a configuration choice, and a narrower frame reduces what is visible without changing which values are legible.
Can an instruction hidden in a page reach Claude through a screenshot?
Yes, if the instruction is painted into the image. Anthropic warns that Claude "follows commands found in content even when they conflict with your instructions," and Brave's researchers showed instructions hidden in faint light blue text that screenshot-reading agents act on (Anthropic, Computer use, 2026; Sahib and Chaikin, 2025). The image is content, so a page an attacker controls can speak to the model. Isolating the agent from sensitive data reduces what an instruction can reach.
Do Anthropic's injection classifiers make the screenshot safe?
No. The documentation reviewed describes classifiers as an extra layer, not a guarantee, and Anthropic's guidance says to isolate Claude from sensitive data and actions to avoid prompt-injection risk (Anthropic, Computer use, 2026). A classifier that catches most attempts still leaves what the model received when one gets through. The durable answer is to reduce what is on the screen, so a successful instruction has less to collect.
What record does a screenshot run leave?
An archive of images and a conversation, which shows what the model saw but not who approved an action or which values were real. FINRA's 2026 oversight report names the audit gap: "Complicated, multi-step agent reasoning tasks can make outcomes difficult to trace or explain, complicating auditability" (FINRA, 2025). A record built for review names the screen, the approver and the destination.
Who approves what the agent does on the page?
The developer's workflow decides, and the documentation reviewed names no per-action approver. Anthropic assigns the precautions to the developer: isolate Claude from sensitive data and actions. Approval belongs to the surrounding application, which may be a person watching, a rule in code, or nothing. For consequential actions, the useful design names a person and records the approval beside the screen.
How RedactSure Keeps a Screenshot to What the Task Needs
A developer who points computer use at a CRM record sends the model every field the page draws, not the three the task needs. Anthropic's documentation warns that Claude will follow commands found in content.
The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company.
Before, the model receives the whole account page, including the contract value, a contact's email address and a failed-payment alert. Inside that environment, the page renders with ACCT_001 and CONTACT_002, and the agent works on those tokens. A named person approves the record change, the real value resolves only at the destination that person approved, and the screen, action and approval go to the AI Control Record.
RedactSure is deploying now with enterprise and government customers.
Everyone else controls where data can go. RedactSure controls what the AI can see.
See it on your own workflow
A screenshot taken mid-task carries every field on the page, including the ones the task never needed. Bring one page your agent captures and see, with test values, what the model reads and who approves the change. Book a demo