Skip to content
redactsure
Book a demo

Explore.

Data Report · By Chris Sowa · Published
Last updated

How Do Websites Manipulate AI Agents?

Websites manipulate AI agents with content a person never sees, such as hidden text, faint image text, fake pop-ups and injected form fields, which the agent reads as instructions and acts on with the user's logged-in authority. Check what your agent can see and do on any page it visits, because the page decides what it reads.

An abstract agent node faces a browser page backed by several overlapping instruction sheets.

Websites manipulate AI agents with content a person never sees, such as hidden text, faint image text, fake pop-ups and injected form fields, which the agent reads as instructions and acts on with the user's logged-in authority. Check what your agent can see and do on any page it visits, because the page decides what it reads. A person browsing a page sees what the designer rendered for human eyes. An agent reads the page differently: as a document tree, as a screenshot, or as both, and each of those channels can carry content aimed at the agent alone. The research of the past two years has catalogued the techniques, measured how often agents follow them, and explained why a logged-in browser makes the consequences severe. This page sets out the techniques web pages use to address an agent rather than a person, how image-based injections reach screenshot-reading agents, how often agents comply, why a logged-in session turns page content into control, what the browser vendors have concluded, and what an organization should verify before letting an agent browse. It extends the general question of AI agent data exposure to the case where the exposure is engineered by the page.

Key findings

  • Injected web elements can steal user data at scale: the Environmental Injection Attack achieves "up to 70% attack success rate (ASR) in stealing users' specific PII" against generalist web agents (Liao et al., 2025).
  • On the WASP benchmark, agents begin following injected instructions in up to 86 percent of cases, though "even state-of-the-art agents often struggle to fully complete the attacker goals" (Evtimov et al., 2025).
  • Visual injections work on both kinds of agent: "CUAs and BUAs can be deceived at rates of up to 51% and 100%, respectively, on certain platforms" (Cao et al., 2026).
  • Attackers can "hide prompt injection instructions in images using a faint light blue text" that screenshot-reading agents act on while a person sees nothing (Brave, 2025).
  • In an agentic browser, "The AI operates with the user's full privileges across authenticated sessions," so same-origin style protections do not contain it (Brave, 2025).
  • OpenAI's own assessment is that "Prompt injection, much like scams and social engineering on the web, is unlikely to ever be fully 'solved'" (OpenAI, 2025).

What techniques do web pages use to address an agent rather than a person?

Four families recur in the research, and all exploit the gap between what is rendered for a person and what is delivered to a model. The first is hidden text: instructions in the page's document tree styled so a person cannot see them, through zero-size fonts, text colored to match the background, elements positioned off screen, or content in attributes and comments the browser never displays. An agent reading the page structure receives all of it as ordinary text.

The second is injected form fields. The Environmental Injection Attack study showed that an attacker who can add elements to a page can place input fields an agent will fill, so that personal data the agent holds is typed into attacker-controlled inputs; the study reports "up to 70% attack success rate (ASR) in stealing users' specific PII" (Liao et al., 2025). The person never sees the field. The agent, looking for where to enter an address, finds it.

The third is the fake interface element: a pop-up, banner or dialog that looks to the agent like part of the task. A message reading "verify your account to continue" is a phishing lure to a person and an instruction to an agent, which may follow it because it appears to stand between the agent and its goal. The fourth is content in images, discussed in the next section. Palo Alto Networks Unit 42 has reported web-based injection against agents observed in the wild, which moves these techniques from catalog to incident (Palo Alto Networks Unit 42, 2026).

Consider an order management screen in a company's purchasing system. The agent's task is to check a supplier's current price, which means leaving the purchasing system for the supplier's page and returning. The supplier's page is where every one of these techniques can live, and the agent comes back to the order screen with whatever that page told it (How to secure AI agents?).

How do image-based and visual injections reach screenshot-reading agents?

Through the agent's eyes. Many browser and computer-use agents work from screenshots rather than from the page's structure, which closes the hidden-text channel and opens a visual one. Brave's researchers showed that attackers can "hide prompt injection instructions in images using a faint light blue text" that a person perceives as blank space or texture, while a model reading the screenshot extracts and follows it (Brave, 2025). The text is present in the pixels. Only the model reads it.

VPI-Bench, a benchmark of visual prompt injection attacks, measured this across computer-use agents, which see the whole desktop, and browser-use agents, which see the page. It found that "CUAs and BUAs can be deceived at rates of up to 51% and 100%, respectively, on certain platforms" (Cao et al., 2026). The browser agents were the more vulnerable, which follows: the page occupies their entire field of view, and the page is the attacker's.

Channel the agent reads Technique that addresses it What a person sees
Page structure (document tree) Hidden text, off-screen elements, attribute content Nothing
Rendered form Injected input fields Nothing, or an ordinary form
Screenshot pixels Faint or low-contrast text in images Blank space or a normal image
Interface Fake pop-ups and dialogs A plausible prompt they would ignore

The table makes one point: there is no reading channel an agent uses that a page cannot address. Defenses that filter the document tree miss the pixels; defenses that scan the pixels miss the tree. The channel an agent reads is the channel the attacker writes to, and an agent that reads both is exposed on both.

How often do agents follow what a manipulated page tells them?

Often enough to plan for, and the research separates two steps. The first is whether the agent begins to follow the injected instruction, deviating from its task. The second is whether it completes the attacker's goal. WASP, a benchmark of web agent security against prompt injection, found that agents begin following injections in up to 86 percent of cases, while noting that "even state-of-the-art agents often struggle to fully complete the attacker goals" (Evtimov et al., 2025).

The gap between the two steps is sometimes read as reassurance. It should not be. Agents fail to complete attacker goals for the same reasons they fail to complete legitimate ones: multi-step web tasks are hard, and models lose track. That is a capability limit, not a security control, and it shrinks with every model release. An agent that begins following an injected instruction 86 percent of the time has already done the thing a security team needs to prevent, which is to treat page content as authority.

The Environmental Injection Attack results show what completion looks like when the attacker's goal is simple. Stealing a specific piece of personal data requires one action, typing a value into a field, and the attack succeeds at up to 70 percent (Liao et al., 2025). The lesson is that the attacker chooses the goal, and will choose one the agent can complete. A buyer evaluating an agent should ask about the first step, deviation, not the second, because the second is a measure of the attacker's ambition rather than the agent's safety (What Is Sensitive Data Masking for AI Agents?).

Why does a logged-in session turn page content into control?

Because the agent carries the user's authority to every page it reads. Brave's analysis of indirect injection in an agentic browser puts it directly: "The AI operates with the user's full privileges across authenticated sessions," which is why the same-origin style protections that normally isolate one site's content from another's permissions do not contain it (Brave, 2025). The browser's security model assumes a page can act only within its own origin. An agent reads the supplier's page in one tab, holds the purchasing system's session in another, and is the bridge between them.

In the order management example, the consequence is concrete. The supplier's page, through any of the techniques above, tells the agent that the supplier's remittance account has changed and should be updated in the purchasing system. The agent returns to the order screen, where it is signed in with the buyer's permissions, and makes the change. No credential was stolen. No boundary was broken in the browser's terms. The page supplied an instruction and the session supplied the authority, and the agent joined them.

This is why web manipulation is a different problem from web security as traditionally understood. The attacker does not need to compromise the purchasing system or the browser. The attacker needs the agent to read a page while signed in somewhere that matters, and most useful browsing tasks involve exactly that. The controls that address it are not on the page, which the organization does not own, but on the two things the organization does own: what the agent can see on its own systems' screens, and which actions it can take there without a person (How to secure AI Agents in Enterprise?).

What have the browser vendors concluded?

That the problem is permanent and the answer is architectural. OpenAI, describing its work on the ChatGPT Atlas browser, states that "Prompt injection, much like scams and social engineering on the web, is unlikely to ever be fully 'solved'" (OpenAI, 2025). The comparison to social engineering is apt: the attack targets the reader's judgment, and a reader that follows instructions in text will always be subject to text written to be followed.

Google has published its approach to architecting security for agentic capabilities in Chrome, treating the agent as a new principal within the browser whose actions require constraints the browser did not previously need (Google, 2025). Brave's two reports, one on injection through page text and one on injection through images, were published as security research on a competing product and led to fixes, and both frame the issue as structural to agentic browsing rather than specific to one implementation.

What the vendor pages reviewed have in common is the absence of a claim to have solved the problem. Each describes layers that reduce the rate, and each describes a residual that the user, or the organization deploying the agent, must handle. For a buyer, that is the useful conclusion. The browser will get better at resisting manipulation and will never finish. The deployment has to assume that some page, some day, will succeed in steering the agent, and has to be built so that the steered agent sees little of value and can do little without a person agreeing (What Is AI Agent Data Security?).

What does a governed environment change?

When a browsing agent runs in a governed environment, a manipulated page can still address the model, but the organization's own screens reach that model as tokens and any action the page requests waits for a named person.

The mechanism is a class of control, not a product. Sensitive values on the rendered screen are replaced with consistent tokens before any model reads them, so the task runs on tokens. Real values resolve only at an approved destination at the moment of an approved action. A named person confirms what is tokenized and approves consequential actions such as payments, submissions and record changes. Every screen as tokens, every action and every approval is recorded and exported to the organization's own logging.

This addresses the second and third of the checks this page sets out. The research on this page says every channel a page can use, hidden text, faint image text, injected fields, is addressable, so the agent will read things a person does not; the variable the organization controls is what is on its own screens when that happens. In an agentic browser the model operates with the user's full privileges across authenticated sessions (Brave, 2025), and in a governed environment those privileges open screens of tokens. Agents begin following injected instructions in up to 86 percent of cases on the WASP benchmark (Evtimov et al., 2025), so the question is whether anyone is positioned to say no when a change to a remittance account or a form submission is proposed; in this class of control, a named person is, and the refusal is recorded.

The environment does not change the pages, the user's permissions or the model; it changes what a successful injection can read and what it can finish.

Where does RedactSure sit?

The environment that enforces this is built by RedactSure, an AI agent controls, governance and data protection company. The design accepts that a web page will sometimes steer the agent and limits what steering reaches. The agent works inside a governed environment in the customer's cloud. Substitution happens at the render layer before any model reads the screen, so the order management screen, the customer account or any other page of the organization's own systems reaches the model with sensitive values replaced by consistent tokens such as ACCT_001 or PATIENT_001 for the whole run; a manipulated page that extracts the screen extracts tokens. Real values resolve only at approved destinations at the moment of an approved action. A named person confirms the tokenization policy and approves every payment, submission and record change before it executes, which RedactSure calls Supervised Delegation, so a remittance change a supplier's page requested reaches a person who can refuse it. Every screen as tokens, every action and every approval lands in the AI Control Record and exports to the customer's SIEM. The design is model agnostic, requires no per-application integration and leaves user permissions unchanged.

Deployments are in pilot. A pilot on the reader's own browsing workflow, with an instruction planted on a test page, would show the organization's screens as the model saw them, the action the page requested, and the person who refused it. Everyone else controls where data can go. RedactSure controls what the AI can see.

Methodology and limitations

This page rests on peer-reviewed research, vendor security research and documentation, and industry analysis: the Environmental Injection Attack study (arXiv:2409.11295, ICLR 2025), the WASP benchmark (arXiv:2504.18575, NeurIPS 2025 Datasets and Benchmarks), VPI-Bench (arXiv:2506.02456, ICLR 2026), Brave's two reports on prompt injection in agentic browsers, Palo Alto Networks Unit 42's report on web-based injection observed in the wild, Google's post on securing agentic capabilities in Chrome, and OpenAI's post on hardening ChatGPT Atlas. All sources were read as of October 16, 2026. Figures are quoted as the papers report them, under their own benchmark conditions and against the models they tested.

No hands-on testing of any product was performed, and no attack was run for this page. Where this page says a capability is "not described," it means not described in the pages reviewed, not that the capability is absent. The vulnerabilities Brave reported were addressed by the vendor concerned; browser agent features and attack techniques change quickly, and the page carries its date. Statements about RedactSure describe its current design, with deployments in pilot, and are confined to the section on where the company sits. The order management screen and supplier page examples describe record types, not any organization; no customer or prospect is described.

Whether a given deployment's browsing controls, screen handling and action approvals satisfy an organization's security policy, its regulatory obligations or its contractual commitments is a determination for the organization's counsel, its assessors and its own security and compliance officers, not for a research page.

What the record shows

Websites manipulate AI agents by writing to the channels the agent reads and the person does not: hidden text in the document tree, injected form fields, faint text in images and fake interface elements. The research record shows agents beginning to follow such content most of the time, completing simple attacker goals such as data theft at high rates, and doing so with the user's full privileges across every authenticated session the browser holds, which is what makes a page's instruction into control. The browser vendors describe the problem as permanent and their defenses as layers with a residual. The organization's controls are therefore on its own side of the boundary: what the model sees on the organization's screens, and which actions wait for a named person. The test is to name one agent and one task and ask: if the next page this agent visits tells it to change a record in our system, what does the model see on that record, and who has to agree before the change lands?

Frequently asked questions

Can a web page really contain text that only an AI agent can read?

Yes, in more than one way. Text in the page's document tree can be styled so a person cannot see it, through zero-size fonts, matching colors or off-screen positioning, while an agent reading the structure receives it as ordinary text. Text can also be placed in images at a contrast so faint a person perceives blank space, which Brave's researchers showed screenshot-reading agents extract and follow. Every channel an agent reads can carry content a person never sees.

Does using screenshots instead of page code protect the agent?

No, it changes the channel. An agent working from screenshots does not read hidden text in the document tree, but it reads the pixels, and faint text in images is written for exactly that reader. VPI-Bench measured visual prompt injection against browser-use agents at success rates up to 100 percent on some platforms. Filtering the document tree misses the pixels; scanning the pixels misses the tree. An agent that reads both is exposed on both.

Agents often fail to complete attacker goals. Is that a defense?

No. The WASP benchmark found agents begin following injected instructions in up to 86 percent of cases while often failing to complete the attacker's full goal. They fail for the same reasons they fail legitimate multi-step tasks, which is a capability limit that shrinks with each model release. An attacker also chooses goals the agent can complete; stealing one piece of data is a single action and succeeds at high rates. Deviation is the event to prevent.

Why does it matter that the agent is logged in?

Because the page's instruction and the session's authority combine. Brave's analysis found that an agentic browser's AI operates with the user's full privileges across authenticated sessions, so protections that isolate one site's content from another's permissions do not contain it. An agent that reads a supplier's page while signed in to the purchasing system can carry the page's instruction into the purchasing system, with no credential stolen and no browser boundary broken.

Can we solve this by only allowing trusted websites?

Only partly. An allow list bounds which pages the agent visits, which reduces exposure, but trusted sites carry content from untrusted authors in reviews, comments, listings and advertisements, and agents navigating on their own judgment leave the list in one click. The browser vendors themselves describe prompt injection as unlikely to ever be fully solved. The allow list is a useful layer; it is not a reason to leave the agent's view of sensitive screens and its unsupervised actions unaddressed.

What is the single most useful question to ask a vendor?

Show me what the model received from a real page, and show me what happens when that page tells the agent to change a record in my system. The first half reveals whether the agent reads content a person does not, which it will. The second reveals what the model sees on the organization's own screen when it returns and whether the change pauses for a person. The answers to those two questions, not the vendor's detection rate, determine the exposure.

Book an AI Agent Security Review →

Bring your hardest questions.

Bring one sensitive workflow and the decisions your team needs to keep in its hands.

Book a demo

About the author

Christopher Sowa
Chris SowaCo-Founder, President and Chief Commercial Officer, RedactSure

Chris Sowa is a founder of RedactSure and a former CEO of an AI company; he started it years before ChatGPT existed. He previously led AI at Accenture, served as Global VP of Strategy & Innovation at Schneider Electric, was CCO of Sovos, and spent more than a decade at Oracle, with earlier roles at SAP and IBM.

Book a demo

Book a demo

Having trouble? Open the booking page or email us.