A browser agent can correctly identify a suspicious page and still mishandle the task that follows. That is why a strong prompt-injection detection score should not be treated as a certificate that an agent can safely act inside business applications.
BrowseSafe++ provides useful evidence for examining that gap. Its paper was submitted on October 12, 2026; this is an analysis of already-public research, not a November 10 product launch. The benchmark covers attacked and clean page pairs in several representations, while the associated tooling also addresses adaptive attacks against agents. BrowseSafe++ paper
For a business considering browser automation, the practical question is narrower than whether a model is generally “secure”: can this agent complete its authorised task, using this account and tool access, when a page contains misleading instructions? Answering that requires several measurements with different denominators.
- 01Separate detection from actionSeparate detection of suspicious content from the agent's behaviour after reading it; they are different evaluation tasks.
- 02Match the page representationMatch the benchmark representation to what the deployed agent receives: HTML, Markdown, accessibility information or screenshots.
- 03Report coverageReport detector coverage alongside accuracy on valid decisions, including abstentions, errors and invalid outputs.
- 04Read the attack budgetRead adaptive attack results with their attempt budget, baseline and confirmation procedure; they are not ordinary page-level detection rates.
- 05Enforce action permissionsKeep permissions and action checks in the application, and measure whether the complete workflow respects them.
01 — Instruction boundaryStart with the boundary between a page and an instruction
A browser agent reads content to perform a user-authorised task. That content may describe products, display account data or contain instructions written by someone else. The application must distinguish information useful to the task from text that attempts to redirect the agent's behaviour.
Consider a hypothetical assistant asked to compare supplier delivery terms. It needs to read supplier pages and summarise relevant facts. A page's request to disclose unrelated internal information is outside that task, even if the page presents the request with confident formatting or apparent authority. The issue is the origin and authority of the instruction, not merely whether its wording looks unusual.
This distinction becomes more consequential when the browser is authenticated. The same reading mistake can have different effects depending on whether the agent can only inspect public pages or can also submit forms, download private material or change records. Evaluation must therefore describe both the content and the available actions.
The production prompt-injection taxonomy provides broader categories of these failures. BrowseSafe++ adds a more specific resource for testing page-based attacks and their clean counterparts.
Do not make the detector responsible for defining the entire task boundary. The application should already know which account, destination and action types are authorised. A detector can contribute evidence about a page; it does not create permission for an action that the user never requested.
02 — Benchmark unitsRead the dataset as pairs and representations
The paper describes 38,675 attacked/clean page pairs, spanning 12 website types, 24 attack types and 18 placements. Each pair is represented as HTML, Markdown, an accessibility tree and a screenshot. The pairing lets an evaluation ask whether a system detects the injected version without also rejecting the corresponding clean page. BrowseSafe++ paper
Those numbers describe the research collection, not a count of independent production incidents or a guarantee that every possible browser interaction is represented. The four views also should not be treated as four independent websites. They are alternative representations of related content.
| Representation | Deployment question | Limitation to check |
|---|---|---|
| HTML | Does the agent consume page markup? | Production extraction may remove or retain different material |
| Markdown | Does a conversion layer prepare the text? | Conversion can change ordering and visibility |
| Accessibility tree | Does the browser interface expose semantic elements? | Capture settings affect what reaches the agent |
| Screenshot | Does the agent interpret rendered pixels? | Viewport and rendering conditions matter |
For a team using screenshots, a text-only detector result is incomplete evidence. For a team using extracted text, a screenshot result may test information its actual agent never receives. Match the evaluation to the deployed observation path before comparing scores.
The repository also distinguishes public labelled material from a held-out test process. Its test interface presents one label-free page from each of 9,175 test pairs, with opaque identifiers and aggregate leaderboard evaluation. Do not assume that the entire research collection is a freely labelled test set available for unrestricted local scoring. BrowseSafe++ repository
Record the dataset revision and selected view with any result. If the material or rendering pipeline changes, a score without those details cannot establish that two systems faced the same inputs.
03 — Separate behavioursKeep detection and resistance separate
A detector answers whether a page contains a prompt injection. An agent-resistance test asks what the agent does while attempting a task in the presence of that content. The two questions can produce different outcomes for the same page.
Perplexity's analysis evaluates detection and task execution independently, rather than giving the execution run the detector's previous verdict. It reports cases where recognising an injection did not translate into resisting it. That supports examining the gap, but it is not the same experiment as deploying a detector in front of every production action. Perplexity's BrowseSafe++ analysis
A useful application test distinguishes four outcomes: the agent completes the legitimate task; it refuses or abandons a clean task; it is redirected into an unauthorised action; or it stops safely but leaves the task incomplete. Combining these into one “safe” percentage can conceal the operational trade-off.
For example, a browser assistant that refuses every page may avoid acting on an injected instruction while being useless for procurement research. A system that completes many tasks may still be unacceptable if a small subset performs unauthorised actions. The required balance depends on the action's consequences and reversibility.
Test the application response
Evaluate a detector and the complete application separately, then evaluate their integration. What happens when the detector flags a page? Does the application stop, request review, use a different source or continue with restricted tools? What happens when the detector times out? These choices determine the deployed behaviour.
Measure the completed task
The agent evaluation pipeline guide helps organise this distinction: component scores can diagnose a system, while end-to-end task evidence establishes what users actually experience.
04 — Metric coverageRead detector metrics together with coverage
The BrowseSafe++ classification documentation distinguishes valid classification decisions from abstentions, errors and invalid outputs. Its valid-only metrics exclude those non-decisions, while coverage records how much of the evaluation received a valid decision. Reporting only valid-only accuracy can hide a detector that avoids difficult pages. Classification evaluation documentation
The denominator is the first thing to inspect. Precision concerns the pages flagged by the detector. Recall concerns attacked pages. A false-positive rate concerns clean pages. Coverage concerns all evaluated pages. None is interchangeable with successful completion of an authorised browser task.
| Measurement | Question it answers | Companion evidence needed |
|---|---|---|
| Detection recall | How many attacked pages were found? | Misses, coverage and attack composition |
| Clean-page false positives | How often was clean content flagged? | Legitimate task interruption and review burden |
| Valid-decision accuracy | How often were completed classifications correct? | Abstentions, errors and invalid predictions |
| End-to-end action outcome | Did the agent respect its task boundary? | Legitimate completion and permission context |
The SDK also reports measures that count failures on attacked pages as misses and a pair-level measure requiring both members to be valid and correct. These help expose weaknesses hidden by a valid-only headline. Compare them alongside the simpler rates rather than selecting whichever metric makes one system look strongest. Classification evaluation documentation
For a production team, abstention is not inherently a defect. A deliberate handoff can be appropriate for uncertain content. The important questions are where that handoff goes, what it costs and whether the original task eventually completes.
Keep technical errors distinct from deliberate abstentions. A timeout and a considered refusal may both prevent a classification, but they call for different engineering responses and should not disappear into one miscellaneous category.
05 — Alert workloadTranslate false alarms into the workload you expect
Even a low false-positive rate can create substantial review work when most pages are benign. The proportion of real attacks in the workload affects how many alerts turn out to be useful.
Consider this hypothetical calculation, not a BrowseSafe++ result. An agent encounters 10,000 pages, of which 100 contain attacks and 9,900 are clean. A detector catches 90 of the attacked pages and incorrectly flags 1% of clean pages.
| Hypothetical quantity | Calculation | Result |
|---|---|---|
| Attacks detected | 90 of 100 | 90% recall |
| Clean pages flagged | 1% × 9,900 | 99 false alerts |
| Total alerts | 90 + 99 | 189 alerts |
| Alerts that are real attacks | 90 ÷ 189 | About 47.6% precision |
The detector in this example misses 10 attacks and sends 99 clean pages into whatever alert process the application uses. The arithmetic does not say that detection is pointless. It shows why an alert workflow, rather than a single percentage, determines practical value.
If every alert blocks a customer-facing task, interruptions matter. If every alert goes to an analyst, review time matters. If alerts merely add a note while the agent continues unchanged, the presence of a detector may have little effect on the action being evaluated.
Do not substitute the balanced composition of a benchmark for the expected composition of production traffic. Estimate the relevant range, show the assumption and revisit it as the application gains evidence. Detector-flag counts are not automatically confirmed attack counts.
For buyers, request both the raw evaluation counts and an explanation of the operating threshold. A vendor can trade recall against false alarms by changing that threshold. You need to know whether the reported setting matches the one available in the product you would deploy.
06 — Adaptive conditionsInterpret adaptive attacks as a different experiment
An adaptive attacker can adjust candidates based on feedback, rather than submitting one fixed page and accepting the result. The repository's campaign design includes a clean baseline, a fresh target for each candidate, confirmation runs for successful candidates and a bounded attempt budget. Those conditions belong beside any reported outcome. BrowseSafe++ repository
This is different from the classification interface, where each page receives one detector call without retries. Comparing a many-attempt adaptive success rate directly with a one-call classification score confuses the unit of evaluation. Classification evaluation documentation
A buyer should ask what counts as success. Did the target merely repeat suspicious text, fail the legitimate task, or perform an unauthorised action? Were results confirmed from fresh state? How many attempts were allowed? Which tools and credentials were available? Did the clean baseline complete the intended task?
The last question is essential. If an agent cannot perform the clean task, its behaviour under attack is difficult to interpret as evidence of useful resilience. Refusing everything is not a substitute for a working baseline.
Keep comparisons within matched conditions. A result from a narrow read-only task does not establish the behaviour of an agent with write access. Likewise, a result from a particular model, prompt and browser harness should not be described as a property of all products using that model.
The practical lesson is to budget evaluation effort explicitly. A fixed test collection is useful for regression checks. A controlled adaptive exercise explores whether someone can find weaknesses beyond those known examples. They answer complementary questions, and neither removes the need to constrain the application's available actions.
07 — Bounded evaluationTurn the research into a bounded application evaluation
Start with an isolated test environment containing synthetic records, disposable accounts and local or otherwise controlled pages. Reproduce the application's observation path and permissions without exposing real customer data or production actions. The initial objective is to understand the boundary, not to execute attacks against external services.
Choose a small set of legitimate tasks with observable success criteria. Examples include extracting specified facts, filling a draft form without submitting it, or navigating to an authorised record. Define the forbidden outcomes separately, such as opening an unrelated private record or attempting an unapproved external submission.
Run the clean task first. Then use approved evaluation fixtures and record the full outcome: observation, detector decision, application response, attempted tool action and final task state. Distinguish an attempted action blocked by the application from an action successfully performed. Both are useful evidence, but they identify different strengths and weaknesses.
The browser-agent account and session checks help keep the test's permissions explicit. Confirm the account and scope at the action boundary, not only at the start of the chat. A browser tab can change state during a long task.
Consider whether each action needs the browser at all. The browser-or-API action guide explains how a structured integration can make some permissions and outcomes easier to validate. That is an architectural choice for a particular workflow, not a claim that APIs eliminate injection risks.
Use review gates where consequences justify them, and make failure behaviour explicit. A detector outage should not accidentally widen permissions because the surrounding application treats missing evidence as approval.
08 — Buying evidenceAsk for a deployment claim you can verify
The strongest procurement answer describes a system and its operating conditions, not a bare leaderboard number. Ask the supplier to connect evidence to the exact product configuration, available actions and failure handling you would receive.
| Procurement question | Useful evidence | Unresolved if missing |
|---|---|---|
| What did the model see? | Representation, capture method and revision | Whether the benchmark matches deployment |
| What could the agent do? | Tool permissions and account scope | Consequence of a missed attack |
| What was counted? | Task, page, pair and attempt denominators | Meaning of the reported rate |
| What happens after a flag? | Observed stop, review or restricted continuation | Whether detection changes behaviour |
BrowseSafe++ is released as a research artifact rather than a production-supported security service. Dataset access and licensing also differ across the code, training material and test process. Check those conditions before incorporating it into a commercial evaluation programme. BrowseSafe++ repository
A team can still use the research to improve its own acceptance criteria. Keep a revisioned test record, review failures by consequence and rerun relevant cases when the model, prompt, browser representation or permissions change. Do not treat a past score as permanent evidence for a changed system.
The resulting decision may be to approve read-only research while withholding write actions, or to allow a narrow form workflow with application-enforced checks. Those scoped decisions are more useful than an unsupported statement that a browser agent is safe for any task.
09 — Next stepConclusion
Verify the system you will deploy
Read BrowseSafe++ as evidence about specific detection and agent-behaviour experiments. Before granting a browser agent more access, verify the representation it sees, the denominator behind its scores and the actions it can actually take. Then evaluate legitimate completion and unauthorised behaviour together in the application you intend to deploy.
Digital Applied's AI transformation services help teams define those task boundaries, choose appropriate integrations and build evaluations around observable business outcomes.