feat(peek): get_page_view — live ref-tagged snapshot + ref-based targeting (R1)#101
Merged
Conversation
…eting R1 of the MCP token-optimization work. The agent perceives the live page as a compact, masked, ref-tagged list of interactive elements and targets a ref in execute_action/request_authorization instead of authoring a CSS selector from get_dom_snapshot HTML — far fewer tokens per perception and deterministic. - snapshot.ts: self-contained MAIN-world buildPageView walker + a window-global ref registry (rrweb-invisible; replaced per snapshot; wiped on navigation). - action-schema.ts: page_view action + optional ref on click/type/dblclick/ scroll/enter (selector now optional; ref-or-selector enforced at dispatch). - dispatcher.ts: resolveElement/resolveTarget resolve ref via the registry first; a missing/detached ref yields a clear 'ref expired' error. - background.ts: dispatchInMainWorld runs buildPageView, masks names/values SW-side (maskTextContent), returns compact ref-tagged lines in details. - action-handler.ts: page_view intercepted before the gate, auto-allowed at effective Level >= 1 (read); new 'level-1-read' approver (4 mirrors updated). - server.ts: get_page_view tool (rides the execute_action rails, surfaces the view via response.details). Masking (two-layer): password/email/tel + PII-autofill values (card/address/ bday/name/org) and fields marked .rr-mask/data-private/data-dd-privacy/ data-peek-mask are dropped in-page; the SW scrubs structured PII from names + remaining values. Free-text values may still be returned (same as the recorder). Ref-targeted destructive actions go through the destructive-confirm override (the matcher resolves the same ref'd element). Non-mutating, audit-logged, Level 1+. Selector targeting unchanged. Reviewed (independent pass) + fixes applied; 23 new unit tests; 580 ext + 294 mcp green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: harry-harish <22562634+harry-harish@users.noreply.github.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds a new Changesget_page_view: live ref-tagged page snapshot
Sequence DiagramsequenceDiagram
participant Agent as AI Agent
participant MCPServer as get_page_view MCP tool
participant ActionHandler as action-handler
participant SWBackground as background.ts (SW)
participant MainWorld as buildPageView (MAIN)
participant PeekRefs as window.__peekRefs
Agent->>MCPServer: get_page_view sessionId maxElements?
MCPServer->>ActionHandler: execute_action type=page_view
ActionHandler->>ActionHandler: check effectiveLevel >= 1
ActionHandler->>SWBackground: dispatchInMainWorld page_view
SWBackground->>MainWorld: scripting.executeScript buildPageView
MainWorld->>PeekRefs: create Map, populate ref→el
MainWorld-->>SWBackground: PageViewResult nodes url title
SWBackground->>SWBackground: maskTextContent on names/values
SWBackground-->>ActionHandler: ok=true details
ActionHandler-->>MCPServer: verdict=allow level-1-read details
MCPServer-->>Agent: ref-tagged element lines
Note over Agent,PeekRefs: Subsequent ref-targeted action
Agent->>SWBackground: dispatchAction type=click ref=e5
SWBackground->>PeekRefs: lookup e5
alt ref valid and isConnected
SWBackground->>MainWorld: click element
MainWorld-->>SWBackground: ok=true
else ref missing or detached
SWBackground-->>Agent: ok=false error "ref expired e5"
end
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/peek/PRIVACY_POLICY.md`:
- Around line 72-80: The privacy policy text for the get_page_view feature
currently states it works "instead of guessing a CSS selector," which implies
CSS selectors are not used, but the actual behavior allows selectors as a
fallback. Update the wording in the "Live page snapshot (get_page_view)" section
to soften this language and indicate that refs are the preferred method when
available, while acknowledging that the action flow can still accept CSS
selectors as a fallback when a stable reference is not available. This will
align the documented policy with the actual system behavior.
In `@packages/peek-mcp/src/mcp/action-schema.ts`:
- Line 88: The EnterActionSchema has an inconsistent selector field validation
compared to other action schemas. Change the selector field in EnterActionSchema
from z.string().optional() to z.string().min(1).optional() to match the
validation pattern used in ClickActionSchema, TypeActionSchema, and
DblClickActionSchema. This ensures consistent behavior across all action schemas
by preventing empty strings from being accepted, maintaining parity with how the
dispatcher and other schemas handle selector validation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro Plus
Run ID: cb200ae9-620c-4a73-aebc-0af57b1ffc88
📒 Files selected for processing (19)
.changeset/peek-live-page-view.mddocs/peek/PRIVACY_POLICY.mdpackages/peek-extension/entrypoints/background.tspackages/peek-extension/src/__tests__/action-handler.test.tspackages/peek-extension/src/__tests__/dispatcher.test.tspackages/peek-extension/src/__tests__/snapshot.test.tspackages/peek-extension/src/permissions/action-handler.tspackages/peek-extension/src/permissions/action-protocol.tspackages/peek-extension/src/permissions/dispatcher.tspackages/peek-extension/src/permissions/snapshot.tspackages/peek-mcp/README.mdpackages/peek-mcp/src/mcp/action-schema.tspackages/peek-mcp/src/mcp/host-bridge.tspackages/peek-mcp/src/mcp/server.tspackages/peek-mcp/src/native-host/action-protocol.tspackages/peek-mcp/src/native-host/audit.tspackages/peek-mcp/test/action-schema.test.tspackages/peek-mcp/test/server.test.tspackages/peek-mcp/test/stdio-smoke.test.ts
- action-schema: EnterActionSchema.selector now z.string().min(1).optional() to match Click/Type/DblClick (was .optional(), allowing ""). Consistency nit. - PRIVACY_POLICY: soften "act on a stable reference instead of guessing a CSS selector" — selector targeting still works as a fallback, so phrase get_page_view as preferred-over, not instead-of. 294 mcp tests pass; typecheck + biome clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: harry-harish <22562634+harry-harish@users.noreply.github.com>
harry-harish
added a commit
that referenced
this pull request
Jun 20, 2026
Mirror the two fixes landed on the R1 base branch so the stack stays in sync and #102 merges cleanly when retargeted to main: - EnterActionSchema.selector -> .min(1).optional() - PRIVACY_POLICY get_page_view wording (prefer-over, not instead-of) 303 mcp tests pass; typecheck + biome clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Signed-off-by: harry-harish <22562634+harry-harish@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Implements R1 from the MCP token-optimization research: a new
get_page_viewMCP tool that returns a live, masked, ref-tagged list of interactive page elements, and ref-based targeting for the act verbs. The agent can now target a stableref(e.g.e5) instead of authoring a CSS selector fromget_dom_snapshot's ~24KB of HTML — far fewer tokens per perception, and deterministic (no selector guessing).Spec:
_context/docs/research/2026-06-20-peek-mcp-token-optimization.md+_context/docs/specs/2026-06-20-peek-live-snapshot-refs-spec.md.How
snapshot.ts— self-contained MAIN-worldbuildPageViewwalker + awindow.__peekRefsregistry (a JS global, so rrweb never records it; replaced per snapshot; wiped on navigation).action-schema.ts—page_viewaction + optionalrefon click/type/dblclick/scroll/enter (selector optional; ref-or-selector enforced at dispatch — kept optional in the schema so members stay plain objects for the discriminated union).dispatcher.ts—resolveElement/resolveTargetresolverefvia the registry first; a missing/detached ref returns a clearref expirederror.background.ts—dispatchInMainWorldrunsbuildPageView, masks names/values SW-side, returns compact ref-tagged lines indetails; the destructive matcher now threadsreftoo.action-handler.ts—page_viewintercepted before the gate, auto-allowed at effective Level ≥ 1 (read), newlevel-1-readapprover (4 mirrors updated).server.ts—get_page_viewtool (rides the execute_action rails; surfaces the view viaresponse.details).Safety / privacy
.rr-mask/data-private/data-dd-privacy/data-peek-mask, are dropped in-page; the SW scrubs structured PII (emails, cards, tokens) from names + remaining values. Honest scope: free-text field values may still be returned (same as peek's existing recorder) — the tool description + privacy policy say so.selectortargeting unchanged (backward-compatible).Review + testing
Built subagent-driven + put through an independent code-quality/security review; all top findings addressed (C1 destructive-gate bypass, H1 imprecise autocomplete masking, H2 overstated masking claims, M1 clearer dispatch error, + the privacy-policy doc). 23 new unit tests; peek-extension 580 + peek-mcp 294 green; typecheck + biome + build clean. Changeset included (minor, both packages).
Follow-ups (out of scope, noted in the spec)
R2 diff-after-action, CDP
getFullAXTreeaccurate-accname mode,structuredContent, and a real-browser e2e (the rrweb-invisibility is guaranteed by construction —__peekRefsis a JS global, not DOM).🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
New Features
get_page_view/page_view) for Level 1+ viewing of interactive elements.ref) in addition to CSS selectors, enabling more reliable click/type/scroll/enter/double-click targeting.Privacy & Security
Bug Fixes
Documentation