Skip to content

feat(browser): add cross-origin iframe support via CDP execution contexts#1084

Merged
jackwener merged 3 commits intojackwener:mainfrom
xzygis:feat/cross-origin-iframe-support
Apr 19, 2026
Merged

feat(browser): add cross-origin iframe support via CDP execution contexts#1084
jackwener merged 3 commits intojackwener:mainfrom
xzygis:feat/cross-origin-iframe-support

Conversation

@xzygis
Copy link
Copy Markdown
Contributor

@xzygis xzygis commented Apr 19, 2026

Summary

Add cross-origin iframe support using CDP execution contexts, enabling AI agents to discover and interact with elements inside cross-origin iframes without content scripts.

Closes #1077

Problem

Currently, opencli browser state detects iframes but marks cross-origin ones as (cross-origin) with no way to interact. This is because:

  1. Snapshot: el.contentDocument returns null for cross-origin iframes (browser security)
  2. Execution: Runtime.evaluate via chrome.debugger.sendCommand({ tabId }) only runs in the main frame
  3. Protocol: No frameIndex field exists to target a specific frame

Changes

6 files, +153/-4 lines — focused and backward-compatible.

Extension layer

extension/src/protocol.ts

  • Add 'frames' to Action union type
  • Add frameIndex?: number to Command interface

extension/src/cdp.ts

  • Track per-tab frame execution contexts via Runtime.executionContextCreated/Destroyed/Cleared events
  • Add registerFrameTracking() for event listener setup
  • Add getFrameTree(tabId) — wraps CDP Page.getFrameTree
  • Add evaluateInFrame(tabId, expression, frameId) — evaluate JS in a specific frame's context

extension/src/background.ts

  • Call registerFrameTracking() during initialization
  • Add Page.getFrameTree to CDP_ALLOWLIST
  • Add handleFrames() — returns flat list of child frames with index/frameId/url/name
  • Modify handleExec() — when frameIndex is set, resolve it to a frameId and use evaluateInFrame()
  • Add 'frames' case to command switch

CLI layer

src/browser/daemon-client.ts

  • Add 'frames' to DaemonCommand.action union
  • Add frameIndex?: number to DaemonCommand

src/browser/page.ts

  • Add frames() method — list all child frames
  • Add evaluateInFrame(js, frameIndex) — evaluate JS in a specific frame

src/browser/dom-snapshot.ts

  • Tag cross-origin iframes with [F0], [F1] indices in snapshot output
  • Add hint text: use: opencli browser frames

Design Decisions

  1. CDP execution contexts over content scripts: Stays consistent with the existing pure-CDP architecture. No manifest changes needed, no all_frames injection.

  2. Frame index (not frameId): Users and AI agents work with simple 0-based indices from opencli browser frames output, not opaque CDP frame IDs.

  3. Per-command frameIndex (not stored state): Follows the same pattern as windowFocused and idleTimeout — stateless, per-command relay.

  4. Backward compatible: frameIndex is optional everywhere. When absent, all behavior is identical to before.

Usage

# List all frames (including cross-origin)
opencli browser frames

# Output:
# index: 0  url: https://ads.example.com/banner  name: ad-frame
# index: 1  url: https://payment.stripe.com/...   name: payment

# Execute JS in a specific frame
opencli browser eval --frame 1 "document.querySelector('input').value"

# DOM snapshot now shows frame indices
# |iframe|[F0]<iframe src="https://ads.example.com" /> (cross-origin, use: opencli browser frames)

Testing

  • ✅ TypeScript type-check passes (0 new errors in CLI, 0 new errors in extension)
  • ✅ Backward compatible — no behavioral change without frameIndex

xuezhangying and others added 2 commits April 19, 2026 19:26
…exts

Enable interaction with cross-origin iframes through CDP's execution
context mechanism, without requiring content scripts or all_frames.

- Track frame execution contexts via Runtime.executionContextCreated events
- Add 'frames' action to list all child frames (including cross-origin)
- Support frameIndex in 'exec' action to evaluate JS in specific frames
- Add Page.frames() and Page.evaluateInFrame() APIs for CLI consumers
- Tag cross-origin iframes with [F0]/[F1] indices in DOM snapshots
- Add Page.getFrameTree to CDP allowlist

Closes jackwener#1077

Change-Id: Id03361ddb616912dff3bfa8e59e8b68716de590b
@jackwener jackwener force-pushed the feat/cross-origin-iframe-support branch from 5846bc8 to 19b56c9 Compare April 19, 2026 11:33
@jackwener jackwener merged commit 6b2f4cd into jackwener:main Apr 19, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature Request] browser commands: cross-origin iframe support

2 participants