Browser Use cannot inspect, screenshot, or click visible cross-origin iframe content in Codex app

[BradenHartsell]

Summary

In the Codex app, Browser Use with the in-app browser (iab) can attach to the selected tab and inspect the parent page, but cannot operate on the visible content when the meaningful app surface is inside a cross-origin iframe.

This caused a Browser Use task to become impossible without changing the page or using a non-Browser-Use workaround. The agent could see the parent page and the iframe element, but could not inspect the iframe DOM, could not click into the iframe, and screenshot capture returned the Codex shell/sidebar rather than the actual browser viewport.

Environment / Context

• Product surface: Codex desktop app / in-app browser
• Browser Use backend: iab
• Current tab URL:
  https://vibecodr.space/player/110503ee-044c-43e2-ab0d-ea8235f08b04
• Page title reported by Browser Use:
  Ultimate Tic-Tak-Toe | Vibecodr Player
• Meaningful app content is embedded in a cross-origin iframe:
  https://c-f405ee78-99b0-4ed2-a897-e1d00bb0d44c-rt.vxbe.space/__vibecodr/frame?artifactId=41bc68d9-351e-4c86-be84-7fe6b52c98d3
• The user explicitly required preserving the current page, not opening the iframe directly, not reading source code, not changing settings, and not navigating away.

What Worked

Browser Use successfully attached to the selected in-app browser tab:

{
  "ok": true,
  "title": "Ultimate Tic-Tak-Toe | Vibecodr Player",
  "url": "https://vibecodr.space/player/110503ee-044c-43e2-ab0d-ea8235f08b04"
}

The parent DOM exposed the player shell and iframe:

- heading "Ultimate Tic-Tak-Toe" [level=1]
- generic "Dedicated cross-origin sandbox on vxbe.space keeps the runtime isolated from the main Vibecodr app."
- text: Dedicated sandbox
- iframe [active]
- button "Restart vibe"
- button "Stop vibe"
- generic: Running

Iframe attributes were visible from the parent page:

{
  "count": 1,
  "attrs": {
    "title": "Vibe Runner",
    "class": "block w-full h-full border-0 bg-transparent h-full w-full border-0",
    "sandbox": "allow-scripts allow-forms allow-modals allow-pointer-lock allow-orientation-lock allow-presentation allow-same-origin",
    "allow": "fullscreen"
  }
}

What Failed

1. Browser Use could not enter the iframe DOM

frameLocator('iframe') and frameLocator('iframe[title="Vibe Runner"]') did not expose the iframe document/content:

{
  "htmlCount": 0,
  "rootCount": 0,
  "anyCount": 0,
  "buttons": 0
}

A scoped body lookup failed with:

Timed out after 3000ms waiting for selector iframe >> internal:control=enter-frame >> body: Browser Use encountered an error interacting with this webpage: Error: No element matched selector

2. Browser Use could not click into the visible iframe

Clicking the visible iframe itself failed before the click reached the app:

Unable to translate Input.dispatchMouseEvent in the in-app browser: Input targets inside cross-origin or inaccessible iframes are not currently supported in the in-app browser
waiting on click for selector iframe[title="Vibe Runner"]

3. Browser Use screenshots captured the Codex shell, not the browser page

Both screenshot APIs returned an image of the Codex desktop shell/sidebar rather than the visible in-app browser content:

await tab.cua.get_visible_screenshot()
await tab.playwright.screenshot({ fullPage: false })

This removed the normal visual fallback path. Even if DOM access is intentionally blocked by cross-origin iframe boundaries, a screenshot of the actual visible browser viewport plus coordinate clicks would have allowed the agent to proceed.

4. Browser Use page export command was unavailable

Trying the content export fallback failed with:

Unknown command: tab_content_export

Expected Behavior

For Browser Use in the Codex app, at least one of these should work reliably:

1. Attach directly to the visible cross-origin iframe as the active Browser Use target without changing the user's current page.
2. Forward coordinate clicks into visible iframe content from the parent page.
3. Capture screenshots of the actual in-app browser viewport, including cross-origin iframe pixels, so visual reasoning and coordinate interaction can work.
4. Provide a clear documented capability error and recovery path when cross-origin iframe interaction is unsupported.

Actual Behavior

Browser Use attached to the parent tab but could not inspect, screenshot, or interact with the meaningful app surface inside the iframe. Under a user requirement to preserve the current page and avoid direct iframe navigation/source inspection, the task became impossible through Browser Use.

Why this matters

This is a Codex/Browser Use issue because the failure is in the browser-control surface, not the website logic:

• The parent page rendered and exposed the iframe.
• The iframe was visible and active.
• Browser Use explicitly reported that cross-origin/inaccessible iframe targets are not currently supported.
• Screenshot capture did not capture the selected browser content, so Browser Use could not fall back to visual control.

The practical result is that agents fall back to scripts, source inspection, or direct runner URLs instead of using the built-in browser, which is exactly the behavior Browser Use is intended to avoid.

Suggested direction

A robust fix could be any of:

• Add cross-origin iframe target selection/attachment for the in-app browser.
• Add compositor-correct viewport screenshots for the actual selected browser pane.
• Add coordinate mapping/click forwarding into visible iframe bounds, with appropriate safety gates.
• Improve Browser Use diagnostics so the agent can tell the user immediately that the current visible iframe is not controllable and what action is needed.

[github-actions]

Potential duplicates detected. Please review them and close your issue if it is a duplicate.

• Codex in-app browser cannot complete D&D Beyond Google sign-in #19373

Powered by Codex Action