Skip to content

fix(webview): support input for elements inside cross-origin <iframe>#41463

Open
dcrousso wants to merge 2 commits into
microsoft:mainfrom
dcrousso:fix-WKWebView-cross-origin-iframe-input
Open

fix(webview): support input for elements inside cross-origin <iframe>#41463
dcrousso wants to merge 2 commits into
microsoft:mainfrom
dcrousso:fix-WKWebView-cross-origin-iframe-input

Conversation

@dcrousso

@dcrousso dcrousso commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

No description provided.

@dcrousso dcrousso requested review from pavelfeldman and yury-s June 25, 2026 15:05
@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from 2bac9db to 4d7c0b8 Compare June 25, 2026 15:06
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

return { target, frame: null, doc: null, x, y };
const frame = target as HTMLIFrameElement | HTMLFrameElement;
const frameRect = frame.getBoundingClientRect();
const frameStyle = frame.ownerDocument.defaultView!.getComputedStyle(frame);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This (and in other places) looks like a cross origin reach?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the ownerDocument is the ancestor of the <iframe> (as opposed to the contentDocument which is inside the <iframe>)

this is a replacement for a direct window.getComputedStyle(frame) which may not work since the window could be the wrong object if at this point the context is already inside an <iframe>

@dcrousso dcrousso requested a review from pavelfeldman June 26, 2026 02:34
if (active && (active.localName === 'iframe' || active.localName === 'frame')) {
let inner: Element | null = null;
try {
const doc = (active as HTMLIFrameElement | HTMLFrameElement).contentDocument;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't think the bulk of this change makes sense. Each frame has its own injected script with its own document and window object, available on the injected script itself. They are only ones, you can't switch between them in the injected script.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you're right that each <iframe> get its own injected script instance and that there's no way to just magically move stuff between them

the injected script will attempt drill down as deep as possible for the desired operation (e.g. positionInIFrame will elementFromPoint the desired coordinates, activeIFrame will look at the activeElement, etc.). if the result is an <iframe> (i.e. we've drilled down as deep as we can go) then include a handle for that as part of the reply alongside all of the other data computed so far (i.e. the local x, y, target, etc.)

the server will then look at the result to see if it's included the <iframe> and if so, it will then use the the protocol DOM.Node.contentDocument to cross the boundary and use that as the new execution context to repeat the above

(note that the code referenced in this comment was only to avoid the round-trip back to the server when dealing with same-origin <iframe>)

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from 4d7c0b8 to de15944 Compare July 1, 2026 23:42
@dcrousso dcrousso requested a review from pavelfeldman July 1, 2026 23:42
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from de15944 to 467ba5f Compare July 7, 2026 03:40
@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from 467ba5f to f0b3c02 Compare July 7, 2026 04:17
@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from f0b3c02 to bde5642 Compare July 7, 2026 04:34
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from bde5642 to f81977b Compare July 7, 2026 05:19
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from f81977b to 0e68b37 Compare July 7, 2026 16:25
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from 0e68b37 to ea8814c Compare July 7, 2026 17:38

// Descend through open shadow roots so synthetic events land on the actual
// element under the pointer rather than on the shadow host.
private _deepElementFromPoint(x: number, y: number): Element | null {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a readability regression - why removing a fine utility method instead of calling it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah i pushed before i committed undoing this change. my apologies

}

private async _dispatchWebViewInput(progress: Progress, method: string, params: any): Promise<void> {
if (method === 'mouseMove') {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You don't want to go from the strongly typed called, into dispatched and then start sniffing dispatched code. Stay in the strongly typed world and only start dispatching when crossing the injected script boundary, like previously.


private async _dispatchWebViewInput(progress: Progress, method: string, params: any): Promise<void> {
if (method === 'mouseMove') {
await this._dispatchMouseMove(progress, params);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is now quite hard to follow the mouse move dispatching - it has special handling in 3 layers

// Mouse move maintains hover state across frames: as the pointer crosses an <iframe>
// boundary it leaves the elements in one frame and enters elements in another.
private async _dispatchMouseMove(progress: Progress, params: any): Promise<void> {
const path = await this._pointerPath(progress, params.x as number, params.y as number);

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code should be in the RawMouseImpl, straightforward.

const childFrame = await this._childFrame(progress, iframe);
if (!childFrame)
break;
({ x, y } = await progress.race(position.evaluate(result => ({ x: result.x, y: result.y }))));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do not reassign method input params, keep things readable.

try {
return await progress.race(this.getContentFrame(iframe));
} finally {
iframe.dispose();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Either don't pass the ownership, or call childFrameAndDispose for clarity.

if (framesToHover.includes(hoveredFrame) || !frames.includes(hoveredFrame))
continue;
const context = await progress.race(hoveredFrame.mainContext());
await progress.race(context.evaluate(() => (globalThis as any).__pwWebViewInput.clearHover()));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This feels wrong, what if I just move the cursor 5 pixels to the right, without existing and elements? Should be no mouseout events, etc ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we havent moved out of any <iframe> then we wont hit this as every value in _lastHoveredFrames will be present in framesToHover

ill add a comment to clarify

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

dcrousso added 2 commits July 7, 2026 16:35
each `<iframe>` has its own mouse position

as such, moving the pointer into or out of an `<iframe>` did not dispatch `mouseleave`/`mouseout`/etc.

instead, track what `<iframe>` are hovered on the server so that movement is coordinated globally
@dcrousso dcrousso force-pushed the fix-WKWebView-cross-origin-iframe-input branch from ea8814c to 9bba18a Compare July 7, 2026 22:41
@dcrousso dcrousso requested a review from pavelfeldman July 7, 2026 22:41
@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Test results for "MCP"

3 failed
❌ [chrome] › mcp/cli-navigation.spec.ts:43 › tab-new with url @mcp-macos-latest-chrome
❌ [firefox] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate @mcp-macos-latest-firefox
❌ [firefox] › mcp/annotate.spec.ts:497 › should disengage annotate mode when --annotate client disconnects @mcp-macos-latest-firefox

7706 passed, 1235 skipped


Merge workflow run.

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

3 flaky ⚠️ [chromium-library] › library/chromium/oopif.spec.ts:282 › should click `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:680 › screencast › should capture full viewport on hidpi `@realtime-time-library-chromium-linux`
⚠️ [playwright-test] › ui-mode-test-output.spec.ts:118 › should collapse repeated console messages for test `@ubuntu-latest-node22`

49515 passed, 1164 skipped


Merge workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants