test_driver.click(button) doesn't work in iframes

[jan-ivar]

The awesome user gesture automation tool test_driver.click() added in #6897, unfortunately doesn't work in iframes, which blocks us from testing the "display-capture" feature-policy, because getDisplayMedia requires user activation.

cc @gsnedders @jgraham

[gsnedders]

#17260 should have made it work, I thought. Maybe it doesn't, maybe we regressed it. :(

[jan-ivar]

@gsnedders We're hitting this line:

wpt/resources/testdriver.js

Line 99 in bdad828

return Promise.reject(new Error("can only click in top-level window"));

[jan-ivar]

And I tried removing that line, but it then causes timeouts—presumably from missing buttons?

TIMEOUT Default "display-capture" feature policy ["self"] allows same-origin iframes. - Test timed out
TIMEOUT Default "display-capture" feature policy ["self"] disallows cross-origin iframes. - Test timed out
TIMEOUT Feature policy "display-capture" can be enabled in cross-origin iframes using "allow" attribute. - Test timed out
TIMEOUT /screen-capture/feature-policy.https.html

They used to pass before we added user gesture in Firefox.

[gsnedders]

Well that doesn't seem very useful. We should fix this more usefully. 🙃

[foolip]

In https://chromium-review.googlesource.com/c/chromium/src/+/2283968/14/third_party/blink/web_tests/external/wpt/resources/chromium/webxr-test.js there's a workaround for this issue, making the setup non-cross-browser.

That seems unfortunate.

@stephenmcgruer have you come across other tests working around this? I'm wondering if we might have accumulated enough cases to make this worth prioritizing.

[klausw]

FWIW, I had copied this non-cross-browser setup from Chromium's web_tests/fullscreen/trusted-click.js, it's used by several fullscreen tests there. It looks as if these correspond to multiple manual fullscreen tests in WPT.

[klausw]

Correction, my workaround was based on a simplified version that just uses eventSender to click at coordinates (0, 0) which was good enough for this specific use case. A more general solution would get more complicated if it needs to calculate specific coordinates for elements, especially since AFAIK a cross-origin iframe can't portably determine its position within the parent. The Chromium trusted_click function takes a clickRectInRootFrameCoordinate argument to explicitly pass this in.

(I had experimented with using sendMessage to store the iframe position in an ad-hoc window property that can be read by simulateUserActivation, but that seems rather hackish.)

[stephenmcgruer]

This came up in discussion with @jgraham today too. It seems like at some point some support was added specifically for actions in testdriver.js - a9d6d1b - but not for any other sort of interaction.

I may be over-looking something important, but it seems like we could just naively do a switch-to-frame, do-command, switch-back approach for every testdriver.js interaction and that would work? Would need some investigation and debugging to prove it out.

[jgraham]

Yes, I think for frames that would work. For windows it's a bit more complex because currently it's a pain to get the window handle for a given dom window (WebDriver supports this but implementations don't). It's also a pain to start an interaction from outside the main window; if we want to support that we'll probably need some API surface for postMessaging actions up to the main window.

[gsnedders]

And as @jgraham pointed out on IRC, noopener also makes windows harder.

[jgraham]

This is now fixed for all cases where the test window and window where the interaction is taking place can get handles to each other (i.e. excluding rel=noopener and similar).