Use webdriver to set permission on getUserMedia calls #31847

dontcallmedom · 2021-12-02T14:54:17Z

this includes removing the default fake-ui from chrome wpt runner - this makes obtaining permission a prerequisite for using getUserMedia in tests.

Rely instead of SetPermission webdriver hook; this allows testing permission denial.

Rewrite mediacapture-streams using async/await for consistency in the process Move a couple of manual tests back to automated as a result

reillyeon

Overall this change seems reasonable but I only took a close look at mediacapture-image/ImageCapture-creation.https.html since that's the file I'm an owner for.

alvestrand · 2021-12-06T21:30:47Z

mediacapture-record/MediaRecorder-bitrate.https.html

@@ -5,6 +5,9 @@
 <link rel="help" href="https://w3c.github.io/mediacapture-record/MediaRecorder.html">
 <script src="/resources/testharness.js"></script>
 <script src="/resources/testharnessreport.js"></script>
+<script src=/resources/testdriver.js></script>
+<script src=/resources/testdriver-vendor.js></script>
+<script src='../mediacapture-streams/permission-helper.js'></script>


can we please have one convention on how filenames are quoted in "script src=" statements?
This makes 3 different versions in a single file.

alvestrand · 2021-12-06T21:32:23Z

mediacapture-streams/MediaStream-MediaElement-preload-none.https.html

+<html>
+    <head>
+        <title>Test that the HTMLMediaElement preload 'none' attribute value is ignored for MediaStream used as srcObject and MediaStream object URLs used as src.</title>
+        <link rel="author" title="Matthew Wolenetz" href="mailto:wolenetz@chromium.org"/>


This file seems like it belongs to a different PR.

that file was a -manual file, presumably because when it was created, one would not gain access to getUserMedia. Now that we do, it feels reasonable to make the move here.

alvestrand · 2021-12-06T21:35:26Z

mediacapture-streams/permission-helper.js

+    for (let s of scope) {
+      await test_driver.set_permission({ name: s }, status, true);
+    }
+  } catch {


generic catch is usually a bad idea. What error do we expect here?

this is to deal with browsers that don't implement the set_permission webdriver command (i.e. both Firefox and Safari at this time point).

so we should catch "no such function". I'm worried about the case where it fails for an unrelated reason (such as calling with "scope=['cammera']" and getting a "no such value" exception - that could be hard to debug when the error is caught and ignored).

alvestrand · 2021-12-06T21:36:56Z

mediacapture-streams/permission-helper.js

@@ -0,0 +1,10 @@
+async function setPermission(status="granted", scope=["camera", "microphone"]) {


the "status" argument is never used. Is it a future goal to be able to set setPermission(status=denied, scope=["camera"])? If so, document (or even better, implement).

it's used in GUM-deny.html

alvestrand · 2021-12-06T21:45:18Z

So the harness-setting-permission function has finally landed? Great!

A couple of things:

the name setPermission() is a bad idea. setMediaDevicePermission() is longer, but won't conflict with everyone else's helpers for setting their particular permissions. When we go to tests that mix mulitple interfaces with separate permissions they want to set, it's important that they don't get to clobber each other.
setting permission for camera when microphone only is needed by the test is sloppy and will fail to detect errors involving checking the wrong permission
this is a murderous lot of file changes for doing the same thing in multiple places. Would it be possible to wrap this inside a helper function rather than repeating the same 3 lines over and over again?
I can't see that this PR actually removes the Chrome flag. So the tests will pass on Chrome whether or not the test driver function exists or not. Is this intentional, or is it "delayed to the next PR"?

dontcallmedom · 2021-12-07T07:43:05Z

* the name setPermission() is a bad idea. setMediaDevicePermission() is longer, but won't conflict with everyone else's helpers for setting their particular permissions. When we go to tests that mix mulitple interfaces with separate permissions they want to set, it's important that they don't get to clobber each other.

Will fix to setMediaDevicePermission()

* setting permission for camera when microphone only is needed by the test is sloppy and will fail to detect errors involving checking the wrong permission

I can take a pass at setting permissions more specifically.

* this is a murderous lot of file changes for doing the same thing in multiple places. Would it be possible to wrap this inside a helper function rather than repeating the same 3 lines over and over again?

I sure wish it was; but doing this cleanly would require using JS modules, which I don't think the WPT infrastructure is ready for at this point.

* I can't see that this PR actually removes the Chrome flag. So the tests will pass on Chrome whether or not the test driver function exists or not. Is this intentional, or is it "delayed to the next PR"?

It does remove the flag, see https://github.com/web-platform-tests/wpt/pull/31847/files#diff-0df24b5b583c460182e687f7dc7a6a79dd2cd3389bc4a96f48483f60fceb51f7

per web-platform-tests#31847 (review)

per web-platform-tests#31847 (comment)

alvestrand

This is a great step forward in testing permission-gated functionality!

per web-platform-tests#31847 (review)

per web-platform-tests#31847 (comment)

…erMedia calls , a=testonly Automatic update from web-platform-tests Use webdriver to set permission on getUserMedia calls (#31847) * Remove fake-ui from chrome wptrunner Rely instead of SetPermission webdriver hook; this allows testing permission denial. * Use webdriver to set permission on getUserMedia calls Rewrite mediacapture-streams using async/await for consistency in the process Move a couple of manual tests back to automated as a result * Catch web driver error more specifically when setting permission per web-platform-tests/wpt#31847 (review) * Rename setPermission in setMediaPermission per web-platform-tests/wpt#31847 (comment) * Limit media capture permissions to what is strictly needed for the test per web-platform-tests/wpt#31847 (comment) * Align quoting styles for script loading -- wpt-commits: 319a0a7580aaee957801545901150a0dabd9e735 wpt-pr: 31847

wpt-pr-bot added mediacapture-streams wg-s_webrtc labels Dec 2, 2021

wpt-pr-bot assigned youennf Dec 2, 2021

wpt-pr-bot requested review from agouaillard, alvestrand, jan-ivar and youennf December 2, 2021 14:54

dontcallmedom force-pushed the gum-manual branch from ebecf02 to 93e8800 Compare December 3, 2021 13:12

wpt-pr-bot added infra wptrunner The automated test runner, commonly called through ./wpt run labels Dec 3, 2021

wpt-pr-bot requested review from foolip and jgraham December 3, 2021 13:12

Remove fake-ui from chrome wptrunner

bab57b1

Rely instead of SetPermission webdriver hook; this allows testing permission denial.

dontcallmedom force-pushed the gum-manual branch from 93e8800 to 1f8ee12 Compare December 3, 2021 13:16

dontcallmedom marked this pull request as draft December 3, 2021 13:19

dontcallmedom force-pushed the gum-manual branch 7 times, most recently from 5eac0e6 to 125fa16 Compare December 6, 2021 10:30

Use webdriver to set permission on getUserMedia calls

ea93b94

Rewrite mediacapture-streams using async/await for consistency in the process Move a couple of manual tests back to automated as a result

dontcallmedom force-pushed the gum-manual branch from 125fa16 to ea93b94 Compare December 6, 2021 15:04

dontcallmedom changed the title ~~Use test_driver to automate permission deny with getUserMedia~~ Use webdriver to set permission on getUserMedia calls Dec 6, 2021

dontcallmedom marked this pull request as ready for review December 6, 2021 20:36

dontcallmedom removed the request for review from agouaillard December 6, 2021 20:36

wpt-pr-bot added audio-output feature-policy mediacapture-image labels Dec 6, 2021

wpt-pr-bot added webrtc webrtc-encoded-transform wg-webappsec labels Dec 6, 2021

wpt-pr-bot requested review from bakulf, clelland, guidou, henbos, reillyeon, rwaldron, snuggs and yellowdoge December 6, 2021 20:37

reillyeon approved these changes Dec 6, 2021

View reviewed changes

alvestrand reviewed Dec 6, 2021

View reviewed changes

dontcallmedom added a commit to dontcallmedom/web-platform-tests that referenced this pull request Dec 7, 2021

Catch web driver error more specifically when setting permission

709469d

per web-platform-tests#31847 (review)

dontcallmedom added a commit to dontcallmedom/web-platform-tests that referenced this pull request Dec 7, 2021

Rename setPermission in setMediaPermission

94b21e2

per web-platform-tests#31847 (comment)

dontcallmedom added a commit to dontcallmedom/web-platform-tests that referenced this pull request Dec 7, 2021

Limit media capture permissions to what is strictly needed for the test

2deda0b

per web-platform-tests#31847 (comment)

alvestrand approved these changes Dec 7, 2021

View reviewed changes

dontcallmedom added 4 commits December 7, 2021 12:08

Catch web driver error more specifically when setting permission

b57c02c

per web-platform-tests#31847 (review)

Rename setPermission in setMediaPermission

a70eb08

per web-platform-tests#31847 (comment)

Limit media capture permissions to what is strictly needed for the test

3657345

per web-platform-tests#31847 (comment)

Align quoting styles for script loading

328aa26

dontcallmedom force-pushed the gum-manual branch from 13a254a to 328aa26 Compare December 7, 2021 11:08

dontcallmedom merged commit 319a0a7 into web-platform-tests:master Dec 7, 2021

dontcallmedom deleted the gum-manual branch December 7, 2021 18:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use webdriver to set permission on getUserMedia calls #31847

Use webdriver to set permission on getUserMedia calls #31847

dontcallmedom commented Dec 2, 2021 •

edited

Loading

reillyeon left a comment

alvestrand Dec 6, 2021

alvestrand Dec 6, 2021

dontcallmedom Dec 7, 2021

alvestrand Dec 6, 2021

dontcallmedom Dec 7, 2021

alvestrand Dec 7, 2021

alvestrand Dec 6, 2021

dontcallmedom Dec 7, 2021

alvestrand commented Dec 6, 2021

dontcallmedom commented Dec 7, 2021

alvestrand left a comment

		@@ -0,0 +1,10 @@
		async function setPermission(status="granted", scope=["camera", "microphone"]) {

Use webdriver to set permission on getUserMedia calls #31847

Use webdriver to set permission on getUserMedia calls #31847

Conversation

dontcallmedom commented Dec 2, 2021 • edited Loading

reillyeon left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alvestrand commented Dec 6, 2021

dontcallmedom commented Dec 7, 2021

alvestrand left a comment

Choose a reason for hiding this comment

dontcallmedom commented Dec 2, 2021 •

edited

Loading