-
Notifications
You must be signed in to change notification settings - Fork 61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add semantics choice to getUserMedia() for in-browser picker. #667
Conversation
A few comments:
|
getusermedia.html
Outdated
<td> | ||
<p>When application-specified constraints do not widdle | ||
multiple choices between devices down to one per kind, the user | ||
agent MUST |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
and if it doesn't? Is prompting in that case a MUST NOT, a SHOULD NOT or MAY?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Otherwise," below kicks in. There wasn't a MUST there before, but I can add one.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be nice if we got as close as possible to not letting implementation-specific logic decide whether or not to prompt. Like with getDisplayMedia()... it always prompts. Can getUserMedia() always prompt no matter what?
When you lost audio during the meeting @youennf made a point of what if the user wants to share only audio and not video or vice versa? That may require further discussion but I like the idea of "user-chooses" to be "audio and/or video" so that you never have to reject and then re-prompt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Idea in the future "user-chooses" world.
- getUserMedia() always prompts. It gives audio and/or video, depending on user choice. "kind" is optional.
- If you want to change device mid-call, track.changeUserMedia() (or track.applyConstraints("user-chooses") if that makes transition easier) re-prompts the user for audio and/or video, depending on user choice.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can getUserMedia() always prompt no matter what?
No-one wants that. Users don't want to be prompted for their camera every time.
Many have only one camera, thus no need for a picker at all (ditto microphone).
Even with "user-chooses"
(a semantic we aspire to default to as sole modus operandi), it remains up to the app to request what it wants: a specific device, or any device fitting its constraints. A user agent picker would kick in (regardless of permission) only when that request encounters a user with multiple valid devices meeting those constraints.
I think that's the right division of concerns. App needs meet user needs.
The following then guarantees a picker, e.g. letting us replace in-content selection:
if (devices.filter(({kind}) => kind == "videoinput").length > 1) {
await navigator.mediaDevices.getUserMedia({
video: true, semantics: "user-chooses"
});
}
what if the user wants to share only audio and not video or vice versa?
Firefox used to do this, but we had to stop because web sites were expecting two tracks. I'd argue it's an inherent problem with the API, not solvable in the spec.
In fact, the spec algorithm is already written as concurrent steps per kind that "User agents are encouraged to bundle ... into a single user-facing permission prompt."
In fact, Hangouts already asks for camera and microphone separately, and I have a bug filed on Firefox somewhere to combine these requests into one prompt. That problem goes away then.
As I recall, we did not reach consensus on a transition plan last virtual interim, just option B for now. So there are no transition impacts from this PR, since it effectively defaults to This is the minimal consensus needed to replace labels and in-content selection.
Like it does today, count devices per kind using enumerateDevices. Although if constraints are used, this might narrow choices, and it could be hard to detect. Right now I think the API would succeed right away without a prompt in that case (assuming you already have access to the current device). Would you like it to fail? That would be a bigger semantic difference. The feature detection is there to let us start using this before labels go away.
It does not. Only when there's a choice. Otherwise, only a permission prompt is mandated. |
...otherwise users would instead see a permission prompt if permission is needed, or no prompt at all if the app already has permission. |
Summary of PR: Based on consensus in February call (Option A/B in slides). New semantics: navigator.mediaDevices.getUserMedia({video: constraints, semantics: "user-chooses"}); ...to work as follows:
Sites avoid picker on revisits by remembering w/ Use this to replace today's in-content device pickers. May also be used on initial call. |
@jan-ivar Please be sure any bug around this gets DDN, or file a bug against the MDN content (https://github.com/mdn/sprints/issues/new/choose) so we can be sure this change is documented. |
I just wanted to chime in that I have used a number of web-based video chatting apps over the last few weeks that don't offer me any choice of my camera (my setup has multiple cameras). Of course, those sites could have used APIs to enumerate my cameras and let me pick, but they didn't. And since that's left to the code, I the user am given no choice in the matter. I really wish a change could be made that would, without any changes to existing code, give me the option to pick or authorize a different camera. In other words, what I wish is that this "user-chooses" semantics was the default. Even if the browser UI still only let me pick/authorize a single camera (and not giving me the option to select/authorize multiple cameras and mics), at least I as a user would have the option in the browser to choose to force-expose a specific camera to the site rather than them getting to pick my default camera. |
@getify Is the browser tested Chrome or Chromium? |
Chrome (windows). It happens to me every other day or so, I have to connect to webchat service (like telehealth, whatever) and their UI doesn't let me choose what mic or camera I want... I assume JS wise they just grab the first camera and mic, but from my perspective it feels random... sometimes it's the right one of each, sometimes it's the right camera wrong mic, sometimes right mic wrong camera, sometimes both are wrong. Even though I go out of my way to set the default mic in windows, that's not always the mic that such a webchat app tries to pick up. That boggles my mind. But worse: my windows laptop doesn't let me set a "default camera" -- it just assumes my built-in camera is what I want to use -- though it almost never is! Chrome doesn't offer me any UI option to select which camera or mic are exposed to a web page. All I can do is just turn on or off camera/mic, but not select. If this "user-chooses" semantic isn't made the default, then this feature won't do anything to help me, because all those poorly designed webchat apps will never update their code to turn it on -- either they don't know about it or don't care. |
@getify Have not used Windows is some time. FWIW a workaround for Linux #693 (comment) to select audio devices other than default microphone. If Native File System is not enabled by default you can launch Chrome with |
@getify Per https://answers.microsoft.com/en-us/windows/forum/windows_10-hardware/how-to-change-default-webcam-in-windows-10/1e8377c8-4fd5-4e9c-8ab6-67913c5c8e0c you should be able to launch Control Panel to change the camera device and per https://docs.microsoft.com/en-us/windows/uwp/audio-video-camera/camera-profiles it should be possible to change the device progammatically. Either program can be launched in the browser session using the linked approach. |
@getify Another alternative is to file an issue with the service to resolve
which should be possible utilizing the current iteration (using
|
Agree. Not an issue at Firefox or Nightly. Perhaps Chromium authors will listen to you. Have been asking for this feature for some time in various ways, too many to list here, e.g., https://bugs.chromium.org/p/chromium/issues/detail?id=1032815; https://bugs.chromium.org/p/chromium/issues/detail?id=865799; for reasoning at Chrome, Chromium see https://bugs.chromium.org/p/chromium/issues/detail?id=1013881#c9. Eventually decided to stop asking at specifications or implementers and create code that achieves the expected result. |
Moved to w3c/mediacapture-extensions#9. |
Initial version w/user-chooses semantics from w3c/mediacapture-main#667
Fixes https://github.com/w3c/mediacapture-main/issues/652