Handling of contradictory hints

[eladalon1983]

It is possible for applications to supply contradictory hints when invoking getDisplayMedia(). For example:

// 1.
// Audio generally not requested, but system-audio marked as desired.
navigator.mediaDevices.getDisplayMedia({
  audio: false,
  systemAudio: "include",
});

// 2.
// Audio requested, including an explicit request for system-audio,
// but monitors asked to be excluded. (Monitors are the surface type
// associated with system-audio.)
navigator.mediaDevices.getDisplayMedia({
  audio: true,
  systemAudio: "include",
  monitorTypeSurfaces: "exclude"
});

// 3.
// Application requested monitors to be displayed most prominently,
// while simultaneously asking for monitors to not be offered.
navigator.mediaDevices.getDisplayMedia({
  video: {
    displaySurface: "monitor"
  },
  monitorTypeSurfaces: "exclude"
});

I think any of these is indicative of a logical error in the Web application, and had better be handled by rejecting the gDM call.

Wdyt?

[jan-ivar]

My concerns:

1. Throwing on options that some browsers may parse and others may not adds unnecessary web compat headaches
2. A parsing requirement would make implementations non-conformant (these are optional to implement modulo #290)
3. "Preferences" are generally understood to be just that — saved and not tailored to every single use of them
4. A preference "conflict" in one browser may make sense in another

E.g. I would prefer a mint on my pillow, but don't cancel my scuba diving trip over it.

Since preferences are optional to implement, behavior should match that of non-implementing browsers. E.g.

// 1.
// Audio generally not requested, but system-audio marked as desired.
navigator.mediaDevices.getDisplayMedia({
  audio: false,
  systemAudio: "include",
});

...should act the same as navigator.mediaDevices.getDisplayMedia().

// 2.
// Audio requested, including an explicit request for system-audio,
// but monitors asked to be excluded. (Monitors are the surface type
// associated with system-audio.)
navigator.mediaDevices.getDisplayMedia({
  audio: true,
  systemAudio: "include",
  monitorTypeSurfaces: "exclude"
});

Since systemAudio is a sub-option "for monitor display surfaces." there's no logical conflict. I want zero apples, all green.

// 3.
// Application requested monitors to be displayed most prominently,
// while simultaneously asking for monitors to not be offered.
navigator.mediaDevices.getDisplayMedia({
  video: {
    displaySurface: "monitor"
  },
  monitorTypeSurfaces: "exclude"
});

Since "The user agent MAY use the presence of the displaySurface constraint and its value to influence the presentation" there's no logical conflict. Monitor types will be excluded IF this browser supports it, and put up front if not and that is supported.

Not every combination needs to be useful as long as it's deterministic.

[dontcallmedom-bot]

This issue had an associated resolution in WebRTC December 12 2023 meeting – 12 December 2023 (Issue #276: Handling of contradictory hints):

RESOLUTION: Consensus to specify an interoperable behavior and iterate initially on a pull request to be proposed by Elad

[eladalon1983]

1. Throwing on options that some browsers may parse and others may not adds unnecessary web compat headaches

This issue would only affect the user agents that did implement the spec. Those that don't - like the current versions of Safari and Firefox - won't be adversely affected.

2. A parsing requirement would make implementations non-conformant (these are optional to implement modulo #290)

Would you prefer it if we used MAY?
Or possibly we could say that the UA MUST throw if it implemented the hints and MUST NOT otherwise.

3. "Preferences" are generally understood to be just that — saved and not tailored to every single use of them

If the UA does not warn developers when they provide self-contradictory sets of preferences, the behavior the UA exhibits will be surprising to some of them. And note that users will be impacted too when developers fail to correct their programs to exhibit scenario-appropriate behaviors.

4. A preference "conflict" in one browser may make sense in another

In theory, about the future, maybe.
In practice, looking at the exact cases I have shown here, no. (Think otherwise? Let me know.)

I want zero apples, all green.

One of my kids is 3 years old. If I gave him a bag with "zero candy, all of your favorite type," he would be upset. Given his age, I would even add "rightfully upset" - why would I mislead him like that? And this applies to developers too. Bugs happen when constraints+options are programmatically set. It's better to fail loudly than silently.

Not every combination needs to be useful as long as it's deterministic.

It's more deterministic across browsers and browser versions if we specify what happens.