MediaTrackCapabilities discovery via enumerateDevices()

[stefhak]

This proposal was recently sent to the list (and should be discussed there), filing an Issue to keep track of any decisions: https://lists.w3.org/Archives/Public/public-media-capture/2015Jun/0034.html

Summary:

The proposed change will allow the MediaDeviceInfos in the array returned by enumerateDevices() to include a new capabilities attribute of type MediaTrackCapabilities, available only in the already-defined case where anti-fingerprinting filtering is not applied.

Rationale:

It is not possible for audio or video applications today to offer the user a choice of available devices that takes account of their capabilities or characteristics, e.g. channel count, pixel dimensions, sample rate, etc.

While it is possible to enumerate devices and obtain their names, kinds and opaque IDs, applications also need to a) filter device lists to only include usage-appropriate entries (e.g. only audio inputs >= 44k sample rate), or b) display useful information to accompany each device's name (e.g. whether it's mono, stereo, etc.). Other use cases also exist.

The existing Media Capture API proposal already addresses potential fingerprinting exploits by prohibiting the return of device labels unless the user has explicitly granted access to at least one device. This proposal would apply the same restriction to the return of device capabilities.

Proposed changes:

• Add the following attribute to MediaDeviceInfo:

  readonly attribute MediaTrackCapabilities capabilities;

Its definition is as follows:

• If access to this information is disallowed to the caller, the attribute's value is null.
• Otherwise, the attribute contains a MediaTrackCapabilities object identical to that which would be obtained by the following algorithm:
  • Let info be the MediaDeviceInfo whose capabilities attribute is being determined.
  • If info.type is "audioinput" or "videoinput",
    • Call getUserMedia({deviceId: info.deviceId})
    • If the promise succeeds
      • Let track be the first MediaStreamTrack in the returned MediaStream whose type is "audio" (if the device type was "audioinput") or "video" (if the device type was "videoinput").
      • Assign the attribute's value from track.getCapabilities()
  • Otherwise if info.type is "audiooutput" the attribute's value is a MediaTrackCapabilities describing the capabilities of the audio output device (if these can be obtained) or else null.
• Change the description of enumerateDevices() in 9.2.2 as follows:

In step 4, to the text " let filteredList be a copy of resultList, and all its elements, where the label member is the empty string", append "and the capabilities member is null".

• Change the text of section 9.2.3 (Access control model) as follows:

Modify the text "If access has been granted for a media device, the MediaDeviceInfo dictionary will contain the deviceId, kind, label, and groupId" to read "If access has been granted for a media device, the MediaDeviceInfo dictionary will contain the deviceId, kind, label, groupId and capabilities."

[stefhak]

This has been on the list, and there were no objection. As a result PR #211 was created.

[alvestrand]

Fixed by #211