CTA-5000-A (MSE, Audio)

[haudiobe]

when revisiting the CTA-5000-A, I recognized that we have 4 references in the media specification.
Devices MUST be conforming implementations of the following specifications:
• Encrypted Media Extensions [ENCRYPTED-MEDIA]
• Media Source Extensions [MEDIA-SOURCE]
• Scalable Vector Graphics (SVG) 1.1 (Second Edition) [SVG11]
• Web Audio API [WEBAUDIO]
• Exceptions: Since not all environments currently support Media Streams [MEDIACAPTURE-STREAMS], MediaStreamAudioSourceNode and MediaStreamAudioDestinationNode are not yet widely supported.

I have two main questions:

• On Media Source Extensions, we do not limit this to the ISO BMFF byte stream format. Is this intentional? Or said differently, should it not mandate the support of ISO BMFF byte stream format
• On Web Audio API:
  o Has anyone checked how the Web Audio API interacts with our audio profiles?
  o Does W3C has tests for the Web Audio API that may be combined reused for DPCTF tests?

[squapp]

On Web Audio API:
o Has anyone checked how the Web Audio API interacts with our audio profiles?

just for clarification: Web Audio API is about processing/modifying audio that is played back via HTML media element (and MSE). I believe you're asking "Has anyone checked how MSE interacts with our audio profiles?", correct?

o Does W3C has tests for the Web Audio API that may be combined reused for DPCTF tests?

Web Audio tests included in WMATS18 are here: https://github.com/cta-wave/WMAS/tree/wmas2018-tests/webaudio

MSE tests are here: https://github.com/cta-wave/WMAS/tree/wmas2018-tests/media-source
They include a couple of audio-only AAC and Vorbis files

[JohnRiv]

Why not leave everything audio & video specific to the Device Playback Capabilities Task Force?

[JohnRiv]

@haudiobe & @squapp, it seems that it does in fact make sense to address this within the HATF.

As this has been open for a few months now, do either of you have any further comments on this?

[jpiesing]

just for clarification: Web Audio API is about processing/modifying audio that is played back via HTML media element (and MSE). I believe you're asking "Has anyone checked how MSE interacts with our audio profiles?", correct?

I think the question that is actually being asked is this - what audio profiles are supported with the Web Audio API & are these consistent with the WAVE audio profiles?

The Web Audio API spec requires support for PCM audio with the AudioBuffer and I've seen MP3 (MP1_L3) required for (Base)AudioContext.decodeAudioData().

I don't believe either of these are in our content spec. Neither of them are used with DASH.

[jpiesing]

Just to be clear, I don't believe it's a problem if the Web Audio API supports audio formats not in the WAVE content spec. The WAVE content spec is all about CMAF and streaming interop. In my experience, the main uses of the Web Audio API are not related to streaming. For example, adding sound effects to a UI, e.g. for accessibility.

For the first part of the original problem ...

On Media Source Extensions, we do not limit this to the ISO BMFF byte stream format. Is this intentional? Or said differently, should it not mandate the support of ISO BMFF byte stream format

I agree. It's reasonable to mandate support for the ISO BMFF byte stream format.

[JohnRiv]

As discussed during the CTA Fall Forum, we will not make any updates to the spec regarding this issue, as the Web Audio API is supported across the modern browsers, and regarding ISO BMFF, any tests that don't pass across all 4 modern browsers are not included in the test suite. Any specific details around support for ISO BMFF should be covered in DPCTF.