Support AdaptationSet@group

[joeyparrish]

DASH IOP says encrypted content where SD/HD/UHD have different keys, they should be in separate AdaptationSets. To support this, we need to support the group attribute to group AdaptationSets together logically. The player should be able to adapt between AdaptationSets that are in the same group.

Protected content throws another wrinkle into this. We should be able to tolerate a license failure for a particular AdaptationSet, while still making use of other AdaptationSets in the same group. For this to work smoothly, sets should be sorted by resolution and only the 0th should be used initially. Licenses should be requested for all sets. As license requests succeed for other sets, they can be made available for adaptation.

[joeyparrish]

Alternately, we can start making use of key statuses from EME and default key IDs from the manifest to determine what we can and can't play before trying to. This would be much simpler, but I'm not sure what the fallback would be if either feature was missing. Also not sure if this would cover all bases regardless of what key system is in use, or if we still would have to fetch licenses and tolerate failures in some cases.

[sanbornhilland]

We have a manifest with 3 video adaptation sets all with group=2 and they all have the same protection. I can see that the mpd_processor is correctly grouping them into the same StreamSetInfo but I notice that because of this:

If the streams are encrypted then assume the key system can probably
decrypt the lowest quality streams. This enables the VideoSource to
select at least one stream for stream startup without having to query the
key system first.

The lowest quality streams are the streams that are contained within
the AdaptationSet which contains THE lowest quality stream.`

Only the representations in the first adaptation set have allowedByKeySystem=true initially.

I thought that on keystatuseschanged these other adaptation sets in the group would end up having allowedByKeySystem set to true in shaka.player.StreamVideoSource.prototype.onKeyStatusesChange.

But it looks like this: streamInfo.allowedByKeySystem = !message on line 959 of stream_video_source is never getting hit so they are never being exposed as available tracks.

Can you help to clarify this situation for me and why this is happening. I'm still hazy on the DASH spec and it's unclear to me if having multiple adaptation sets in the same group with the same content protection is unusual/non-standards-compliant or if this was an unanticipated scenario and a bug in Shaka.

[joeyparrish]

The usefulness of AS groups, as far as we are concerned, is to group together AdaptationSets and adapt between them. Otherwise, you would only adapt within a single AdaptationSet.

The DASH interoperability (IOP) docs, as we understand them, state that ContentProtection should only exist at the AdaptationSet level, and that all Representations within an AdaptationSet must have the same key.

Therefore, if you want to use different keys for SD/HD/UHD, and you also want to be DASH-IOP-compliant, you have to use groups. Supporting this scenario is tricky for us, and requires the use of keystatuschanged events to determine whether or not we have keys for the higher-resolution parts of the group.

Finally, the only way to correlate the key statuses with an AdaptationSet is through the cenc:default_KID element inside ContentProtection. Otherwise, we don't have any way to know in advance what key IDs the different parts of the group will use, and we can't enable them.

So here's the checklist I would suggest:

1. Do you use different keys to encrypt the AdaptationSets in the group? If not, combine them into one AdaptationSet and stop using groups.
2. If you are using groups, do you have cenc:default_KID inside of ContentProtection? If not, group support can't function correctly, so you'll need to add these.

[sanbornhilland]

We are using cenc:default_KID but this has the same value for each of the Adaptation Sets within the same group and it seems that Shaka then ends up ignoring the adaptation sets with the higher bandwidths onkeystatuseschanged. From what I understand of your explanation above though it sounds like unless we have different keys there is no reason not to be putting all of our Representations into a single Adaptation Set.

[joeyparrish]

Correct. If all the codecs and key IDs are the same, there's no reason to split up your AdaptationSets.

To enable debug logging, either run shaka.log.setLevel(shaka.log.Levels.DEBUG) in the app or add ?debug in the demo app URI. With this enabled, do you see a message that says "Key blah was not specified in the manifest"? This would indicate that you have the wrong key ID in your manifest, which would also explain the symptoms you're seeing.

If this message does not occur, then we may have a bug. The code, as written, should still enable streams from other parts of the group once the license is processed.

Please let us know what you find out.

[sanbornhilland]

Yes I was seeing the message "Key... was not specified in the manifest". However, I am confused by what this is supposed to mean because if I check for the keyId that it is referring to I can see that it is the default key in each adaptation set.

[joeyparrish]

If you have cenc:default_KID in the manifest, but it's not in the stream info, then perhaps the issue is somewhere in the manifest parser/processor. Can you file a new issue and include a manifest which reproduces the issue?