v3.24.0

Release v3.24.0 (2021-04-01)

(No, it's not an april fools joke, unless it doesn't work, in which case we will pretend it was)

A new release is here! Amongst the new features and improvements:

• a new inbandEvent event is now triggered when event metadata is encountered in a segment (for example through "emsg" boxes in an ISOBMFF segment), just before that segment is pushed

• a new singleLicensePer option now allows to optimize the loading of contents with multiple decryption keys, to be able to perform only a single license request.

• we improved a lot the compatibility of the RxPlayer with several devices, most importantly many Tizen versions (e.g. Samsung TVs) and the PlayStation 4 (as to the PlayStation 5, it should already be handled).

• multiple optimizations have been added to allow lower content loading times

• many other bug fixes and improvements (you can see the changelog at the bottom of this release note)

A new player event: inbandEvent

"Inband" events

Streaming formats specify different ways to emit events through a content.
Those events can be sent at any time when streaming the content, and can be used to signal generic or format-specific information to a player or the application (e.g. signaling about parental ratings on the current content or about the Manifest's validity).

The RxPlayer already supported one kind of event: the MPEG-DASH manifest EventStream node, allowing an event to be declared directly in the Manifest.
But instead of only having events in the manifest, those can also be found inside media segments.

MPEG-DASH uses such format, by exploiting the "emsg" box of the ISOBMFF standard. This box is included in the metadata part of ISOBMFF segments (as in: it is not included in the media data itself) and is usually related to the media presentation time.

Those type of events, which are included directly in the segments, are usually called "inband events".

In the RxPlayer: inbandEvents event

The RxPlayer now parses these boxes (only the version 0 of it for now) and :

• Either the event has a specific meaning for a player (generally to give information
 about the manifest validity, allowing the player to consider when it may refresh the manifest) in which case it will handled it by itself.
• Either, the event is generic, or specific but not yet handled by the player. In which case it will be parsed and emitted 
by the RxPlayer through an event.

This new event is called "inbandEvents".
It emits one or more parsed inband events, as several "emsg" boxes may be found when parsing a unique media segment.

The "inbandEvents" event is emitted just after the corresponding segment is parsed, unlike the already-existing "streamEvent", which is sent at the presentation time of the concerned event.

And as an "emsg" box is associated with a specific segment, the player send it at the moment it gets it, before that segment is actually pushed.
This is important as it may be used to signal that the current segment contains content that may be subject to rating restrictions.
In that case, the client may need to interrupt playback before the segment's data is buffered.

This new event and its format is documented here.

singleLicensePer option: single license request for contents with multiple keys

Previous situation

When playing encrypted contents, the RxPlayer has a simple logic to know when it needs to perform a licence request:

1. Let's say that the player switches to a new Representation/quality
2. It looks at that quality's linked encryption initialization data if it exists:
   • if it was already encountered, it does nothing in particular with it
   • if it wasn't yet encountered, it goes to the next step
3. The player creates a new decryption session (called MediaKeySession) associated to that data and ask that session to generate a license request with it
4. The browser sends the player an event back with a "challenge" allowing it to perform a license request by calling the getLicense callback
5. Your application fetches the license and give it to the RxPlayer, which then pass it down through the MediaKeySession.update API
6. The RxPlayer then looks at which keys were contained in that license and reacts accordingly (it might fallback for unsupported qualities etc.)

This works pretty well, but it generally leads to multiple license requests when a content has different keys depending on the quality chosen.

This is because Representations linked to different keys will generally have different initialization data. The RxPlayer will thus create a new MediaKeySession and perform a new license request for each of them.

Optimization: performing only a single license request

We've looked at possible ways to optimize this logic. The most evident solution is to only perform a single license request.

Yet that would mean that the license returned contains all needed keys.
As such, this optimization relies on two things:

1. the license server has to be aware of this "trick" and return all keys even when a single one is asked for
2. the RxPlayer has to only perform a single license request.
   It could still technically perform multiple ones but this would be pointless as the first one already gave everything needed.

For that second part, we added the new singleLicensePer properties to the keySystems option of the loadVideo API.
By setting it to "content" (more possible values might come in the future), you can signal that there's only a single license available for the whole content.

It can be used as such:

rxPlayer.loadVideo({
  // ...
  keySystems: [{
    // ...
    singleLicensePer: "content",
  }]
}

This option is documented inside the keySystems documentation.

Remaining problem: what about missing keys?

But if the RxPlayer only decided to perform a single license request and did not change anything else in its behavior, it might lead to an other issue: what should happen if one of the keys is not in the license?

This might happen when the CDM, the module plugged in the browser allowing content decryption, is not trusted enough by the license server, resulting in the latter withholding some keys (the more restrictive ones).

The decision we've made was that, in the case an awaited decryption key isn't present in the pushed license, the player will automatically decide to mark the corresponding Representations as "undecipherable".

In this situation, those qualities won't be played. If segments corresponding to that quality were already pushed (we load media segments in parallel of the license request as an optimization measure), the RxPlayer will first try to remove them and might in some very rare conditions switch to the "RELOADING"state while doing so.

Improved compatibility with several platforms

At Canal+ Group, we recently had a wave of new platforms on which the RxPlayer was ported (I'll just name-drop the PlayStation 5, PlayStation 4 and Samsung TVs but there's others).
This was the occasion to improve our records on portability.

The main issues we encounter when porting the RxPlayer are subtle browser behavior differences, sometimes still specification-compliant and sometimes not, most of them being related to their respective MSE/EME implementations.

There was a lot of changes on our side (and multiple times on the implementer's side! We don't always want to fix another application's mess :p).

Most notably, they're fixing:

• A mysterious infinite loading situation when beginning a live content on some platforms (amongst which, Samsung TVs), most likely due to an overflow on the browser-side after setting the duration (#914)
• A situation where both the browser and the RxPlayer wanted to seek at different times, leading to a conflict (#922)
• decoding issues we encountered when pushing segments on top of the current position (#925)
• A leak of MediaKeySession on platforms where the generateRequest API is especially long (#920)
• Persistent MediaKeySessions being thrown away because too much time passed before the player was aware of its linked keys (#928, #926)

Loading time improvements

In this release, we also began (and still continuing) trying to heavily optimize the initial content loading time:

• One of the main improvements is that now when calling loadVideo when a content was already playing, the new content's manifest will be loaded in parallel of the old content being stopped (implementation details: #894)

• Another one, only for encrypted contents, is to depend only on the encryption initialization data in the Manifest if it already contains the right one (instead of the default behavior, which waits for a segment to be fetched first).

  This optimization was a complex one to implement because it implied recognizing which encryption initialization data is needed, whereas the previous implementation just used all available data - a solution which always work (implementation details: #911, #919).

  If you rely on persistent licenses, this optimization won't be available to you by default. You will need to set the disableRetroCompatibility property to true in the licenseStorage option (itself part of the keySystems loadVideo option). Please note however that in this case, you might not be able to load drm sessions persisted through an older version of the RxPlayer.
  As usual all keySystems options are documented here.

• Also for encrypted contents, the singleLicensePer option documented above allows to perform only one license requests instead of many (implementation details: #904)

• To reduce loading time on CPU-constrained devices, we removed the logic reacting to the HTML5 "progress" event, as it should be unnecessary (implementation details: #893)

• When a serverCertificate is given through the keySystems option of loadVideo, the RxPlayer will now set it much sooner. Before it waited for the first encryption initialization data to be encountered first, but this should be unnecessary (implementation details: #895)

The amount of loading time all these modifications save is highly dependent on the options you use and on the tested platform.
We're however most likely not speaking about seconds here but most likely in the hundreds of milliseconds in the worst encountered case, which is still a plus!

Changelog

Features

• Add inbandEvent event for when an event is encountered in a media segment [#892]
• DRM: Add singleLicensePer keySystems option to be able to signal in advance that the current content has a single license, even if it has multiple decryption keys [#863, #904]
• DRM: Add keySystems[].licenseStorage.disableRetroCompatibility boolean to unlock optimizations when compatibility with EME sessions persisted in older RxPlayer versions is not important [#919]

Bug fixes

• DASH: Fix rounding error that could cause infinite buffering issues when going from a DASH Period to the next [#897, #899]
• DRM: Always pass on server certificate before any challenge is generated. Contents with multiple licenses previously could lead to the latter being done before the former. [#895]
• DRM: Fix possible leaks of MediaKeySessions if closed at the wrong time [#920]
• Fix issue making sudden and acute fall in bandwidth not being considered soon enough [#906]
• On some devices when maxBufferAhead is set, avoid removing the initially loaded data if done before the initial seek could be performed [#907]
• Avoid cases of infinite rebuffering on Tizen 4 by avoiding pushing segments "on top of others" too close to the current position [#925]
• Avoid seeking issues on Tizen by not seeking over discontinuities that will be already handled by the browser [#922]
• Fix initial seek on Tizen (Samsung TVs) on live contents by setting a much lower duration (to work-around a Tizen overflow) [#914]
• DASH: Consider multiple defined <Accessibility> tags for a single AdaptationSet [#903]
• Fix error that could be thrown on Safari when calling the getStatusForHDCP method from the experimental MediaCapabilitiesProber tool [#927]

Other improvements

• Avoid to push on top of the current position if there's already a segment there as it can provoke minor decoding issues on some devices [#925]
• Update video element's duration if the content duration changes [#917]
• DASH: Improve loading time with encrypted contents by only using the encrypted initialization data found in the Manifest when found in it [#911, #919]
• Record redirections made on a manifestUpdateUrl to request directly the right URL on next update. [#929]
• Improve loading time when a serverCertificate is given by calling the setServerCertificate API earlier [#895]
• Improve loading time when switching contents by fetching the Manifest at the same time the previous content is cleaned-up [#894]
• Improve loading time on some CPU-constrained devices by not running unnecessary playback checks on the "progress" HTMLMediaElement event anymore [#893]
• DASH: Consider DASH audio AdaptationSet with a "urn:mpeg:dash:role:2011" schemeIdUri and a "description" role as audioDescription tracks [#903]
• Warn through the logger when the autoplay attribute is enabled on the media element but not on RxPlayer [#924]
• Avoid switching to a SEEKING state if the seek operation was performed inside the RxPlayer's code [#872, #887]
• DRM: Wait up to 100 milliseconds after a persistent MediaKeySession has been loaded to wait for possibly late keyStatuses updates [#928]
• DRM: Only store persistent MediaKeySession once at least one key is known [#926]
• DRM: Reconsider Representations that have been fallbacked from if they become decipherable [#905]
• DRM: Lower the maximum size of the MediaKeySession cache from 50 to 15 to improve compatibility, even more now that license with multiple keys are properly handled
• Doc: Move architecture documentation closer to the code it documents [#764, #900]
• Doc: add "Quick links" to the top of the API documentation [#909]