Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer #1236

Merged
merged 2 commits into from Jun 15, 2023

Conversation

peaBerberian
Copy link
Collaborator

Note: This may be a controversial PR :p, don't hesitate to exchange here if you see problem with this logic or if you have ideas for improving it.
This should only impact contents relying on DRM.


The majority of our bugs are now DRM-related and device-specific, generally platform lower-level (CDM, browser integration...) bugs. Even if we very frequently exchange with partners to obtain fixes, this is not always possible (sometimes because there are too many people relying on the older logic and thus risks in changing it, sometimes because there are a lot of higher priorities on their side).

We are now encountering frequently, for some contents with DRM, what we call a "playback freeze": playback does not advance despite having data in the buffer, all being known to be decipherable (we know that we have pushed a license and that their linked key has a key id set to "usable" in the corresponding MediaKeyStatusMap).

Recently we've seen this similar issue on Microsoft Edge, UWP applications (Windows, XBOX...) and LG TV.

In all of those cases, what we call "reloading" after the license has been pushed always fixes the issue. The action of reloading means principally to re-create the audio and video buffers, to then push again segments on it.
Other work-arounds may work on some platform, but do not seem to work on other.

Although we prefer providing more targeted fixes or telling to platform developpers to fix their implementation, this issue is so frequent that we're now wondering if we should provide some heuristic in the RxPlayer, to detect if that situation arises and reload in that case, as a fallback mechanism.
The logic being the following: we prefer to reload over having an infinite loading or buffering phase.

What we are most afraid here is the risk of false positives: falsely considering that we are in a "decipherability-freeze" situation, where it's just a performance issue on the hardware side or an issue in a segment.

To limit greatly the risk of false positives, here are the rules that will lead to a reload under that heuristic.
All sentences after dashes here are mandatory:

  • we have a readyState set to 1 (meaning that the browser announces that it has enough metadata to be able to play, but does not seem to have any media data to decode)
  • we have at least 6 seconds in the buffer ahead of the current position (so compared with the previous dash, we DO have enough data to play - generally this happens when pushed media data is not being decrypted).
  • The playhead (current position) is not advancing and has been in this frozen situation for at least 4 seconds
  • The content has at least 1 audio or video Representation with DRM.
  • One of the following is true:
    1. There is at least one segment that is known to be undecipherable in the buffer (it should not happen, the RxPlayer already preventing this situation elsewhere, this was added as a security)
    2. There are ONLY segments that are known to be decipherable (licenses have been pushed AND their key-id are all "usable") in the buffer, for both audio and video.

If all those conditions are true we will reload. For now with no limit (meaning we could have several reloads for one content if the situation repeats).
Note that at the request level, this only might influence segment requests (which will have to be reloaded after 4 seconds) and not DRM-related requests nor Manifest requests.

We tested that logic with success on some of the problematic devices.

We're now in a testing/reviewing phase where we are testing and considering the false positive risks, we're still adding and removing tweaks to the conditions.

This is only planned to be added on the v4 for our official release, for API-breakability reasons BUT we plan to test it internally at Canal+ in some special v3 pre-releases, to ensure that this works well.

@peaBerberian peaBerberian added the DRM Relative to DRM (EncryptedMediaExtensions) label Mar 30, 2023
@peaBerberian peaBerberian added this to the 4.0.0 milestone Mar 30, 2023
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch 2 times, most recently from 223de89 to 8bdeb7d Compare April 6, 2023 10:00
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from 8bdeb7d to 4ff4e0d Compare April 11, 2023 14:16
@peaBerberian peaBerberian force-pushed the next-v4 branch 2 times, most recently from 13a21a1 to 8324542 Compare May 17, 2023 16:23
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from 4ff4e0d to 967954b Compare May 17, 2023 16:25
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch 2 times, most recently from 9754bff to 44d202c Compare June 2, 2023 13:58
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from 44d202c to bfc1c99 Compare June 2, 2023 13:59
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch 2 times, most recently from 8584da3 to 4a30598 Compare June 13, 2023 15:25
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from 4a30598 to 6a1472c Compare June 15, 2023 13:08
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from 6a1472c to c3caaae Compare June 15, 2023 13:46
…ecipherable data in the buffer

Note: This may be a controversial commit :p, don't hesitate to exchange
in the linked PR if you see problem with this logic or if you have
ideas for improving it.
This should only impact contents relying on DRM.

---

The great majority of our bugs are now DRM-related and device-specific,
generally platform lower-level (CDM, browser integration...) bugs.
Even if we very frequently exchange with partners to obtain fixes,
this is not always possible (sometimes because there are too many
people relying on the older logic and thus risks in changing it,
sometimes because there are a lot of higher priorities on their side).

We are now encountering frequently, for some contents with DRm, what
we call a "playback freeze": playback does not advance despite having
data in the buffer, all being known to be decipherable (we know that
we have pushed a license and that their linked key has a key id set to
"usable").

Recently we've seen this similar issue on Microsoft Edge, UWP
applications (Windows, XBOX...) and LG TV.

In all three cases, what we call "reloading" after the license has been
pushed always fixes the issue. The action of reloading means principally
to re-create the audio and video buffers, to then push again segments on
it.
Other work-arounds may work on some platform, but do not seem to work on
other.

Although we prefer providing more targeted fixes or telling to platform
developpers to fix their implementation, this issue is so frequent that
we're now wondering if we should provide some heuristic in the RxPlayer,
to detect if that situation arises and reload in that case, as a fallback
mechanism.
The logic being the following: we prefer to reload over having an infinite
loading or buffering phase.

What we are most afraid here is the risk of false positives:
falsely considering that we are in a "decipherability-freeze" situation,
where it's just a performance issue on the hardware side or an issue in
a segment.

To limit greatly the risk of false positives, here are the rules that
will lead to a reload under that heuristic.
All sentences after dashes here are mandatory:
  - we have a `readyState` set to `1` (meaning that the browser
    announces that it has enough metadata to be able to play, but does
    not seem to have any media data to decode)
  - we have at least 6 seconds in the buffer ahead of the current
    position (so compared with the previous dash, we DO have enough data
    to play - generally this happens when pushed media data is not being
    decrypted).
  - The playhead (current position) is not advancing and has been in this
    frozen situation for at least 4 seconds
  - The content has at least 1 audio or video Representation with DRM.
  - One of the following is true:
      1. There is at least one segment that is known to be
         undecipherable in the buffer (it should not happen, the RxPlayer
         already preventing this situation elsewhere, this was added as a
         security)
      2. There are ONLY segments that are known to be decipherable
         (licenses have been pushed AND their key-id are all `"usable"`)
         in the buffer, for both audio and video.

If all those conditions are `true` we will reload. For now with no limit
(meaning we could have several reloads for one content if the situation
repeats).
Note that at the request level, this only might influence segment
requests (which will have to be reloaded after 4 seconds) and not
DRM-related requests not Manifest requests.

We tested that logic with success on the problematic devices.

We're now in a testing/reviewing phase where we are testing and
considering the false positive risks, we're still adding and removing
tweaks to the conditions.

This is only planned to be added on the v4 for our official release, for
API-breakability reasons BUT we plan to test it internally at Canal+ in
some special v3 pre-releases, to ensure that this works well.
@peaBerberian peaBerberian force-pushed the misc/decipherability-reload-v4 branch from c3caaae to d2c09b9 Compare June 15, 2023 14:13
@peaBerberian peaBerberian merged commit cd56ce0 into next-v4 Jun 15, 2023
3 checks passed
@peaBerberian peaBerberian modified the milestones: 4.0.0, 4.0.0-beta.2 Jun 15, 2023
peaBerberian added a commit that referenced this pull request Jun 27, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Jul 4, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
@peaBerberian peaBerberian deleted the misc/decipherability-reload-v4 branch July 6, 2023 12:01
peaBerberian added a commit that referenced this pull request Jul 21, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Jul 21, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Jul 24, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 7, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 22, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 23, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 31, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 31, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Aug 31, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 15, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 15, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 22, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 26, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 26, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 27, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 27, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 27, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Sep 29, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Oct 13, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Oct 13, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Oct 13, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Oct 19, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
peaBerberian added a commit that referenced this pull request Oct 26, 2023
DRM: Reload when playback is unexpectedly frozen with encrypted but only decipherable data in the buffer
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
DRM Relative to DRM (EncryptedMediaExtensions)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant