Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Live HLS streams freeze on variant switches when ignoreManifestProgramDateTime is true. #6377

Closed
JulianDomingo opened this issue Mar 29, 2024 · 0 comments · Fixed by #6378
Closed
Assignees
Labels
component: HLS The issue involves Apple's HLS manifest format priority: P1 Big impact or workaround impractical; resolve before feature release type: bug Something isn't working correctly
Milestone

Comments

@JulianDomingo
Copy link
Collaborator

JulianDomingo commented Mar 29, 2024

Have you read the FAQ and checked for duplicate open issues?
Yes. No related open issue exists regarding this behavior when ignoreManifestProgramDateTime is true.

If the problem is related to FairPlay, have you read the tutorial?

N/A

What version of Shaka Player are you using?

Latest from main, and reproducible on all maintained version branches.

Can you reproduce the issue with our latest release version?
Yes.

Can you reproduce the issue with the latest code from main?
Yes.

Are you using the demo app or your own custom app?
Custom app, initially reported by Pluto TV's Cast app which uses Shaka Player.

If custom app, can you reproduce the issue using our demo app?
Yes.

What browser and OS are you using?
Chrome, ChromeOS

For embedded devices (smart TVs, etc.), what model and firmware version are you using?
Chromecast with Google TV, latest prod firmware

What are the manifest and license server URIs?

N/A (Pluto TV app library content)

What configuration are you using? What is the output of player.getConfiguration()?

Pluto TV-specific configuration; to avoid surfacing the entirety of their configs, the only relevant one to note is ignoreManifestProgramDateTime: true.

What did you do?

  1. Cast any live channel from Pluto TV's production app from Google Play Store.
  2. Let the live stream run and observe.

What did you expect to happen?
The live stream will continue to run without interruption and/or errors.

What actually happened?

The live stream eventually freezes (the time it takes for this varies randomly). Console logs indicate this is caused by Shaka infinitely stalling after a variant switch due to the timestamp offsets applied to the segments when it needs to drop old ones:

segmentIndex.dropFirstReferences(numSegmentsToDrop);
// Now adjust timestamps back to begin at 0.
const segmentN = segmentIndex.earliestReference();
if (segmentN) {
const streamOffset = -segmentN.startTime;
// Modify all SegmentReferences equally.
streamInfo.stream.segmentIndex.offset(streamOffset);
// Update other parts of streamInfo the same way.
this.offsetStreamInfo_(streamInfo, streamOffset);

Based on the media times of the segment references after the offset is applied:
pluto_tv_freeze_console_logs
The media times are not properly aligned.

Looking at the upstream code leading to the console log line (video:N) looking up segment from new stream endTime: ${TIME}, the ${TIME} is determined by mediaState in StreamingEngine. However, these are based on the old media times, whereas the times maintained by the mediaSequenceToStartTime in HlsParser are the new media times.

While a preferable fix here would be further investigating why/how a time misalignment occurs during variant switches, a simple (and proven fix according to PlutoTV) is to simply ignore the time offset-related logic when Shaka Player doesn't have any segments to drop. This will be the fix tagged to this issue.

Other justifications to consider this 'hotfix' route:

  1. This is a production issue for PlutoTV, so a timely fix is appreciated.
  2. Technically an optimization, and prevents risks of regression(s) for existing A/V sync logic relying on EXT-X-MEDIA-SEQUENCE by otherwise modifying other areas of the code.
@JulianDomingo JulianDomingo added type: bug Something isn't working correctly component: HLS The issue involves Apple's HLS manifest format platform: Cast Issues affecting Cast devices labels Mar 29, 2024
@JulianDomingo JulianDomingo self-assigned this Mar 29, 2024
@shaka-bot shaka-bot added this to the v5.0 milestone Mar 29, 2024
@avelad avelad added priority: P1 Big impact or workaround impractical; resolve before feature release and removed platform: Cast Issues affecting Cast devices labels Apr 5, 2024
theodab pushed a commit that referenced this issue Apr 5, 2024
…QUENCE (#6378)

Fixes #6377

When choosing to synchronize HLS streams using `EXT-X-MEDIA-SEQUENCE` instead of `EXT-X-PROGRAM-DATE-TIME` during LIVE playlist variant switches, Shaka unnecessarily drops 'old' segments and offsets the segment references of the new playlist so that the earliest reference represents media time `0`: https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/hls/hls_parser.js#L610-L613

This is problematic, as the `StreamingEngine`'s media time used to download new segments is based off the latest segment references:
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1248-L1250
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1385

For example:

```
            Playlist download #1         
EXT-X-MEDIA-SEQUENCE       Media Time
           0                   0
           1                   6
           2                   12
           3                   18

           Playlist download #2 (what happens now)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   0
          7                   6
          8                   12
          9                   18

           Playlist download #2 (desired behavior)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   36
          7                   42
          8                   48
          9                   54

```

Without this fix, and given the above example, if Shaka tries to request the segment at `time=36`, it will fail because the media state only has segment references up to `time=18`. Until the manifests, 'catch up', the player freezes; this can be especially problematic when a large amount of time accumulates before a variant switch occurs.

This has been confirmed by Pluto TV to fix their freezing issues.
avelad pushed a commit that referenced this issue Apr 8, 2024
…QUENCE (#6378)

Fixes #6377

When choosing to synchronize HLS streams using `EXT-X-MEDIA-SEQUENCE` instead of `EXT-X-PROGRAM-DATE-TIME` during LIVE playlist variant switches, Shaka unnecessarily drops 'old' segments and offsets the segment references of the new playlist so that the earliest reference represents media time `0`: https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/hls/hls_parser.js#L610-L613

This is problematic, as the `StreamingEngine`'s media time used to download new segments is based off the latest segment references:
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1248-L1250
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1385

For example:

```
            Playlist download #1         
EXT-X-MEDIA-SEQUENCE       Media Time
           0                   0
           1                   6
           2                   12
           3                   18

           Playlist download #2 (what happens now)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   0
          7                   6
          8                   12
          9                   18

           Playlist download #2 (desired behavior)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   36
          7                   42
          8                   48
          9                   54

```

Without this fix, and given the above example, if Shaka tries to request the segment at `time=36`, it will fail because the media state only has segment references up to `time=18`. Until the manifests, 'catch up', the player freezes; this can be especially problematic when a large amount of time accumulates before a variant switch occurs.

This has been confirmed by Pluto TV to fix their freezing issues.
avelad pushed a commit that referenced this issue Apr 8, 2024
…QUENCE (#6378)

Fixes #6377

When choosing to synchronize HLS streams using `EXT-X-MEDIA-SEQUENCE` instead of `EXT-X-PROGRAM-DATE-TIME` during LIVE playlist variant switches, Shaka unnecessarily drops 'old' segments and offsets the segment references of the new playlist so that the earliest reference represents media time `0`: https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/hls/hls_parser.js#L610-L613

This is problematic, as the `StreamingEngine`'s media time used to download new segments is based off the latest segment references:
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1248-L1250
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1385

For example:

```
            Playlist download #1         
EXT-X-MEDIA-SEQUENCE       Media Time
           0                   0
           1                   6
           2                   12
           3                   18

           Playlist download #2 (what happens now)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   0
          7                   6
          8                   12
          9                   18

           Playlist download #2 (desired behavior)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   36
          7                   42
          8                   48
          9                   54

```

Without this fix, and given the above example, if Shaka tries to request the segment at `time=36`, it will fail because the media state only has segment references up to `time=18`. Until the manifests, 'catch up', the player freezes; this can be especially problematic when a large amount of time accumulates before a variant switch occurs.

This has been confirmed by Pluto TV to fix their freezing issues.
joeyparrish pushed a commit that referenced this issue May 7, 2024
…QUENCE (#6378)

Fixes #6377

When choosing to synchronize HLS streams using `EXT-X-MEDIA-SEQUENCE` instead of `EXT-X-PROGRAM-DATE-TIME` during LIVE playlist variant switches, Shaka unnecessarily drops 'old' segments and offsets the segment references of the new playlist so that the earliest reference represents media time `0`: https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/hls/hls_parser.js#L610-L613

This is problematic, as the `StreamingEngine`'s media time used to download new segments is based off the latest segment references:
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1248-L1250
https://github.com/shaka-project/shaka-player/blob/ea740ba2468f3b035d463ea9933aa7eeccf5c748/lib/media/streaming_engine.js#L1385

For example:

```
            Playlist download #1         
EXT-X-MEDIA-SEQUENCE       Media Time
           0                   0
           1                   6
           2                   12
           3                   18

           Playlist download #2 (what happens now)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   0
          7                   6
          8                   12
          9                   18

           Playlist download #2 (desired behavior)
EXT-X-MEDIA-SEQUENCE       Media Time
          6                   36
          7                   42
          8                   48
          9                   54

```

Without this fix, and given the above example, if Shaka tries to request the segment at `time=36`, it will fail because the media state only has segment references up to `time=18`. Until the manifests, 'catch up', the player freezes; this can be especially problematic when a large amount of time accumulates before a variant switch occurs.

This has been confirmed by Pluto TV to fix their freezing issues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: HLS The issue involves Apple's HLS manifest format priority: P1 Big impact or workaround impractical; resolve before feature release type: bug Something isn't working correctly
Projects
None yet
3 participants