Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(HLS): Reset textSequenceModeOffset on discontinuity #6388

Merged
merged 11 commits into from
Apr 4, 2024

Conversation

willdharris
Copy link
Contributor

@willdharris willdharris commented Apr 1, 2024

Fixes #5168
Fixes #6325
Fixes #5096
Fixes #4828
Fixes #5604

  • Currently the offset used for time.periodStart in computeHlsSequenceModeOffset_, is calculated and set at stream start and never updated.
  • In MediaSourceEngine the correct offset is actually being calculated on every call to appendBuffer. But when it is applied to the text engine textEngine_.setTimestampOffset, in order to update time.periodStart, it only ever uses the initial calculated offset.
  • this.textSequenceModeOffset_ is initiated as a shaka.util.PublicPromise(). The value is updated by calling this.textSequenceModeOffset_.resolve(value). However, once the promise is resolved, calling resolve on it again has no effect. So the first value passed is locked in and used in all subsequent calculations.
  • This PR resets this.textSequenceModeOffset_ to a new shaka.util.PublicPromise() after it's previously fulfilled value has been read.

lib/media/media_source_engine.js Outdated Show resolved Hide resolved
@avelad avelad added type: bug Something isn't working correctly component: HLS The issue involves Apple's HLS manifest format component: captions/subtitles The issue involves captions or subtitles priority: P1 Big impact or workaround impractical; resolve before feature release component: WebVTT The issue involves WebVTT subtitles specifically labels Apr 2, 2024
@avelad avelad added this to the v5.0 milestone Apr 2, 2024
@willdharris willdharris changed the title fix(HLS): Reset textSequenceModeOffset on each offset calculation fix(HLS): Reset textSequenceModeOffset on discontinuity Apr 2, 2024
@littlespex littlespex changed the base branch from main to v4.7.x April 2, 2024 16:14
@shaka-bot
Copy link
Collaborator

shaka-bot commented Apr 2, 2024

Incremental code coverage: 100.00%

@littlespex littlespex changed the base branch from v4.7.x to main April 2, 2024 16:14
* fix: reset textSequenceModeOffset on resync
Copy link
Collaborator

@avelad avelad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but please, add a test to avoid this error in the future.

@littlespex
Copy link
Collaborator

littlespex commented Apr 3, 2024

@avelad The only way I can think of to properly test this is with a full integration or E2E test. That would require a public stream which we don't have available. Can we merge the fix while we figure out a way to test this?

@avelad
Copy link
Collaborator

avelad commented Apr 3, 2024

@littlespex We have https://github.com/shaka-project/shaka-player/blob/main/test/hls/hls_parser_integration.js for HLS integration tests. Since this problem also occurs on VOD, it should be easy to do a test.

@littlespex
Copy link
Collaborator

@avelad The issue is we don't have an asset that we can share publicly.

Copy link
Collaborator

@theodab theodab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I... think this new approach should be safe? I don't see any obvious problems with it anyway.

I'll try to make a test for this; I might be able to hack some of our existing sample content into a repro. I'll do it as a follow-up PR so that this can be in the nightly build for practical testing ASAP.

@littlespex
Copy link
Collaborator

littlespex commented Apr 3, 2024

@avelad @theodab @joeyparrish We've added an integration test for this change.

@avelad
Copy link
Collaborator

avelad commented Apr 4, 2024

@shaka-bot test

@shaka-bot
Copy link
Collaborator

@avelad: Lab tests started with arguments:

  • pr=6388

Copy link
Collaborator

@theodab theodab left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've run the test locally, and I can confirm that it passes, and stops passing if the fix is disabled (and that if the fix is disabled the third text cue is positioned very incorrectly).

I don't know if how many times the cuechange even was fired is the right thing to test, though. It seems kind of tangential to the actual problem (that the cues are not being inserted with the correct time offsets).
You can keep that check if you want, but it'd be best to also do something like:

    expect(textTrack.cues.length).toBe(3);
    expect(textTrack.cues[0].startTime).toBeCloseTo(0, 0);
    expect(textTrack.cues[0].endTime).toBeCloseTo(2, 0);
    expect(textTrack.cues[1].startTime).toBeCloseTo(2, 0);
    expect(textTrack.cues[1].endTime).toBeCloseTo(4, 0);
    expect(textTrack.cues[2].startTime).toBeCloseTo(6, 0);
    expect(textTrack.cues[2].endTime).toBeCloseTo(8, 0);

To verify that all of the cues are inserted, and that they have the correct offsets.

test/hls/hls_parser_integration.js Outdated Show resolved Hide resolved
@littlespex
Copy link
Collaborator

Tests updated and unused assets removed.

@avelad
Copy link
Collaborator

avelad commented Apr 4, 2024

@shaka-bot test

@shaka-bot
Copy link
Collaborator

@avelad: Lab tests started with arguments:

  • pr=6388

@theodab theodab dismissed joeyparrish’s stale review April 4, 2024 21:02

Joey is out at the moment. I can confirm that the concern he raised has been addressed.

@theodab theodab merged commit 966302d into shaka-project:main Apr 4, 2024
30 of 32 checks passed
avelad pushed a commit that referenced this pull request Apr 8, 2024
Fixes #5168
Fixes #6325
Fixes #5096
Fixes #4828
Fixes #5604

- Currently the offset used for `time.periodStart` in [computeHlsSequenceModeOffset_](https://github.com/shaka-project/shaka-player/blob/03bb463a724483c88df818b11c807a0fdc11cccb/lib/text/vtt_text_parser.js#L174), is calculated and set at stream start and never updated.
- In `MediaSourceEngine` the correct offset is actually being calculated on every call to `appendBuffer`. But when it is applied to the text engine `textEngine_.setTimestampOffset`, in order to update `time.periodStart`,  it only ever uses the initial calculated offset.
- `this.textSequenceModeOffset_` is initiated as a `shaka.util.PublicPromise()`. The value is updated by calling `this.textSequenceModeOffset_.resolve(value)`. However, once the promise is resolved, calling resolve on it again has no effect. So the first value passed is locked in and used in all subsequent calculations.
- This PR resets `this.textSequenceModeOffset_` to a new `shaka.util.PublicPromise()` after it's previously fulfilled value has been read.

Co-authored-by: Casey Occhialini <1508707+littlespex@users.noreply.github.com>
avelad pushed a commit that referenced this pull request Apr 8, 2024
Fixes #5168
Fixes #6325
Fixes #5096
Fixes #4828
Fixes #5604

- Currently the offset used for `time.periodStart` in [computeHlsSequenceModeOffset_](https://github.com/shaka-project/shaka-player/blob/03bb463a724483c88df818b11c807a0fdc11cccb/lib/text/vtt_text_parser.js#L174), is calculated and set at stream start and never updated.
- In `MediaSourceEngine` the correct offset is actually being calculated on every call to `appendBuffer`. But when it is applied to the text engine `textEngine_.setTimestampOffset`, in order to update `time.periodStart`,  it only ever uses the initial calculated offset.
- `this.textSequenceModeOffset_` is initiated as a `shaka.util.PublicPromise()`. The value is updated by calling `this.textSequenceModeOffset_.resolve(value)`. However, once the promise is resolved, calling resolve on it again has no effect. So the first value passed is locked in and used in all subsequent calculations.
- This PR resets `this.textSequenceModeOffset_` to a new `shaka.util.PublicPromise()` after it's previously fulfilled value has been read.

Co-authored-by: Casey Occhialini <1508707+littlespex@users.noreply.github.com>
@shaka-bot shaka-bot added the status: archived Archived and locked; will not be updated label Jun 3, 2024
@shaka-project shaka-project locked as resolved and limited conversation to collaborators Jun 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
component: captions/subtitles The issue involves captions or subtitles component: HLS The issue involves Apple's HLS manifest format component: WebVTT The issue involves WebVTT subtitles specifically priority: P1 Big impact or workaround impractical; resolve before feature release status: archived Archived and locked; will not be updated type: bug Something isn't working correctly
Projects
None yet
6 participants