Choppy HLS MPEG-2 TS VOD playback with some gap jumping

[swac]

Have you read the FAQ and checked for duplicate open issues?

Yes, this issue is similar to #4516 but seems to be happening for MPEG-2 TS.

What version of Shaka Player are you using?

4.3.4

Can you reproduce the issue with our latest release version?

Yes

Can you reproduce the issue with the latest code from main?

Haven't tried yet, but it happens in the live demo app

Are you using the demo app or your own custom app?

Custom

If custom app, can you reproduce the issue using our demo app?

Yes

What browser and OS are you using?

Latest Chrome, macOS 12.4

What are the manifest and license server URIs?

Try demo app here.

What configuration are you using? What is the output of player.getConfiguration()?

Demo site default config

What did you do?

Playing an HLS MPEG-2 TS video has a choppy start, where the first segment seems to not play any video, but still plays audio. Subsequent segments play fine. It sounds a lot like what #4516 mentions.

I'm also seeing some gap jumping, where playback jumps a bit between contiguous segments. In the demo app I linked, I consistently see this output on a fresh page load:

Jumping forward 0.44800099999999965 seconds because of gap starting at 7.935999 and ending at 7.978
gap_jumping_controller.js:225 Jumping forward 0.35819199999999896 seconds because of gap starting at 19.881999 and ending at 19.945
gap_jumping_controller.js:225 Jumping forward 0.33518899999999974 seconds because of gap starting at 31.848999 and ending at 31.912
gap_jumping_controller.js:225 Jumping forward 0.5407979999999952 seconds because of gap starting at 43.815999 and ending at 43.879
gap_jumping_controller.js:225 Jumping forward 0.3507719999999992 seconds because of gap starting at 55.782999 and ending at 55.845999
gap_jumping_controller.js:225 Jumping forward 0.3580620000000039 seconds because of gap starting at 67.749998 and ending at 67.812999
gap_jumping_controller.js:225 Jumping forward 0.5554790000000054 seconds because of gap starting at 79.716998 and ending at 79.78
gap_jumping_controller.js:225 Jumping forward 0.5450749999999971 seconds because of gap starting at 91.683999 and ending at 91.747
gap_jumping_controller.js:225 Jumping forward 0.3479109999999963 seconds because of gap starting at 103.650999 and ending at 103.714
gap_jumping_controller.js:225 Jumping forward 0.5444590000000034 seconds because of gap starting at 115.617999 and ending at 115.681

What did you expect to happen?

Smooth playback from the start of the video and no gap jumping.

[swac]

As with #4516, I'm able to bisect it to this commit: 58182605. It seems easiest to reproduce with the Apple Advanced TLS Stream (TS) demo, but the Art of Motion demo also sometimes has the same issue.

[martinstark]

I tried to reproduce, and dug into the content a bit since we've also had some unrelated issues with MPEG2TS.

I can only reproduce the frame freeze with audio only on the second play in Chrome, when the Shaka instance is re-used. The first play works as expected. In Firefox it works as expected every time.

The manifest has several MUST FIX issues when passed through the Apple HLS validator.

Regarding the gap jumping, running ffprobe on the segments, the audio segments are ~3.97s long while the video segments are 4s. The A/V sync is noticeably off when people are beatboxing/talking.

Video:

#EXTM3U

#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4

#EXTINF:4.0
../video/1080_4800000/hls/segment_0.ts
#EXTINF:4.0
../video/1080_4800000/hls/segment_1.ts
#EXTINF:4.0
../video/1080_4800000/hls/segment_2.ts
#EXTINF:4.0
../video/1080_4800000/hls/segment_3.ts
# ...

ffprobe

[h264 @ 0x558709c6bec0] missing picture in access unit with size 8
Input #0, mpegts, from 'segment_3.ts':
  Duration: 00:00:04.00, start: 12.080000, bitrate: 5617 kb/s
  Program 1
  Stream #0:0[0x102]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn

TARGETDURATION, EXTINF, and true duration all line up.

Audio:

#EXTM3U

#EXT-X-VERSION:3
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-TARGETDURATION:4

#EXTINF:3.989
../audio/1_stereo_128000/hls/segment_0.ts
#EXTINF:3.989
../audio/1_stereo_128000/hls/segment_1.ts
#EXTINF:3.989
../audio/1_stereo_128000/hls/segment_2.ts
#EXTINF:3.989
../audio/1_stereo_128000/hls/segment_3.ts
# ...

ffprobe

Input #0, mpegts, from 'segment_3 (1).ts':
  Duration: 00:00:03.97, start: 11.904000, bitrate: 174 kb/s
  Program 1
  Stream #0:0[0x101]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 128 kb/s

TARGETDURATION, EXTINF, and true duration don't quite line up. The start is noticeably different from video start.

Media Playlists
8. Media Playlist requirements

8.1. You MUST use sufficiently accurate segment durations to ensure that the sum of the EXTINF durations of any contiguous group of segments is within one video frame duration of the actual duration of the content.

8.2. Audio and video playlists MUST all use the same target duration.

8.3. Audio and video playlists MUST contain the same duration of the content.

8.22. * For maximum interoperability, all audio/video variants and renditions SHOULD have segment boundaries at the same points in time.

I'm not very knowledgeable about the Shaka internals, but I would guess this is what creates multiple buffer ranges on the video element, which in turn causes the gap jumping, and the A/V sync being off.