hls.js can not play correctly a live stream with multiple dimensions

[Steven086]

What version of Hls.js are you using?

latest version

What browser and OS are you using?

Chrome, Linux/WIndows

Test stream:

http://cacheawslive.5iketang.com/live/1516-237379/playback/E4ACCAFC.m3u8?vh=aHR0cDovL2RxenouNWlrZXRhbmcuY29tL21lZGlhL3BsYXllci8yMjA5OTkvMjM3Mzc5L2F1dGgva2V5P210b2tlbj1leUoxYzJWeVNXUWlPakFzSWw5MElqb3hOVFExT0RjME5ETXlMQ0owYjJ0bGJpSTZiblZzYkN3aWMybG5iaUk2SWpnME16azJNV0l5WXpJeE16QTVaV00wTURWa05EUmxPVGhrTURjd1lXWmlJbjA

Checklist

• [ X] The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• [ X] The issue occurs in the stable client on https://hls-js.netlify.com/demo and not just on my page

• [ X] The issue occurs in the latest client on https://hls-js-latest.netlify.com/demo and not just on my page

• [ X] The stream has correct Access-Control-Allow-Origin headers (CORS)
• [ X] There are no network errors such as 404s in the browser console when trying to play the stream

Steps to reproduce

1. prepare a list of ts files with multiple dimensions, ex, segments during 00:00:00 to 01:40:00 have video size of 960x540 and the others have 1280x720.
2. play a m3u8 url with the above ts files. while the current playing position is more than 100(ts), player can not work corrently.
3. if we change the SPS related codes of TSDemuxer.prototype._parseAVCPES in hls.js as follows:
   // SPS
   case 7:
   push = true;
   spsfound = true;
   if (debug && avcSample) {
   avcSample.debug += 'SPS ';
   }
   var expGolombDecoder2 = new exp_golomb_1.default(unit.data);
   var config2 = expGolombDecoder2.readSPS();
   console.log("dimension: ", track.width, track.height);
   if ( track.sps && config2.width != track.width ){
   _this.remuxer.resetInitSegment();
   track.sps = null;
   }

the player can work well.

[michaelcunningham19]

Hi @Steven086

I checked out your stream and confirmed that playback does stall at the time of the resolution switch.

I was curious if this was allowable by the specification and it turns out it is:

The encoding parameters for samples in a Media Segment and across
multiple Media Segments in a Media Playlist SHOULD remain consistent.
However, clients SHOULD deal with encoding changes as they are
encountered, for example, by scaling video content to accommodate a
resolution change. If the Variant Stream includes a RESOLUTION
attribute, clients SHOULD display all video within a rectangle with
the same proportions as that resolution.
Source: https://tools.ietf.org/html/draft-pantos-hls-rfc8216bis-03#section-6.3.3

So I went ahead and tested out native playback in Safari, and it plays back the problem time range without issue.

Here's the behaviour I noted in Safari:

• seek to 1:38:00
• allow playback to continue past 1:40:00
• view the video element's videoWidth and videoHeight attributes before and after the resolution changes
• curious enough, the reported width and height of the media is always 960x540
• seek ahead 15s via the native controls
• observe that the reported width and height of the media is now 1280x720

Regardless of the nuance between the reported dimensions on Safari, it allows playback to continue without issue

On a sidenote, I ran your stream through Apple's mediastreamvalidator and it output a few must fix issues involving Error: Playlist vs segment duration mismatch

[robwalch]

Hi @Steven086 and @michaelcunningham19,

So I went ahead and tested out native playback in Safari, and it plays back the problem time range without issue.

This is good to know.

I just closed a duplicate #1842 with wontfix but I wasn't aware of how Safari dealt with this issue, or how simple the solution might be.

The playlist should mark resolution changes with a DISCONTINUITY to handle resolution change / init segment creation.

This is not a priority for us to fix, but if generating a new init segment can be done simply without side-effects based on TS demuxer data, I would accept a PR as long as the behavior matches Safari's.

[robwalch]

Hoping to close this out this issue with #5794.

The test stream in the description is long gone so I tested by combining segments from 720p and 540p TS playlists into one. Chrome 115 handles the sps change without any intervention, Safari does not.

Resetting the init segment when a resolution change is detected outputs a new init segment from the remuxer which is appended before the following media segments.