Safari Fragment Loop Error before DAI Discontinuity in Event Stream

[dailos2coders]

Environment

• The stream has correct Access-Control-Allow-Origin headers (CORS)
• There are no network errors such as 404s in the browser console when trying to play the stream
• The issue observed is not already reported by searching on Github under https://github.com/video-dev/hls.js/issues
• The issue occurs in the latest reference client on http://video-dev.github.io/hls.js/demo and not just on my page

• Link to playable M3U8 file:
  The stream is an event stream from 12:30 to 17:00 uk time (stream is geoblocked in UK... need to use VPN to be able to play it... I can provide more event streams in other time span if needed)
  https://sport.voltest.bt.com/diva/DivaConf/github/index.html?src=https%3A%2F%2Ft1-btsport-live-hls-test-cluster-c.akamaized.net%2Fout%2Fu%2Fdiva_bts2%2FAUTO170906AT1230TO1700.m3u8%3Fbt.mode%3Dlive_event%26bt.channel%3Dl2v-diva_bts2%26bt.event%3DAUTO170906AT1230TO1700%26yo.ac%3Dtrue%26yo.ep%3Dtrue%26yo.up%3Dhttps%253A%252F%252Ft1-btsport-live-hls-test-cluster-c.akamaized.net%252Fout%252Fu%252Fdiva_bts2%252F%26yospace.initurl%3Dhttps%253A%252F%252Fcsm-e-cebteurxaws416-2hjvnjz85og7.tls1.yospace.com%252Fcsm%252Fextevent.AUTO170906AT1230TO1700%252Fbtsport01%252Cl2v-diva_bts2.m3u8%26bt.yosecure%3Dtrue%26yo.ch%3Dtrue%26hdnea%3Dst%3D1504665008~exp%3D1504717200~acl%3D%252Fout%252Fu%252Fdiva_bts2%252F%252A~hmac%3Db996a8e250c7bfcf18b837a8c8d7de2ecbcac1f90bca15895fba7917b42a4378&enableStreaming=true&autoRecoverError=true&enableWorker=true&dumpfMP4=false&levelCapping=-1&defaultAudioCodec=undefined
• Hls.js version: 0.8.1
• Browser name/version: Safari Versión 10.1.2 (12603.3.8) (Not working in any safari version)
• OS name/version: macOS Sierra (10.12.6 (16G29))

Steps to reproduce

The event stream has dynamic adverts inserted every 7 mins aprox. The adverts have #EXT-X-DISCONTINUITY tags. "Frag Loop Loading Error" is thrown in Safari

Expected behavior

The adverts should play smoothly.

Actual behavior

In Chrome, FireFox and IE the advert plays as expected when the discontinuity is reached, but in safari we get this error:  "Frag Loop Loading Error” and the player freezes in the previous fragment. Then we see that safari loads the to first segments of the ad, but in the log we see that the player is not able to buffer the segment 794 (the first segment of the advert). In Chrome we don’t get this error and we see buffer growing after discontinuity.

If we play the stream in safari directly putting the stream in the address input, the adverts are played correctly.

There is a screenshot attached of what we see in safari. When reaching the discontinuity… after loading the 2 first segments of the ad, the player starts switching between level 5 and 6 playlists but no segments are loaded.

Debugging the session I see some differences in the

fragment duration parsed in safari and the passed in the INF tag, while in chrome the fragment duration and the tag is the same. (Also attached screenshots)

log-chrome.txt
log-safari.txt

Console output

Paste the contents of the browser console here.

For media errors reported on Chrome browser, please also paste the output of chrome://media-internals 

[mangui]

interesting. i can't see NaN values reported in log-safari.txt however ?
the parsed fragment is appended into the sourcebuffer but it does not result in additional buffered data.
it would be difficult to investigate without a stream capture. it is pretty hard to move forward from here.

[dailos2coders]

I can give you streams to test if you need. The problem is that they are event streams, so they have a duration of 5 hours max. I can try to capture with Charles the stream and logs. Will send them as soon as I get them.
Thanks!

[dailos2coders]

I have a capture of the player with the issue in safari. This is the url of the player (stream alive until 17:00 uk time and geoblocked to uk):

https://sport.voltest.bt.com/diva/DivaConf/github/index.html?src=https%3A%2F%2Ft1-btsport-live-hls-test-cluster-c.akamaized.net%2Fout%2Fu%2Fdiva_bts1%2FAUTO170915AT1230TO1700.m3u8%3Fbt.mode%3Dlive_event%26bt.channel%3Dl2v-diva_bts1%26bt.event%3DAUTO170915AT1230TO1700%26yo.ac%3Dtrue%26yo.ep%3Dtrue%26yo.up%3Dhttps%253A%252F%252Ft1-btsport-live-hls-test-cluster-c.akamaized.net%252Fout%252Fu%252Fdiva_bts1%252F%26yospace.initurl%3Dhttps%253A%252F%252Fcsm-e-cebteurxaws416-2hjvnjz85og7.tls1.yospace.com%252Fcsm%252Fextevent.AUTO170915AT1230TO1700%252Fbtsport01%252Cl2v-diva_bts1.m3u8%26bt.yosecure%3Dtrue%26yo.ch%3Dtrue%26hdnea%3Dst%3D1505442609~exp%3D1505494800~acl%3D%252Fout%252Fu%252Fdiva_bts1%252F%252A~hmac%3D98ac43e9a80535b6144cd72e4d29257abf912424c7c211a900a51eca29b368bc&enableStreaming=true&autoRecoverError=true&enableWorker=true&dumpfMP4=false&levelCapping=-1&defaultAudioCodec=undefined

The Safari log:

https://sport.voltest.bt.com/diva/DivaConf/github/safari-dai-hls-issue.txt

The charles capture:

https://sport.voltest.bt.com/diva/DivaConf/github/safari-dai-hls-issue.chls.zip

[dailos2coders]

Hi @mangui,

We have been testing the player in different safari versions... this is the feedback we got:

10.0 (Released September 20, 2016) – DAI adverts working with no issue?

10.1.1 (Released May 15, 2017) – Player crashes on DAI ad break + does not recover.
10.1.2 (Released July 19, 2017) – Player crashes on DAI ad break + does not recover.

11.0 (TBC September/October 2017) – Stutters before the advert, then player shows video error, then video plays on after ad finished.

thanks!

[mangui]

looks like after a discontinuity, Safari is not appending anything in the buffer although sourcebuffer.append is triggered.

the only way to get help on that is to have clear repro steps so that Safari engineers could have a look

[dailos2coders]

Thanks for replying @mangui.

The thing is that works fine playing the stream pasting it in the address input, and we have tested with other html5 players, and also works ok.

The problem only happens with the Hls.js + Safari 10.1.* combination.

Were you able to have a look to the charles trace? maybe you see something there (problem with stream or similar) that could be causing this. The only difference I saw between safari and chrome is that in safari the duration parsed for the previous segment before the discontinuity differs form the duration tag in the level playlist, in chrome duration and tag are equal.

Please let me know if you need more info or if you need event streams to test I can prepare them. My skype is dailosmm.

Thanks a lot.

[mangui]

I have limited time to spend on analyzing tickets, this is why having a reduced test case would help.

thanks

[dailos2coders]

Hi @mangui,

A colleague has been able to create a small vod stream where you can easily replicate the issue:

http://2coders.com/streamtest/hls.js/demo/?src=http%3A%2F%2F2coders.com%2Fstreamtest%2Fmanifest.m3u8&enableStreaming=true&autoRecoverError=true&enableWorker=true&dumpfMP4=true&useCredential=false&debugMode=true&levelCapping=-1&defaultAudioCodec=undefined

The new safari 11 release, I can see bit different behaviour... safari skips the discontinuity... is able to continue playing the main stream, but no advert is played.

Thanks a lot

[mangui]

switching audio sampling rate on discontinuity from 22.5 kHz to 24kHz might not help ...
pinging @jernoble to ask whether Safari supports audio sampling rate change on a discontinuity.

in the meantime you could also assess my statement by respectively disabling audio / video parsing:
just comment this line for audio and this line for video
and recheck your stream

[dailos2coders]

Disabling Audio I get the same issue. Video freezes and Frag Loop Loading Error is reported.

[dailos2coders]

Hello @mangui any other suggestion or test I could make to help resolving the issue?

Thanks a lot for your help

[mangui]

then we would need help from Safari engineers to understand why source buffer append does not result in extra video being buffered.

[jernoble]

We have a bug on this; but it's at a media framework level, so there's no possible fix which would make it into a STP or Nightly.

[mangui]

anything we could workaround at remuxing side ?

[jernoble]

As far as the "source buffer append does not result in extra video being buffered" issue, only append a full media segment at a time?

Unfortunately, I don't have more information about the proximate cause of the issue, just that the description matches that of another bug I saw go by.

[mangui]

basically just testing this link on Safari is enough to repro.

video from frag 4 never gets buffered

[Log] [log] > Parsed audio,PTS:[36.366,46.444],DTS:[36.366/46.444],nb:217,dropped:0 (hls.js, line 125)
[Log] [log] > AVC:86 ms hole between fragments detected,filling it (hls.js, line 125)
[Log] [log] > Video/PTS/DTS adjusted: 36445/36280,delta:86 ms (hls.js, line 125)
[Log] [log] > Parsed video,PTS:[36.445,46.446],DTS:[36.280/46.406],nb:251,dropped:0 (hls.js, line 125)
[Log] [log] > main stream:PARSING->PARSED (hls.js, line 125)
[Log] [log] > main buffered : [0.099,36.320] (hls.js, line 125)

[dailos2coders]

Hi @jernoble and @mangui,
Is there anything else I can provide to try to resolve the issue? @jernoble the issue seems to happen only when playing the stream using HLS.js. If we play the stream tokenized url in the address input directly the discontinuity plays correctly. Is there any log or any other info I can provide to help you finding the reason of the issue?

Thanks for the help

[zuzzurro]

Hi @jernoble and @mangui,

While searching for ways to move this thread forward, and in particular to help @mangui to develop a workaround, we found out that video.js is able to play the video just fine in Safari.

If you paste the goold old video above into http://videojs.github.io/videojs-contrib-hls/ you will see that the video plays fine.

Hope this helps..

[tchakabam]

@zuzzurro Safari supports HLS natively via the video element src URL property. videojs-contrib-hls uses the native Safari HLS instead of it's own MediaSource based client by default (when not told explictly to overrideNative, see https://github.com/videojs/videojs-contrib-hls#overridenative). Are you sure about wether you are using the native or actual videojs HLS implementation when testing this?

I will test this myself start of next week, can't really get to it right now.

What I can say now is: If videojs-contrib-hls can play it in "MediaSource mode" (overrideNative: true -> don't use native Safari HLS), then Hls.js could in principle also play it. Must then be a matter of remuxing / timestamps or how we talk to MSE API.

If only the native Safari client can play it, and it fails in videojs-contrib-hls when using overrideNative then I can not tell with confidence wether it will be solvable, or wether this is something that needs to be fixed in Safari.

Either way, you can then fall-back to native Safari HLS to play this content (no need to use videojs either, just use the native video DOM element and set the URL as src) - but this has disadvantages against using Hls.js of course.

Get in touch with me via email for further investigation :)

[tchakabam]

http://2coders.com/streamtest/manifest.m3u8

[mangui]

it is indeed working with http://videojs.github.io/videojs-contrib-hls/ which uses transmuxing as well.
so it should be a remuxing issue inside HLS.js
@jernoble if you could give us some hints why sb.appendBuffer() does not result in any additional video being buffered after t=36s on this page, we would be very grateful 🙏

[tchakabam]

Just for reference, here is a quite recent videojs-contrib-hls build (5.12.0) + video.js 6.x, that is configured to use MediaSource (overrideNative:true, nativeAudio/VideoTracks: false):

http://tchakabam.com/test/2coders-videojs/

It has the above URL set by default: http://2coders.com/streamtest/manifest.m3u8

In the console should appear logs that indicate that the videojs-hls client is actually running, as opposed to the built-in Safari one.

[tchakabam]

IMPORTANT: You need to click on "load" button so that the videojs engine loads with the desired options actually. If you just click on play immediately, will use built-in HLS.

Not sure yet if this is intended, related to how the test-page is built, or if that is actually a bug in videojs/hls (at least this problem of not loading the engine on page load only appears on Safari, so its probably also browser related :D)

[tchakabam]

Do you also get these nasty artifacts in there @mangui ? It seems there are still issues there compared to the built-in quicktime component ;) at least that's what I see on my macBook. but may be hw or OS version related again. I m testing on Safari 10.1.

[jernoble]

@mangui said:

@jernoble if you could give us some hints why sb.appendBuffer() does not result in any additional video being buffered after t=36s on this page, we would be very grateful 🙏

Well, one thing I can see immediately from the generated fMP4s is that most of the samples overlap with previous samples. When parsing the movie and sorting the output by PTS, you get something like the following:

 Checking track 481
 Presentation time: 0.04, duration: 0.04, end time: 0.08
 Presentation time: 0.08, duration: 0.04, end time: 0.12
 Presentation time: 0.12, duration: 0.08, end time: 0.2
 Presentation time: 0.16, duration: 0.04, end time: 0.2
 Overlap detected! 0.200000 -> 0.160000
 Presentation time: 0.2, duration: 0.04, end time: 0.24
 Presentation time: 0.24, duration: 0.08, end time: 0.32
 Presentation time: 0.28, duration: 0.04, end time: 0.32
 Overlap detected! 0.320000 -> 0.280000
 Presentation time: 0.32, duration: 0.04, end time: 0.36
 Presentation time: 0.36, duration: 0.08, end time: 0.44
 Presentation time: 0.4, duration: 0.04, end time: 0.44
 Overlap detected! 0.440000 -> 0.400000
 Presentation time: 0.44, duration: 0.04, end time: 0.48
 Presentation time: 0.48, duration: 0.08, end time: 0.56
 Presentation time: 0.52, duration: 0.04, end time: 0.56
 Overlap detected! 0.560000 -> 0.520000

I don't really understand why that's happening; it's pretty obvious that those samples have explicit durations of 0.04 (3600/90000) set in the trun atom. You may be hitting some corner-case bug in our fMP4 parser. You may be able to simplify your trun atoms by providing a default duration if all samples are the same length, or if every moof starts with an I-frame, you can use the first-sample-flags-present flag and set the first_sample_flags field, and your trun would only have to contain sample_composition_time_offsets. That said, since this is happening within the first three frames, it's probably not the cause of an append past 00:36 failing.

I tried to dump the video fMP4, but it doesn't seem to dump the generated fMP4 past 00:36.28, and all of those samples are accounted for in the generated media. Any hints?

[jernoble]

Yeah, I think the samples you append are getting dropped because the trackID changes across the two initialization segments (which is totally legal, so long as there is only one video track). This sounds like a simple bug in WebKit's MSE append algorithm.

[mangui]

indeed it seems that track id is the video PID (from TS) ...
so if we keep using the same track id it should work right ?

[jernoble]

That should be a sufficient workaround; I'll file a WebKit bug to track the work that should make this workaround unnecessary.

[dailos2coders]

Thanks @mangui and @jernoble for investigating this issue.

If you want me to help testing or providing more test streams, I can provide that.

[zuzzurro]

So it looks like we have been hit twice by the same Safari issue as in #1272 .. Right?

[tchakabam]

Ok so it should just be about making sure that we keep the track ID for video consistent. At any time, there can only be one video sourcebuffer always (no browser supports several) anyway.

Therefore, there should be no reason really in dragging the track-ID from TS Program ID into the MP4, since we may get TS from many different sources into our HLS stream, this can be inconsistent, and cause such issues.

Since we are having fixed numbers of tracks per type on the MSE SourceBuffer side (one video, one audio, one text, one ID3), and we are just remuxing all the different "tracks" or renditions or whatever pops up in TS into the one type of MP4, we can as well just have a fixed track ID in the muxed MP4.

I will push the necessary changes on TSdemuxer/remuxer interface so we are simply fixing the track IDs of the resulting MP4 tkhd boxes.

[tchakabam]

Awesome we were able to figure it out that quickly!! Thank you @jernoble and @mangui !! 🍻

[dailos2coders]

I've tested in our environment and this fix is working 👍

[gemxx]

Yeah, I think the samples you append are getting dropped because the trackID changes across the two initialization segments (which is totally legal, so long as there is only one video track). This sounds like a simple bug in WebKit's MSE append algorithm.

This is really helpful for me to fix issue while switch to another audio