New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mac OS desktop Safari compatibility #9

Closed
blacktrash opened this Issue Sep 22, 2015 · 56 comments

Comments

Projects
None yet
@blacktrash
Contributor

blacktrash commented Sep 22, 2015

@mangui - First off: congrats! Very exciting.

Now the bad news ;-)
Mac OS Safari's MediaSource implementation is rather suboptimal. It hangs with the samples at http://dailymotion.github.io/hls.js/demo/
Some other streams make it at least to the second segment.

As I said, I presume it's Safari's MediaSource implementation, with MPEG-DASH one encounters similar 'surprises', for instance MediaSource.isTypeSupported is even less useful than canPlayType, for instance MediaSource.isTypeSupported('video/mp4; codecs="bogus"') returns true.

Not sure whether there's a way around generic hls in Safari yet (e.g. for smooth manual bit rate switching). But maybe you already have ideas.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Sep 22, 2015

Member

for now, stick with native HLS in Safari...

debugging MediaSource on Safari is complex as we don't have access to the underlying sourcecode.
the general issue with Safari and hls.js seems to be at the fragmented MP4 generation level.
it looks like Safari does not like the timestamps/fragment duration provided in the fragmented MP4 chunk.

if you dump video.buffered (click on toggle stats display) you will observe plenty of micro holes in the video buffer: :[0.0332888,0.0999121][0.1,0.1498676][0.1499111,0.1998232][0.1998666,0.2497787][0.2498222,0.3663565][0.3664,0.3996454][0.3996888,0.4662676][0.4663111,0.4995565][0.4996,0.5994676][0.5995555,0.6494232][0.6494666,0.7160454][0.7160888,0.7493343][0.7493777,0.8492454][0.8493333,0.9158676][0.9159111,0.9491565][0.9492,1.0324454][1.0324888,1.0990232]

Safari will stop the playback at the first hole...

Member

mangui commented Sep 22, 2015

for now, stick with native HLS in Safari...

debugging MediaSource on Safari is complex as we don't have access to the underlying sourcecode.
the general issue with Safari and hls.js seems to be at the fragmented MP4 generation level.
it looks like Safari does not like the timestamps/fragment duration provided in the fragmented MP4 chunk.

if you dump video.buffered (click on toggle stats display) you will observe plenty of micro holes in the video buffer: :[0.0332888,0.0999121][0.1,0.1498676][0.1499111,0.1998232][0.1998666,0.2497787][0.2498222,0.3663565][0.3664,0.3996454][0.3996888,0.4662676][0.4663111,0.4995565][0.4996,0.5994676][0.5995555,0.6494232][0.6494666,0.7160454][0.7160888,0.7493343][0.7493777,0.8492454][0.8493333,0.9158676][0.9159111,0.9491565][0.9492,1.0324454][1.0324888,1.0990232]

Safari will stop the playback at the first hole...

@blacktrash

This comment has been minimized.

Show comment
Hide comment
@blacktrash

blacktrash Sep 22, 2015

Contributor

Very useful debugging interface!

With TS segment files it's slightly different, it doesn't make the jump to the first frame of the next segment, but I guess the underlying issue is more or less similar.

No rush, I was just secretly hoping for a little Safari miracle.

Contributor

blacktrash commented Sep 22, 2015

Very useful debugging interface!

With TS segment files it's slightly different, it doesn't make the jump to the first frame of the next segment, but I guess the underlying issue is more or less similar.

No rush, I was just secretly hoping for a little Safari miracle.

@mangui mangui added the Enhancement label Sep 24, 2015

@mangui mangui changed the title from Mac OS desktop Safari compatibiliy to Mac OS desktop Safari compatibility Sep 24, 2015

blacktrash referenced this issue in flowplayer/flowplayer-hlsjs Sep 30, 2015

@domtra

This comment has been minimized.

Show comment
Hide comment
@domtra

domtra Oct 7, 2015

I know this issue is hard to debug but do you think it can be done? Or is there a way to prepare the ts files that these micro holes do not exist?
I would really like to use your library on safari and not have to fallback on native hls. From what I found out native hls in safari does not support manual bitrate switching and that is something I definitely need. Or do you know of another way to achieve manual bitrate switching?

Thanks for the great library btw. 👍

domtra commented Oct 7, 2015

I know this issue is hard to debug but do you think it can be done? Or is there a way to prepare the ts files that these micro holes do not exist?
I would really like to use your library on safari and not have to fallback on native hls. From what I found out native hls in safari does not support manual bitrate switching and that is something I definitely need. Or do you know of another way to achieve manual bitrate switching?

Thanks for the great library btw. 👍

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Oct 7, 2015

Member

thanks @domtra.
the issue should deal with the remuxing process, fmp4 box duration, maybe some rounding or timescale issues ... but without any access to Safari source code, it is hard to understand what is going on.
at that stage the only way to move forward would be to get some help/hints from the Safari team.
if anybody knows somebody there ... it could help !

Member

mangui commented Oct 7, 2015

thanks @domtra.
the issue should deal with the remuxing process, fmp4 box duration, maybe some rounding or timescale issues ... but without any access to Safari source code, it is hard to understand what is going on.
at that stage the only way to move forward would be to get some help/hints from the Safari team.
if anybody knows somebody there ... it could help !

@blacktrash

This comment has been minimized.

Show comment
Hide comment
@blacktrash

blacktrash Oct 7, 2015

Contributor

A 'brutal' alternative would be to use Dash in Safari ;-) But even then it would be extremely brittle, latest dash.js works - or rather happens to work - with Safari 9 (not 8), and only with certain segmentation methods ...

Contributor

blacktrash commented Oct 7, 2015

A 'brutal' alternative would be to use Dash in Safari ;-) But even then it would be extremely brittle, latest dash.js works - or rather happens to work - with Safari 9 (not 8), and only with certain segmentation methods ...

@domtra

This comment has been minimized.

Show comment
Hide comment
@domtra

domtra Oct 7, 2015

I actually did already tried that. Not only for Safari but for all browsers. However, their manual bitrate switching has a much bigger lag that hls.js. They always wait for the buffered content to have been played back completely before switching streams. This is unfortunately not acceptable either and is mainly the reason why I ended up using hls.js.

domtra commented Oct 7, 2015

I actually did already tried that. Not only for Safari but for all browsers. However, their manual bitrate switching has a much bigger lag that hls.js. They always wait for the buffered content to have been played back completely before switching streams. This is unfortunately not acceptable either and is mainly the reason why I ended up using hls.js.

@AxelDelmas

This comment has been minimized.

Show comment
Hide comment
@AxelDelmas

AxelDelmas Oct 28, 2015

Contributor

Is it possible to filter out Safari in the isSupported method? It sems that adding support for Safari is not trivial and that it won't happen short-term, so shouldn't isSupported reflect that?

It's a shame that we can't rely on MediaSource.isTypeSupported, is there any easy way to do it? Apart from parsing the UA?

Contributor

AxelDelmas commented Oct 28, 2015

Is it possible to filter out Safari in the isSupported method? It sems that adding support for Safari is not trivial and that it won't happen short-term, so shouldn't isSupported reflect that?

It's a shame that we can't rely on MediaSource.isTypeSupported, is there any easy way to do it? Apart from parsing the UA?

@blacktrash

This comment has been minimized.

Show comment
Hide comment
@blacktrash

blacktrash Oct 29, 2015

Contributor

Some news about Safari 9:

Contributor

blacktrash commented Oct 29, 2015

Some news about Safari 9:

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 18, 2016

@mangui If you could provide a sample transmuxed MPEG2-TS -> ISO BMFF file, I can probably show you where the holes between samples are coming from.

jernoble commented Mar 18, 2016

@mangui If you could provide a sample transmuxed MPEG2-TS -> ISO BMFF file, I can probably show you where the holes between samples are coming from.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 21, 2016

Member

@jernoble, thanks a lot for your proposal !

please find here the 6 appended ISO BMFF (results of the parsing of the first TS fragment of Big Buck Bunny)

audio_000.fmp4 (init segment)
audio_001.fmp4 (moof)
audio_002.fmp4 (mdat)

video_000.fmp4 (init segment)
video_001.fmp4 (moof)
video_002.fmp4 (mdat)

after appending those 6 segments, HTMLVideoElement.buffered looks like this with Safari Version 9.0.3 (10601.4.4)

media buffered : [0.03328888888888889,0.0999121111111111][0.1,0.14986766666666668][0.1499111111111111,0.19982322222222224][0.19986666666666666,0.24977877777777777][0.24982222222222222,0.3663565555555556][0.3664,0.39964544444444444][0.39968888888888887,0.4662676666666667][0.4663111111111111,0.49955655555555556][0.4996,0.5994676666666666][0.5995555555555555,0.6494232222222223][0.6494666666666666,0.7160454444444444][0.7160888888888889,0.7493343333333333][0.7493777777777778,0.8492454444444445][0.8493333333333334,0.9158676666666666][0.9159111111111111,0.9491565555555556][0.9492,1.0324454444444444][1.0324888888888888,1.0990232222222223][1.0991111111111111,1.1989343333333333][1.1990222222222222,1.2488898888888889][1.2489333333333332,1.2988454444444444][1.298888888888889,1.348801][1.3488444444444445,1.4154232222222223][1.4154666666666667,1.4487121111111112][1.4487555555555556,1.532001][1.5320444444444445,1.5985787777777778][1.5986666666666667,1.665201][1.6652444444444445,1.6984898888888889][1.6985333333333332,1.798401][1.7984888888888888,1.8650232222222223][1.8650666666666667,1.8983121111111112][1.8983555555555556,1.9982232222222223][1.9983111111111111,2.0648454444444444][2.064888888888889,2.0981343333333333][2.098177777777778,2.1980454444444444][2.1981333333333333,2.248001][2.2480444444444445,2.314623222222222][2.3146666666666667,2.347912111111111][2.3479555555555556,2.447823222222222][2.447911111111111,2.5144454444444446][2.5144888888888888,2.5477343333333335][2.5477777777777777,2.6476454444444446][2.6477333333333335,2.7142676666666667][2.714311111111111,2.7475565555555557][2.7476,2.8474676666666667][2.8475555555555556,2.914089888888889][2.9141333333333335,2.947378777777778][2.9474222222222224,3.047289888888889][3.0473777777777777,3.113912111111111][3.1139555555555556,3.147201][3.1472444444444445,3.247112111111111][3.2472,3.313734333333333][3.3137777777777777,3.347023222222222][3.3470666666666666,3.4469343333333335][3.4470222222222224,3.5135565555555557][3.5136,3.5468454444444446][3.5468888888888888,3.6467565555555557][3.6468444444444446,3.696712111111111][3.6967555555555554,3.7633343333333333][3.763377777777778,3.7966232222222223][3.796666666666667,3.846578777777778][3.8466222222222224,3.8965343333333333][3.896577777777778,3.9631565555555555][3.9632,3.9964454444444444][3.996488888888889,4.0963565555555554][4.0964,4.1962676666666665][4.196355555555556,4.2462232222222225][4.246266666666667,4.312845444444444][4.312888888888889,4.3461343333333335][4.346177777777778,4.429423222222222][4.429466666666666,4.512667666666666][4.512711111111111,4.545956555555556][4.546,4.612578777777777][4.612622222222222,4.645867666666667][4.645911111111111,4.712489888888889][4.712533333333333,4.745778777777778][4.745822222222222,4.845689888888889][4.845777777777778,4.895645444444445][4.8956888888888885,4.9622676666666665][4.962311111111111,4.995556555555556][4.9956,5.095467666666667][5.095555555555555,5.145423222222222][5.145466666666667,5.245334333333333][5.245422222222222,5.311956555555556][5.312,5.345245444444444][5.345288888888889,5.428534333333333][5.428577777777778,5.478489888888889][5.478533333333333,5.528445444444444][5.528488888888889,5.595023222222222][5.595111111111111,5.661645444444445][5.6616888888888885,5.694934333333333][5.694977777777778,5.744889888888889][5.744933333333333,5.794845444444444][5.794888888888889,5.844801][5.844844444444444,5.911423222222222][5.911466666666667,5.944712111111111][5.944755555555555,6.0446232222222225][6.044711111111111,6.111245444444444][6.111288888888889,6.1445343333333335][6.144577777777778,6.244445444444445][6.244533333333333,6.311067666666666][6.311111111111111,6.344356555555556][6.3444,6.444267666666667][6.444355555555555,6.510889888888889][6.510933333333333,6.544178777777778][6.544222222222222,6.644089888888889][6.644177777777778,6.710712111111111][6.710755555555555,6.744001][6.7440444444444445,6.7606676666666665][6.760711111111111,6.793956555555556][6.794,6.8106232222222225][6.810666666666667,6.843912111111111][6.843955555555556,6.910534333333334][6.910577777777778,6.943823222222222][6.943866666666667,7.043734333333333][7.043822222222222,7.110356555555556][7.1104,7.193601][7.193688888888889,7.260223222222222][7.260266666666666,7.293512111111111][7.293555555555556,7.393423222222222][7.393511111111111,7.460045444444445][7.4600888888888885,7.493334333333333][7.493377777777778,7.5766232222222225][7.576666666666667,7.609912111111111][7.609955555555556,7.643201][7.643244444444444,7.743112111111111][7.7432,7.809734333333333][7.809777777777778,7.8430232222222225][7.843066666666667,7.9429343333333335][7.943022222222222,8.009556555555555][8.0096,8.042845444444444][8.04288888888889,8.142756555555556][8.142844444444444,8.209378777777777][8.209422222222223,8.242667666666666][8.242711111111111,8.342578777777778][8.342666666666666,8.392534333333334][8.392577777777777,8.459156555555555][8.4592,8.492445444444444][8.49248888888889,8.592356555555556][8.592444444444444,8.658978777777778][8.659022222222223,8.692267666666666][8.692311111111112,8.792178777777778][8.792266666666666,8.858801][8.858844444444445,8.892089888888888][8.892133333333334,8.992001][8.992088888888889,9.058623222222222][9.058666666666667,9.09191211111111][9.091955555555556,9.191823222222222][9.19191111111111,9.258445444444444][9.25848888888889,9.291734333333334][9.291777777777778,9.391645444444444][9.391733333333333,9.458267666666666][9.458311111111112,9.491556555555556][9.4916,9.591467666666667][9.591555555555555,9.658089888888888][9.658133333333334,9.691378777777778][9.691422222222222,9.791289888888889][9.791377777777777,9.85791211111111][9.857955555555556,9.891201][9.891244444444444,9.99111211111111][9.9912,10.00780145351474]
Member

mangui commented Mar 21, 2016

@jernoble, thanks a lot for your proposal !

please find here the 6 appended ISO BMFF (results of the parsing of the first TS fragment of Big Buck Bunny)

audio_000.fmp4 (init segment)
audio_001.fmp4 (moof)
audio_002.fmp4 (mdat)

video_000.fmp4 (init segment)
video_001.fmp4 (moof)
video_002.fmp4 (mdat)

after appending those 6 segments, HTMLVideoElement.buffered looks like this with Safari Version 9.0.3 (10601.4.4)

media buffered : [0.03328888888888889,0.0999121111111111][0.1,0.14986766666666668][0.1499111111111111,0.19982322222222224][0.19986666666666666,0.24977877777777777][0.24982222222222222,0.3663565555555556][0.3664,0.39964544444444444][0.39968888888888887,0.4662676666666667][0.4663111111111111,0.49955655555555556][0.4996,0.5994676666666666][0.5995555555555555,0.6494232222222223][0.6494666666666666,0.7160454444444444][0.7160888888888889,0.7493343333333333][0.7493777777777778,0.8492454444444445][0.8493333333333334,0.9158676666666666][0.9159111111111111,0.9491565555555556][0.9492,1.0324454444444444][1.0324888888888888,1.0990232222222223][1.0991111111111111,1.1989343333333333][1.1990222222222222,1.2488898888888889][1.2489333333333332,1.2988454444444444][1.298888888888889,1.348801][1.3488444444444445,1.4154232222222223][1.4154666666666667,1.4487121111111112][1.4487555555555556,1.532001][1.5320444444444445,1.5985787777777778][1.5986666666666667,1.665201][1.6652444444444445,1.6984898888888889][1.6985333333333332,1.798401][1.7984888888888888,1.8650232222222223][1.8650666666666667,1.8983121111111112][1.8983555555555556,1.9982232222222223][1.9983111111111111,2.0648454444444444][2.064888888888889,2.0981343333333333][2.098177777777778,2.1980454444444444][2.1981333333333333,2.248001][2.2480444444444445,2.314623222222222][2.3146666666666667,2.347912111111111][2.3479555555555556,2.447823222222222][2.447911111111111,2.5144454444444446][2.5144888888888888,2.5477343333333335][2.5477777777777777,2.6476454444444446][2.6477333333333335,2.7142676666666667][2.714311111111111,2.7475565555555557][2.7476,2.8474676666666667][2.8475555555555556,2.914089888888889][2.9141333333333335,2.947378777777778][2.9474222222222224,3.047289888888889][3.0473777777777777,3.113912111111111][3.1139555555555556,3.147201][3.1472444444444445,3.247112111111111][3.2472,3.313734333333333][3.3137777777777777,3.347023222222222][3.3470666666666666,3.4469343333333335][3.4470222222222224,3.5135565555555557][3.5136,3.5468454444444446][3.5468888888888888,3.6467565555555557][3.6468444444444446,3.696712111111111][3.6967555555555554,3.7633343333333333][3.763377777777778,3.7966232222222223][3.796666666666667,3.846578777777778][3.8466222222222224,3.8965343333333333][3.896577777777778,3.9631565555555555][3.9632,3.9964454444444444][3.996488888888889,4.0963565555555554][4.0964,4.1962676666666665][4.196355555555556,4.2462232222222225][4.246266666666667,4.312845444444444][4.312888888888889,4.3461343333333335][4.346177777777778,4.429423222222222][4.429466666666666,4.512667666666666][4.512711111111111,4.545956555555556][4.546,4.612578777777777][4.612622222222222,4.645867666666667][4.645911111111111,4.712489888888889][4.712533333333333,4.745778777777778][4.745822222222222,4.845689888888889][4.845777777777778,4.895645444444445][4.8956888888888885,4.9622676666666665][4.962311111111111,4.995556555555556][4.9956,5.095467666666667][5.095555555555555,5.145423222222222][5.145466666666667,5.245334333333333][5.245422222222222,5.311956555555556][5.312,5.345245444444444][5.345288888888889,5.428534333333333][5.428577777777778,5.478489888888889][5.478533333333333,5.528445444444444][5.528488888888889,5.595023222222222][5.595111111111111,5.661645444444445][5.6616888888888885,5.694934333333333][5.694977777777778,5.744889888888889][5.744933333333333,5.794845444444444][5.794888888888889,5.844801][5.844844444444444,5.911423222222222][5.911466666666667,5.944712111111111][5.944755555555555,6.0446232222222225][6.044711111111111,6.111245444444444][6.111288888888889,6.1445343333333335][6.144577777777778,6.244445444444445][6.244533333333333,6.311067666666666][6.311111111111111,6.344356555555556][6.3444,6.444267666666667][6.444355555555555,6.510889888888889][6.510933333333333,6.544178777777778][6.544222222222222,6.644089888888889][6.644177777777778,6.710712111111111][6.710755555555555,6.744001][6.7440444444444445,6.7606676666666665][6.760711111111111,6.793956555555556][6.794,6.8106232222222225][6.810666666666667,6.843912111111111][6.843955555555556,6.910534333333334][6.910577777777778,6.943823222222222][6.943866666666667,7.043734333333333][7.043822222222222,7.110356555555556][7.1104,7.193601][7.193688888888889,7.260223222222222][7.260266666666666,7.293512111111111][7.293555555555556,7.393423222222222][7.393511111111111,7.460045444444445][7.4600888888888885,7.493334333333333][7.493377777777778,7.5766232222222225][7.576666666666667,7.609912111111111][7.609955555555556,7.643201][7.643244444444444,7.743112111111111][7.7432,7.809734333333333][7.809777777777778,7.8430232222222225][7.843066666666667,7.9429343333333335][7.943022222222222,8.009556555555555][8.0096,8.042845444444444][8.04288888888889,8.142756555555556][8.142844444444444,8.209378777777777][8.209422222222223,8.242667666666666][8.242711111111111,8.342578777777778][8.342666666666666,8.392534333333334][8.392577777777777,8.459156555555555][8.4592,8.492445444444444][8.49248888888889,8.592356555555556][8.592444444444444,8.658978777777778][8.659022222222223,8.692267666666666][8.692311111111112,8.792178777777778][8.792266666666666,8.858801][8.858844444444445,8.892089888888888][8.892133333333334,8.992001][8.992088888888889,9.058623222222222][9.058666666666667,9.09191211111111][9.091955555555556,9.191823222222222][9.19191111111111,9.258445444444444][9.25848888888889,9.291734333333334][9.291777777777778,9.391645444444444][9.391733333333333,9.458267666666666][9.458311111111112,9.491556555555556][9.4916,9.591467666666667][9.591555555555555,9.658089888888888][9.658133333333334,9.691378777777778][9.691422222222222,9.791289888888889][9.791377777777777,9.85791211111111][9.857955555555556,9.891201][9.891244444444444,9.99111211111111][9.9912,10.00780145351474]
@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 21, 2016

@mangui, thanks for the sample files. The first thing I notice is that the moov box for the video media doesn't have an edit box (edts). A common technique for moving the presentation timestamp back to 0:00 is to add an edit list with an empty edit, the duration of which matches the PTS of the first sample. For this file, the edit box should contain an edit list (elst) with a single edit of { duration:0, time:749, rate:1, fraction:0 }. This will take care of the gap at the start of the video file.

(Still looking into the rest of the gaps)

jernoble commented Mar 21, 2016

@mangui, thanks for the sample files. The first thing I notice is that the moov box for the video media doesn't have an edit box (edts). A common technique for moving the presentation timestamp back to 0:00 is to add an edit list with an empty edit, the duration of which matches the PTS of the first sample. For this file, the edit box should contain an edit list (elst) with a single edit of { duration:0, time:749, rate:1, fraction:0 }. This will take care of the gap at the start of the video file.

(Still looking into the rest of the gaps)

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 21, 2016

So, there's not just gaps between the end of one sample and the start of the next, but also presentation time overlaps between two samples.

Here's a simplified table of presentation times for the first ~20 samples in video.fmp4, sorted in presentation order:

Format: [PTS start] + [Duration] =  [PTS end] - [next PTS start] = [overlap/gap]
 749 + 375 = 1124 - 1124 =  0
1124 + 375 = 1499 - 1499 =  0
1499 + 375 = 1874 - 1874 =  0
1874 + 374 = 2248 - 2250 = -2
2250 + 374 = 2624 - 2623 = +1
2623 + 375 = 2998 - 2998 =  0
2998 + 374 = 3372 - 3373 = -1
3373 + 375 = 3748 - 3747 = +1
3747 + 374 = 4121 - 4121 =  0
4121 + 375 = 4496 - 4497 = -1
4497 + 375 = 4872 - 4871 = +1
4871 + 374 = 5245 - 5245 =  0
5245 + 375 = 5620 - 5621 = -1
5621 + 375 = 5996 - 5995 = +1
5995 + 375 = 6370 - 6370 =  0
6370 + 375 = 6745 - 6745 =  0
6745 + 374 = 7119 - 7119 =  0
7119 + 374 = 7493 - 7493 =  0
7493 + 375 = 7868 - 7868 =  0
7868 + 375 = 8243 - 8244 = -1
8244 + 375 = 8619 - 8618 = +1
8618 + 374 = 8992 - 8993 = -1
8993 + 375 = 9368 - 9367 = +1
9367 + 374 = 9741 - 9741 =  0

I can't really discern a pattern here.

jernoble commented Mar 21, 2016

So, there's not just gaps between the end of one sample and the start of the next, but also presentation time overlaps between two samples.

Here's a simplified table of presentation times for the first ~20 samples in video.fmp4, sorted in presentation order:

Format: [PTS start] + [Duration] =  [PTS end] - [next PTS start] = [overlap/gap]
 749 + 375 = 1124 - 1124 =  0
1124 + 375 = 1499 - 1499 =  0
1499 + 375 = 1874 - 1874 =  0
1874 + 374 = 2248 - 2250 = -2
2250 + 374 = 2624 - 2623 = +1
2623 + 375 = 2998 - 2998 =  0
2998 + 374 = 3372 - 3373 = -1
3373 + 375 = 3748 - 3747 = +1
3747 + 374 = 4121 - 4121 =  0
4121 + 375 = 4496 - 4497 = -1
4497 + 375 = 4872 - 4871 = +1
4871 + 374 = 5245 - 5245 =  0
5245 + 375 = 5620 - 5621 = -1
5621 + 375 = 5996 - 5995 = +1
5995 + 375 = 6370 - 6370 =  0
6370 + 375 = 6745 - 6745 =  0
6745 + 374 = 7119 - 7119 =  0
7119 + 374 = 7493 - 7493 =  0
7493 + 375 = 7868 - 7868 =  0
7868 + 375 = 8243 - 8244 = -1
8244 + 375 = 8619 - 8618 = +1
8618 + 374 = 8992 - 8993 = -1
8993 + 375 = 9368 - 9367 = +1
9367 + 374 = 9741 - 9741 =  0

I can't really discern a pattern here.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 21, 2016

Member

ok the way video sample duration was computed was incorrect,
I also changed the timescale (I kept the original PES timescale, 90kHz) to avoid rounding issues
I dumped similar table in JS and now I got relevant data

capture d ecran 2016-03-21 a 23 43 36

I am puzzled as I still get plenty of holes in video buffered.

I dumped the init segment and moof here
mdat is unchanged.

Member

mangui commented Mar 21, 2016

ok the way video sample duration was computed was incorrect,
I also changed the timescale (I kept the original PES timescale, 90kHz) to avoid rounding issues
I dumped similar table in JS and now I got relevant data

capture d ecran 2016-03-21 a 23 43 36

I am puzzled as I still get plenty of holes in video buffered.

I dumped the init segment and moof here
mdat is unchanged.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 22, 2016

Member

@jernoble
here is an updated screenshot and spreadsheet showing that there shouldn't be any overlap/gap
the red color indicates holes location in video.buffered
capture d ecran 2016-03-22 a 13 06 12
safari_hole.xlsx

media buffered : [0.033322222222222225,0.050001][0.05001111111111111,0.100001][0.1000111111111111,0.1666787777777778][0.1666888888888889,0.2000121111111111][0.2000222222222222,0.2500121111111111][0.2500222222222222,0.31667877777777775][0.3167,0.3500232222222222][0.3500333333333333,0.4166787777777778][0.4166888888888889,0.4500121111111111][0.4500222222222222,0.5166787777777778][0.5166888888888889,0.550001][0.5500222222222222,0.6166787777777778][0.6166888888888888,0.6500121111111111][0.6500222222222222,0.7000121111111111][0.7000222222222222,0.7666787777777778][0.7666888888888889,0.800001][0.8000222222222222,0.8666787777777778][0.8666888888888888,0.9000121111111111] ....

Member

mangui commented Mar 22, 2016

@jernoble
here is an updated screenshot and spreadsheet showing that there shouldn't be any overlap/gap
the red color indicates holes location in video.buffered
capture d ecran 2016-03-22 a 13 06 12
safari_hole.xlsx

media buffered : [0.033322222222222225,0.050001][0.05001111111111111,0.100001][0.1000111111111111,0.1666787777777778][0.1666888888888889,0.2000121111111111][0.2000222222222222,0.2500121111111111][0.2500222222222222,0.31667877777777775][0.3167,0.3500232222222222][0.3500333333333333,0.4166787777777778][0.4166888888888889,0.4500121111111111][0.4500222222222222,0.5166787777777778][0.5166888888888889,0.550001][0.5500222222222222,0.6166787777777778][0.6166888888888888,0.6500121111111111][0.6500222222222222,0.7000121111111111][0.7000222222222222,0.7666787777777778][0.7666888888888889,0.800001][0.8000222222222222,0.8666787777777778][0.8666888888888888,0.9000121111111111] ....

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 22, 2016

Member

you can reproduce here
it hosts latest dist of safari-fix branch

Member

mangui commented Mar 22, 2016

you can reproduce here
it hosts latest dist of safari-fix branch

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 22, 2016

@mangul, I have your safari-fix branch checked out, and I see the following at mp4-remuxer.js:234:

> samples
< Array (600) = $2
0 {size: 1044, duration: 375, cts: 749.75, flags: Object}
1 {size: 40, duration: 374.75, cts: 1875, flags: Object}
2 {size: 37, duration: 375.25, cts: 750, flags: Object}
3 {size: 37, duration: 375, cts: 0, flags: Object}
4 {size: 37, duration: 374.75, cts: 375, flags: Object}
5 {size: 8700, duration: 375.25, cts: 750, flags: Object}
6 {size: 6214, duration: 375, cts: 1125, flags: Object}
7 {size: 274, duration: 374.75, cts: 375, flags: Object}
8 {size: 5050, duration: 375.25, cts: 1500.25, flags: Object}
9 {size: 1582, duration: 375, cts: 749.75, flags: Object}
[...snip...]

Those fractional durations and cts make me very suspicious. It looks like they're getting truncated before being inserted into the trun box.

jernoble commented Mar 22, 2016

@mangul, I have your safari-fix branch checked out, and I see the following at mp4-remuxer.js:234:

> samples
< Array (600) = $2
0 {size: 1044, duration: 375, cts: 749.75, flags: Object}
1 {size: 40, duration: 374.75, cts: 1875, flags: Object}
2 {size: 37, duration: 375.25, cts: 750, flags: Object}
3 {size: 37, duration: 375, cts: 0, flags: Object}
4 {size: 37, duration: 374.75, cts: 375, flags: Object}
5 {size: 8700, duration: 375.25, cts: 750, flags: Object}
6 {size: 6214, duration: 375, cts: 1125, flags: Object}
7 {size: 274, duration: 374.75, cts: 375, flags: Object}
8 {size: 5050, duration: 375.25, cts: 1500.25, flags: Object}
9 {size: 1582, duration: 375, cts: 749.75, flags: Object}
[...snip...]

Those fractional durations and cts make me very suspicious. It looks like they're getting truncated before being inserted into the trun box.

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 22, 2016

And if you take those samples and run the following against them:

window.ptss = samples.map((sample, index) => { 
   var retval = {}; 
   var duration = Math.floor(sample.duration);
   retval.cts = lastCTS; lastCTS += duration;
   retval.pts = retval.cts + Math.floor(sample.cts);
   retval.duration = duration;
   retval.endPTS = retval.pts + duration;
   return retval; 
})

Then sort the result by pts, you get the following:

Array (600) = $9
0 {cts: 0, pts: 749, duration: 375, endPTS: 1124}
1 {cts: 1124, pts: 1124, duration: 375, endPTS: 1499}
2 {cts: 749, pts: 1499, duration: 375, endPTS: 1874}
3 {cts: 1499, pts: 1874, duration: 374, endPTS: 2248}
4 {cts: 375, pts: 2250, duration: 374, endPTS: 2624}
5 {cts: 1873, pts: 2623, duration: 375, endPTS: 2998}
6 {cts: 2623, pts: 2998, duration: 374, endPTS: 3372}
7 {cts: 2248, pts: 3373, duration: 375, endPTS: 3748}
8 {cts: 3747, pts: 3747, duration: 374, endPTS: 4121}
9 {cts: 3372, pts: 4121, duration: 375, endPTS: 4496}
[...snip...]

And you can see the gap between samples 3 & 4.

jernoble commented Mar 22, 2016

And if you take those samples and run the following against them:

window.ptss = samples.map((sample, index) => { 
   var retval = {}; 
   var duration = Math.floor(sample.duration);
   retval.cts = lastCTS; lastCTS += duration;
   retval.pts = retval.cts + Math.floor(sample.cts);
   retval.duration = duration;
   retval.endPTS = retval.pts + duration;
   return retval; 
})

Then sort the result by pts, you get the following:

Array (600) = $9
0 {cts: 0, pts: 749, duration: 375, endPTS: 1124}
1 {cts: 1124, pts: 1124, duration: 375, endPTS: 1499}
2 {cts: 749, pts: 1499, duration: 375, endPTS: 1874}
3 {cts: 1499, pts: 1874, duration: 374, endPTS: 2248}
4 {cts: 375, pts: 2250, duration: 374, endPTS: 2624}
5 {cts: 1873, pts: 2623, duration: 375, endPTS: 2998}
6 {cts: 2623, pts: 2998, duration: 374, endPTS: 3372}
7 {cts: 2248, pts: 3373, duration: 375, endPTS: 3748}
8 {cts: 3747, pts: 3747, duration: 374, endPTS: 4121}
9 {cts: 3372, pts: 4121, duration: 375, endPTS: 4496}
[...snip...]

And you can see the gap between samples 3 & 4.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 22, 2016

Member

mmm i removed the scale by 4 factor, so you should not have anymore fractional values.
see dailymotion@d80e4f6
what matters is what is printed on the console here:
https://github.com/dailymotion/hls.js/blob/safari-fix/src/remux/mp4-remuxer.js#L265

Member

mangui commented Mar 22, 2016

mmm i removed the scale by 4 factor, so you should not have anymore fractional values.
see dailymotion@d80e4f6
what matters is what is printed on the console here:
https://github.com/dailymotion/hls.js/blob/safari-fix/src/remux/mp4-remuxer.js#L265

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 22, 2016

Member

all this is dumped on the console if webworker is disabled:
http://streambox.fr/mse/hls.js-safari-fix/demo/?src=http%3A%2F%2Fwww.streambox.fr%2Fplaylists%2Fx36xhzz%2Fx36xhzz.m3u8&enableStreaming=true&autoRecoverError=true&enableWorker=false&levelCapping=-1&defaultAudioCodec=undefined

[Log] start,duration,end,cts (hls.js, line 7326)
[Log] 2999,1501,4500,2999 (hls.js, line 7328)
[Log] 9000,1499,10499,7500 (hls.js, line 7328)
[Log] 5999,1501,7500,3000 (hls.js, line 7328)
[Log] 4500,1499,5999,0 (hls.js, line 7328)
[Log] 7500,1500,9000,1500 (hls.js, line 7328)
[Log] 10499,1501,12000,3000 (hls.js, line 7328)
[Log] 13500,1500,15000,4500 (hls.js, line 7328)
[Log] 12000,1500,13500,1500 (hls.js, line 7328)
[Log] 18000,1500,19500,6001 (hls.js, line 7328)
[Log] 16499,1501,18000,2999 (hls.js, line 7328)
[Log] 15000,1499,16499,0 (hls.js, line 7328)
[Log] 22500,1500,24000,6001 (hls.js, line 7328)
[Log] 20999,1501,22500,2999 (hls.js, line 7328)
[Log] 19500,1499,20999,0 (hls.js, line 7328)
[Log] 25499,1501,27000,4500 (hls.js, line 7328)
[Log] 24000,1499,25499,1500 (hls.js, line 7328)
[Log] 27000,1500,28500,3000 (hls.js, line 7328)
[Log] 32999,1501,34500,7500 (hls.js, line 7328)
[Log] 29999,1501,31500,2999 (hls.js, line 7328)
[Log] 28500,1499,29999,0 (hls.js, line 7328)
[Log] 31500,1499,32999,1501 (hls.js, line 7328)
[Log] 36000,1500,37500,4500 (hls.js, line 7328)
[Log] 34500,1500,36000,1500 (hls.js, line 7328)
[Log] 41999,1500,43499,7500 (hls.js, line 7328)




Member

mangui commented Mar 22, 2016

all this is dumped on the console if webworker is disabled:
http://streambox.fr/mse/hls.js-safari-fix/demo/?src=http%3A%2F%2Fwww.streambox.fr%2Fplaylists%2Fx36xhzz%2Fx36xhzz.m3u8&enableStreaming=true&autoRecoverError=true&enableWorker=false&levelCapping=-1&defaultAudioCodec=undefined

[Log] start,duration,end,cts (hls.js, line 7326)
[Log] 2999,1501,4500,2999 (hls.js, line 7328)
[Log] 9000,1499,10499,7500 (hls.js, line 7328)
[Log] 5999,1501,7500,3000 (hls.js, line 7328)
[Log] 4500,1499,5999,0 (hls.js, line 7328)
[Log] 7500,1500,9000,1500 (hls.js, line 7328)
[Log] 10499,1501,12000,3000 (hls.js, line 7328)
[Log] 13500,1500,15000,4500 (hls.js, line 7328)
[Log] 12000,1500,13500,1500 (hls.js, line 7328)
[Log] 18000,1500,19500,6001 (hls.js, line 7328)
[Log] 16499,1501,18000,2999 (hls.js, line 7328)
[Log] 15000,1499,16499,0 (hls.js, line 7328)
[Log] 22500,1500,24000,6001 (hls.js, line 7328)
[Log] 20999,1501,22500,2999 (hls.js, line 7328)
[Log] 19500,1499,20999,0 (hls.js, line 7328)
[Log] 25499,1501,27000,4500 (hls.js, line 7328)
[Log] 24000,1499,25499,1500 (hls.js, line 7328)
[Log] 27000,1500,28500,3000 (hls.js, line 7328)
[Log] 32999,1501,34500,7500 (hls.js, line 7328)
[Log] 29999,1501,31500,2999 (hls.js, line 7328)
[Log] 28500,1499,29999,0 (hls.js, line 7328)
[Log] 31500,1499,32999,1501 (hls.js, line 7328)
[Log] 36000,1500,37500,4500 (hls.js, line 7328)
[Log] 34500,1500,36000,1500 (hls.js, line 7328)
[Log] 41999,1500,43499,7500 (hls.js, line 7328)




@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 22, 2016

It looks like there's still an off-by-one error somewhere, because (after updating to your most recent commit), the result of parsing is the following:

(start pts, duration, end pts)
2999, 1501, 4500
9001, 1499, 10500
6000, 1501, 7501
4501, 1499, 6000
7500, 1500, 9000
10500, 1501, 12001
13501, 1500, 15001
12001, 1500, 13501
18002, 1500, 19502
16500, 1501, 18001
15002, 1499, 16501
22502, 1500, 24002
21000, 1501, 22501
19502, 1499, 21001
25501, 1501, 27002
24002, 1499, 25501
27001, 1500, 28501
33001, 1501, 34502
30001, 1501, 31502
28503, 1499, 30002
31503, 1499, 33002
36001, 1500, 37501
34501, 1500, 36001
42001, 1500, 43501
39000, 1501, 40501
37502, 1499, 39001
40502, 1499, 42001
47999, 1501, 49500
45001, 1500, 46501
43501, 1501, 45002

jernoble commented Mar 22, 2016

It looks like there's still an off-by-one error somewhere, because (after updating to your most recent commit), the result of parsing is the following:

(start pts, duration, end pts)
2999, 1501, 4500
9001, 1499, 10500
6000, 1501, 7501
4501, 1499, 6000
7500, 1500, 9000
10500, 1501, 12001
13501, 1500, 15001
12001, 1500, 13501
18002, 1500, 19502
16500, 1501, 18001
15002, 1499, 16501
22502, 1500, 24002
21000, 1501, 22501
19502, 1499, 21001
25501, 1501, 27002
24002, 1499, 25501
27001, 1500, 28501
33001, 1501, 34502
30001, 1501, 31502
28503, 1499, 30002
31503, 1499, 33002
36001, 1500, 37501
34501, 1500, 36001
42001, 1500, 43501
39000, 1501, 40501
37502, 1499, 39001
40502, 1499, 42001
47999, 1501, 49500
45001, 1500, 46501
43501, 1501, 45002
@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Mar 22, 2016

Member

thanks for the feedback,
regarding your last comment, how did you obtain these traces ? by adding some JS logs ? or by analyzing offline the generated MP4 trun ?

Member

mangui commented Mar 22, 2016

thanks for the feedback,
regarding your last comment, how did you obtain these traces ? by adding some JS logs ? or by analyzing offline the generated MP4 trun ?

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Mar 22, 2016

Analyzing the generated trun. I think I see the problem with the gap @ 4500 -> 4501. If you look at the logs you've posted:

[Log] 2999,1501,4500,2999 (hls.js, line 7328)
[Log] 9000,1499,10499,7500 (hls.js, line 7328)
[Log] 5999,1501,7500,3000 (hls.js, line 7328)
[Log] 4500,1499,5999,0 (hls.js, line 7328)

You can derive the DTS from the first four durations:

1501 + 0 = 1501
1499 + 1501 = 3000
1501 + 3000 = 4501
1499 + 4501 = 6000

You expect the 4th (in DTS order) sample to start at 4500, but it's actually 4501.

jernoble commented Mar 22, 2016

Analyzing the generated trun. I think I see the problem with the gap @ 4500 -> 4501. If you look at the logs you've posted:

[Log] 2999,1501,4500,2999 (hls.js, line 7328)
[Log] 9000,1499,10499,7500 (hls.js, line 7328)
[Log] 5999,1501,7500,3000 (hls.js, line 7328)
[Log] 4500,1499,5999,0 (hls.js, line 7328)

You can derive the DTS from the first four durations:

1501 + 0 = 1501
1499 + 1501 = 3000
1501 + 3000 = 4501
1499 + 4501 = 6000

You expect the 4th (in DTS order) sample to start at 4500, but it's actually 4501.

@quiz1991

This comment has been minimized.

Show comment
Hide comment
@quiz1991

quiz1991 May 3, 2016

Hi, the edts box works perfectly to make video start on Safari. Thanks!
But the video still cannot play through because of the tiny buffer holes.

And I've checked that these two requirements that you mentioned are already met:
1. sample duration (DTS delta) is constant, and
2. cts is always a multiple of sample duration.

What's your suggestion to fix the buffer holes?

quiz1991 commented May 3, 2016

Hi, the edts box works perfectly to make video start on Safari. Thanks!
But the video still cannot play through because of the tiny buffer holes.

And I've checked that these two requirements that you mentioned are already met:
1. sample duration (DTS delta) is constant, and
2. cts is always a multiple of sample duration.

What's your suggestion to fix the buffer holes?

@drichards

This comment has been minimized.

Show comment
Hide comment
@drichards

drichards Jul 7, 2016

@mangui + @jernoble I took a stab at making the edit box changes you were discussing above. It doesn't look like they made much difference on safari startup though. Maybe one or both of you could take a quick look to see if I'm doing something obviously wrong.

drichards commented Jul 7, 2016

@mangui + @jernoble I took a stab at making the edit box changes you were discussing above. It doesn't look like they made much difference on safari startup though. Maybe one or both of you could take a quick look to see if I'm doing something obviously wrong.

@cconcolato

This comment has been minimized.

Show comment
Hide comment
@cconcolato

cconcolato Jul 8, 2016

Edit lists are a movie-level construct, so anything you add to the edit list will affect all tracks in the movie. I.e., if you are multiplexing audio and video into a single SourceBuffer, then the edit list will affect both tracks.

Just correcting that. The edit list is a track construct. It is usually used to anchor a track on the movie timeline. In other words, if you don't play that track, that edit list has no effect.

cconcolato commented Jul 8, 2016

Edit lists are a movie-level construct, so anything you add to the edit list will affect all tracks in the movie. I.e., if you are multiplexing audio and video into a single SourceBuffer, then the edit list will affect both tracks.

Just correcting that. The edit list is a track construct. It is usually used to anchor a track on the movie timeline. In other words, if you don't play that track, that edit list has no effect.

@mangui

This comment has been minimized.

Show comment
Hide comment
@mangui

mangui Jul 11, 2016

Member

Hi folks,
adding edts/elst box seems to help improving playback startup on Safari
you can check with your streams here
http://streambox.fr/mse/hls.js-elst/demo/
feedback welcome
related to https://github.com/dailymotion/hls.js/pull/553

Member

mangui commented Jul 11, 2016

Hi folks,
adding edts/elst box seems to help improving playback startup on Safari
you can check with your streams here
http://streambox.fr/mse/hls.js-elst/demo/
feedback welcome
related to https://github.com/dailymotion/hls.js/pull/553

@shoremark shoremark referenced this issue Sep 2, 2016

Closed

Twitching in Safari #662

4 of 4 tasks complete
@xqq

This comment has been minimized.

Show comment
Hide comment
@xqq

xqq Sep 14, 2016

Safari Technology Preview works well (even without edts!)

xqq commented Sep 14, 2016

Safari Technology Preview works well (even without edts!)

@blacktrash

This comment has been minimized.

Show comment
Hide comment
@blacktrash

blacktrash Sep 20, 2016

Contributor

Quick tests with Safari 10: seems to work better, but only at first glance.
Some videos work ok, with others there are lots of encoding errors, artifacts, or "psychedelic" colors (which did not happen with Safari < 10!). All videos encoded the same or in a very similar way.

Contributor

blacktrash commented Sep 20, 2016

Quick tests with Safari 10: seems to work better, but only at first glance.
Some videos work ok, with others there are lots of encoding errors, artifacts, or "psychedelic" colors (which did not happen with Safari < 10!). All videos encoded the same or in a very similar way.

@blacktrash blacktrash referenced this issue Nov 24, 2016

Closed

Safari green artifacts in some streams #843

4 of 4 tasks complete
@Urkass

This comment has been minimized.

Show comment
Hide comment
@Urkass

Urkass Dec 28, 2016

Hello, what are the plans for adding edts/elst box, that fixes safari 9 twitching, to master? (this branch)

Urkass commented Dec 28, 2016

Hello, what are the plans for adding edts/elst box, that fixes safari 9 twitching, to master? (this branch)

@murilozilli

This comment has been minimized.

Show comment
Hide comment
@murilozilli

murilozilli Sep 18, 2017

@mangui

but plz note mobile safari does not support MSE, only desktop safari does.

How can I play HLS videos on mobile safari then?

murilozilli commented Sep 18, 2017

@mangui

but plz note mobile safari does not support MSE, only desktop safari does.

How can I play HLS videos on mobile safari then?

@jernoble

This comment has been minimized.

Show comment
Hide comment
@jernoble

jernoble Sep 18, 2017

Safari has native support for HLS, so just make a <video src="manifest.m3u8" controls> element.

jernoble commented Sep 18, 2017

Safari has native support for HLS, so just make a <video src="manifest.m3u8" controls> element.

mangui pushed a commit that referenced this issue Mar 22, 2018

@stale

This comment has been minimized.

Show comment
Hide comment
@stale

stale bot Jun 16, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale bot commented Jun 16, 2018

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the Wontfix label Jun 16, 2018

@stale stale bot closed this Jun 23, 2018

johnBartos pushed a commit that referenced this issue Jun 27, 2018

Merge pull request #9 from jwplayer/feature/play-aac-without-id3-time…
…stamps

Playback streams without ID3 data or ID3 timestamp data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment