New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add support for AVC fmp4 fragments found in Apple Advanced HEVC/H.264 stream #3274
Conversation
Hi @kanongil, You were the last person to provide enhancements to fmp4 parsing. Would you mind reviewing these changes? |
@@ -354,10 +366,17 @@ export function getDuration (data: Uint8Array, initData: InitData) { | |||
// get the track id from the tfhd |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The file here advertised with the major brand mp42, not the CMAF brand, and is multi-trun. So we have durations split across multiple track runs on the same track. I see below you've started to account for this, but it'll be more parsing issues like this. It may make sense in the future to pull apart the parser to make a brand-specific parser for CMAF since it is intended to be much simpler.
src/utils/mp4-tools.ts
Outdated
return baseTime / scale; | ||
} | ||
// eslint-disable-next-line no-console | ||
console.warn(`getStartDTS: No fmp4 track found for tfhd id ${id}`); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This warning is occurring for track-id 2 on the test content because we do not handle the clcp
style of hdlr
. It's a closed caption track.
hls.js/src/demux/mp4demuxer.js
Lines 273 to 287 in c19056b
if (hdlr) { | |
const hdlrType = MP4Demuxer.bin2str(hdlr.data.subarray(hdlr.start + 8, hdlr.start + 12)); | |
let type = { 'soun': 'audio', 'vide': 'video' }[hdlrType]; | |
if (type) { | |
// extract codec info. TODO : parse codec details to be able to build MIME type | |
let codecBox = MP4Demuxer.findBox(trak, ['mdia', 'minf', 'stbl', 'stsd']); | |
if (codecBox.length) { | |
codecBox = codecBox[0]; | |
let codecType = MP4Demuxer.bin2str(codecBox.data.subarray(codecBox.start + 12, codecBox.start + 16)); | |
logger.log(`MP4Demuxer:${type}:${codecType} found`); | |
} | |
result[trackId] = { timescale: timescale, type: type }; | |
result[type] = { timescale: timescale, id: trackId }; | |
} | |
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got rid of that. Now fragment boxes for those tracks are ignored. Thanks so much for pointing this out!
a0fb581
to
2ad2c97
Compare
2ad2c97
to
daece4c
Compare
daece4c
to
c426997
Compare
This PR will...
Add support for AVC fmp4 fragments found in Apple Advanced HEVC/H.264 stream
Why is this Pull Request needed?
Playback of https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_adv_example_hevc/master.m3u8 without JavaScript exceptions.
When playing this stream, mp4-tool functions such as
getStartDTS
,offsetStartDTS
, andgetDuration
would throw exceptions when encountering track fragment boxes for unknown track types* not accounted for in the init data (initData[id]
isundefined
).These changes ignore those track fragments so that we only deal with the audio and video fragments we're parsing time/duration for.
*Track types added to init data:
Are there any points in the code the reviewer needs to double check?
Replaced some array methods where appropriate (
map
>reduce
orforEach
).This is also broken in v0.x, so a patch in master should be made for v0.15.
Demo:
https://deploy-preview-3274--hls-js-dev.netlify.app/demo/?src=https%3A%2F%2Fdevstreaming-cdn.apple.com%2Fvideos%2Fstreaming%2Fexamples%2Fbipbop_adv_example_hevc%2Fmaster.m3u8&demoConfig=eyJlbmFibGVTdHJlYW1pbmciOnRydWUsImF1dG9SZWNvdmVyRXJyb3IiOnRydWUsInN0b3BPblN0YWxsIjpmYWxzZSwiZHVtcGZNUDQiOmZhbHNlLCJsZXZlbENhcHBpbmciOi0xLCJsaW1pdE1ldHJpY3MiOi0xfQ==
Checklist