Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Generation of corrupt assets, with conflicting number of subsegments #862

Open
copo2496 opened this issue Nov 19, 2020 · 12 comments
Open

Generation of corrupt assets, with conflicting number of subsegments #862

copo2496 opened this issue Nov 19, 2020 · 12 comments
Assignees
@kqyang
Labels
type: bug Something isn't working correctly
Milestone
Backlog

Comments

@copo2496
Copy link

System info

Operating System: Amazon Linux 2012 4.9 Kernel
Shaka Packager Version: v2.4.3 dd98700

Issue and steps to reproduce the problem

Packager Command: ./packager-linux in=[obfuscated].mp4,stream=audio,output=[obfuscated].mp4,playlist_name=[obfuscated].m3u8 --hls_master_playlist_output [obfuscated]_master.m3u8 --segment_duration 0.9752380952380952 --fragment_duration 0.9752380952380952

What is the expected result?

Generate an FMP4 asset which can be played

What happens instead?

The FMP4 asset is not playable and we get the following warning during packaging:
Number of subsegment ranges (13445) does not match the number of subsegments notified to OnNewSegment() (78981).

Input Files:

The file passed into the packager was proprietary and so we cannot share it, but we can share metadata about the file obtained using ffmpeg and ffprobe:

ffmpeg version 4.0.1 Copyright (c) 2000-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.0.1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '[obfuscated].mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
creation_time : 2020-11-17T20:00:32.000000Z
Duration: 21:23:44.40, start: 0.000000, bitrate: 63 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 62 kb/s (default)
Metadata:
creation_time : 2020-11-17T20:00:25.000000Z
handler_name : aac@GPAC0.7.0-revrelease
Stream mapping:
Stream #0:0 -> #0:0 (aac (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
encoder : Lavf58.12.100
Stream #0:0(und): Audio: pcm_s16le, 22050 Hz, stereo, s16, 705 kb/s (default)
Metadata:
creation_time : 2020-11-17T20:00:25.000000Z
handler_name : aac@GPAC0.7.0-revrelease
encoder : Lavc58.18.100 pcm_s16le
size=N/A time=21:23:44.39 bitrate=N/A speed=1.88e+03x
video:0kB audio:6634328kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown

ffprobe version 4.0.1 Copyright (c) 2007-2018 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.39.2)
configuration: --prefix=/usr/local/Cellar/ffmpeg/4.0.1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '[obfuscated].mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
creation_time : 2020-11-17T20:00:32.000000Z
Duration: 21:23:44.40, start: 0.000000, bitrate: 63 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 22050 Hz, stereo, fltp, 62 kb/s (default)
Metadata:
creation_time : 2020-11-17T20:00:25.000000Z
handler_name : aac@GPAC0.7.0-revrelease
@ronak2121
Copy link

Hi, this issue is blocking our release at the moment, and happens with many input audio files on our end. Do you have any ideas that could help us figure out what could be happening?

@kqyang
Copy link
Collaborator

kqyang commented Nov 24, 2020

@ronak2121 @copo2496 It is hard to debug with just metadata. Are you able to generate a sample input that can re-produce the problem and send it to us at shaka-packager-issues@google.com?

@copo2496
Copy link
Author

copo2496 commented Dec 2, 2020 •
edited

@kqyang we were able to reproduce the issue using a wave audio file, we've uploaded the file and made it publicly accessible here: https://audible-audio-production-samples.s3.amazonaws.com/shaka/shaka_test.mp4.

Here was the output on OSX:

in=shaka_test.mp4,stream=audio,output=testOut4.mp4,playlist_name=test4.m3u8 --hls_master_playlist_output test4_master.m3u8 --segment_duration 0.9752380952380952 --fragment_duration 0.9752380952380952
[1202/110101:INFO:demuxer.cc(88)] Demuxer::Run() on file 'shaka_test.mp4'.
[1202/110101:INFO:demuxer.cc(160)] Initialize Demuxer for file 'shaka_test.mp4'.
[1202/110113:INFO:single_segment_segmenter.cc(107)] Update media header (moov) and rewrite the file to 'testOut4.mp4'.
[1202/110117:warning:hls_notify_muxer_listener.cc(201)] Number of subsegment ranges (910) does not match the number of subsegments notified to OnNewSegment() (66446).
[1202/110117:INFO:mp4_muxer.cc(171)] MP4 file 'testOut4.mp4' finalized.
Packaging completed successfully.

The file was not playable, and we see the "Number of subsegment ranges does not match the number of subsegments notified to" warning in the output.

@ronak2121
Copy link

To note a fragment size of 9.752380952380952 seems to work, so it seems like there's some problem with math or something that's causing this.

@ronak2121
Copy link

FYI that we found this is not caused by any incorrect math; but because of the limitation in the sidx atom that the reference_count be a short. This is causing you to drop all of the extra segments here.

@ronak2121
Copy link

Question for you @kqyang when I specify to use the HLS LIVE manifest type, you still generate a sidx atom. Also, I see you have options to disable sidx generation for the DASH LIVE profile, but I don't see how to specify it.

@kqyang
Copy link
Collaborator

kqyang commented Dec 4, 2020

@ronak2121 The flag is --nogenerate_sidx_in_media_segments . See https://google.github.io/shaka-packager/html/documentation.html#mp4-output-options.

@ronak2121
Copy link

ronak2121 commented Dec 4, 2020 •
edited

Hey @kqyang I figured that out and have observed a few things:

  1. That option can't be supplied for HLS. You always generate sidx for HLS (even though HLS doesn't even use sidx).

in a command like so:

packager-osx in=input.mp4,stream=audio,output=testOut4.mp4,playlist_name=test4.m3u8 --hls_master_playlist_output test4_master.m3u8 --segment_duration 0.9752380952380952 --fragment_duration 0.9752380952380952 --nogenerate_sidx_in_media_segments

  1. Even when we specify HLS Live profile you are still generating a sidx.

How hard would it be for you to ignore sidx generation for HLS/FairPlay? Sidx isn't used by HLS/FairPlay, even in CMAF mode.

We're getting ready to email you on the email alias above about this, but curious if you could help us here.

@kqyang
Copy link
Collaborator

kqyang commented Dec 4, 2020

What is the error you are seeing? It should work for both DASH or HLS.

It could be caused by a different problem. There may be a rounding error when trying to generate the segments.

Can you try setting --fragment_duration to a value slightly larger than --segment_duration, e.g. set it to 1? It will force packager to generate only one fragment in every segment.

@ronak2121
Copy link

ronak2121 commented Dec 4, 2020 •
edited

The error we see is that you create only 991 segments when there should be more than 65k. I just tried your suggestion:

packager-osx in=input.mp4,stream=audio,output=testOut4.mp4,playlist_name=test4.m3u8 --hls_master_playlist_output test4_master.m3u8 --segment_duration 0.9752380952380952 --fragment_duration 1 --hls_playlist_type LIVE --nogenerate_sidx_in_media_segments

and

packager-osx in=input.mp4,stream=audio,output=testOut4.mp4,playlist_name=test4.m3u8 --hls_master_playlist_output test4_master.m3u8 --segment_duration 0.9752380952380952 --fragment_duration 1 --nogenerate_sidx_in_media_segments

and neither of them work to remove generation of sidx.

Keep in mind that we want to generate VOD streams, not LIVE here. So if we switch to LIVE, it would be only a workaround for this issue. But with the downside of being very slow to segment our content.

@ronak2121
Copy link

ronak2121 commented Dec 4, 2020 •
edited

Hey @kqyang FYI that we've emailed you guys about this issue privately. Let's continue the conversation there.

@kqyang
Copy link
Collaborator

kqyang commented Dec 7, 2020

@ronak2121 Thanks for the detail information.

You are right that

  • The problem is that the mp4 box 'sidx' can only store at most 65535 references (fragments). When there are more than 65535 fragments as in your stream (18 hours with 1 second fragment), Shaka Packager fails to generate the 'sidx' box.

  • 'sidx' box is required for DASH VOD single-segment stream but not needed for HLS.

  • The flag --nogenerate_sidx_in_media_segments was not working for VOD single-segment mode.

There are two options to workaround / fix the problem:

  • Truncate the number of references to the max number (65535) and log a warning instead of fail packaging when the problem occurs.

  • Allow --nogenerate_sidx_in_media_segments to be used for VOD single-segment mode too.

We will provide a fix soon.

@kqyang kqyang added type: bug Something isn't working correctly and removed needs triage labels Dec 7, 2020
@kqyang kqyang self-assigned this Dec 7, 2020
@kqyang kqyang added this to the v2.5 milestone Dec 7, 2020
shaka-bot pushed a commit that referenced this issue Dec 11, 2020
@kqyang
[MP4] Truncate segment references in 'sidx' if necessary
516430b 
The reference count in 'sidx' box is a uint16 field, which allows at
most 0xFFFF entries, i.e. at most 0xFFFF subsegments, which is roughly
18 hours for one second segments.

Do not fail packaging when it happens. Instead, generate a warning and
truncate the number of references to 0xFFFF instead.

Note that the actual number of mp4 fragments in the mp4 file can still
be more than 0xFFFF. The stream will not play to the end in DASH, but
it will play successfully in HLS.

Workarounds #862.

Change-Id: Ib3930418d1528df1f9ea64cda0d0ebaa78d26abb
shaka-bot pushed a commit that referenced this issue Dec 11, 2020
@kqyang
[MP4] Allow not to generate 'sidx' box for single-segment too
10daa39 
I.e. the flag --generate_sidx_in_media_segments,
--nogenerate_sidx_in_media_segments work for both single-segment
and multi-segment mode with this change.

Related to #862.

Change-Id: Icd27fd00e8e036ba0c4709b48650372429cc0351
@kqyang kqyang modified the milestones: v2.5, v2.6 Jun 11, 2021
@joeyparrish joeyparrish modified the milestones: v2.6, Backlog Jun 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kqyang kqyang

Labels
type: bug Something isn't working correctly
Projects
None yet
Milestone
Backlog
Development

No branches or pull requests
5 participants
@ronak2121 @joeyparrish @kqyang @copo2496 @shaka-bot