Packaging failure when there is additional unused mdat box in the source file

[kqyang]

Extracted from #107.

System info

Shaka Packager Version: cc778f6-release

Issue and steps to reproduce the problem

Use sample videos from sample-videos.

root@af3e5b750a97:/media# packager input=SampleVideo_1280x720_30mb.mp4 --dump_stream_info
[1129/100107:INFO:demuxer.cc(88)] Demuxer::Run() on file 'SampleVideo_1280x720_30mb.mp4'.
[1129/100107:INFO:demuxer.cc(160)] Initialize Demuxer for file 'SampleVideo_1280x720_30mb.mp4'.

File "SampleVideo_1280x720_30mb.mp4":
Found 2 stream(s).
Stream [0] type: Video
 codec_string: avc1.4d401f
 time_scale: 12800
 duration: 2186752 (170.8 seconds)
 is_encrypted: false
 codec: H264
 width: 1280
 height: 720
 pixel_aspect_ratio: 1:1
 trick_play_factor: 0
 nalu_length_size: 4

Stream [1] type: Audio
 codec_string: mp4a.40.2
 time_scale: 48000
 duration: 8201216 (170.9 seconds)
 is_encrypted: false
 codec: AAC
 sample_bits: 16
 num_channels: 6
 sampling_frequency: 48000
 language: und

But when try to do something with the video the error occurs:

root@af3e5b750a97:/media# packager input=SampleVideo_1280x720_30mb.mp4,stream=1,output=audio.mp4 input=SampleVideo_1280x720_30mb.mp4,stream=0,output=video.mp4
[1129/100139:INFO:demuxer.cc(88)] Demuxer::Run() on file 'SampleVideo_1280x720_30mb.mp4'.
[1129/100139:INFO:demuxer.cc(160)] Initialize Demuxer for file 'SampleVideo_1280x720_30mb.mp4'.
[1129/100139:ERROR:mp4_media_parser.cc(263)] Not implemented reached in bool shaka::media::mp4::MP4MediaParser::ParseBox(bool *) Files with MDAT before MOOV is not supported yet.
[1129/100139:ERROR:packager_main.cc(461)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file SampleVideo_1280x720_30mb.mp4

[zshenker]

@kqyang I've just run into this in my testing too.

I seem to also be seeing this error when using fragmented MP4 files as inputs, even when there is a moov atom at the start of the file. Will try to dig a bit deeper and see if I can put together a test file.

[kqyang]

I looked into the sample video: SampleVideo_1280x720_30mb.mp4:

[ftyp] size=8+24
  major_brand = isom
  minor_version = 200
  compatible_brand = isom
  compatible_brand = iso2
  compatible_brand = avc1
  compatible_brand = mp41
[free] size=8+0
[mdat] size=8+31393553
[mdat] size=8+0
[moov] size=8+97513

There is an additional unexpected and unused mdat box. That is what causes the problem. It is unusual and not useful but since the spec does not disallow having unused mdat box, we will fix the problem.

@zshenker Let us know if your test failure is caused by the same problem.

[zshenker]

atom_map.txt
@kqyang not sure if its exactly the same issue here, as my file does have a moov atom before any mdats. Attached output from boxdumper.

[kqyang]

@zshenker It does not look like the same problem. Can you attach a sample file or send it to my email?

[zshenker]

@kqyang Just shared a sample file with you via Google Drive.
Getting these log/error messages:

[1130/214844:INFO:demuxer.cc(88)] Demuxer::Run() on file '/media/DAI_Dev_5_1_Test_CBS_NCIS_283_CONTENT_CIAN_vr_20M_5_1_1273370_1500.mp4'.
[1130/214844:INFO:demuxer.cc(160)] Initialize Demuxer for file '/media/DAI_Dev_5_1_Test_CBS_NCIS_283_CONTENT_CIAN_vr_20M_5_1_1273370_1500.mp4'.
[1130/214844:ERROR:box_definitions.cc(623)] Not implemented reached in virtual bool shaka::media::mp4::SampleDescription::ReadWriteInternal(shaka::media::mp4::BoxBuffer *)SampleDecryption type 0 is not handled. Skipping.
[1130/214844:ERROR:box_definitions.cc(2076)] Not implemented reached in virtual bool shaka::media::mp4::MediaInformation::ReadWriteInternal(shaka::media::mp4::BoxBuffer *)
[1130/214844:ERROR:mp4_media_parser.cc(263)] Not implemented reached in bool shaka::media::mp4::MP4MediaParser::ParseBox(bool *) Files with MDAT before MOOV is not supported yet.
[1130/214844:ERROR:packager_main.cc(414)] Packaging Error: 8 (PARSER_FAILURE): Cannot parse media file /media/DAI_Dev_5_1_Test_CBS_NCIS_283_CONTENT_CIAN_vr_20M_5_1_1273370_1500.mp4

[kqyang]

@zshenker Thanks. So the problem with this file is different but related. There is timecode track in this file, which contains one mdat with 4 bytes of data. We don't recongize timecode track thus the associated mdat is not used. It will be addressed in the same fix.