mp4box failed with PCM audio

[slydiman]

I tried to use mp4box to remux MP4 files recorded by cameras SONY.
Most SONY cameras records PCM_S16BE 48000Hz, 2 channels with the audio tag twos.
SONY FX3 records PCM_S24BE 48000Hz, 2 channels with the audio tag ipcm.
I also saw some files with the audio tag lpcm.

mp4box from GPAC v2.0 worked with PCM 16-bit audio:
mp4box -add video.h264 -add audio.wav -new mux.mp4
produces a correct and playable MP4 file (audio.wav is PCM_S16LE, 48000Hz, 2 channels).

mp4box from GPAC v2.2+ (including the latest) creates a MP4 file with the correct audio handler twos but the audio data is missing at all. stco, stsz, etc. are empty for the audio track.
Note most players are used the audio stream for sync with the timeline, so the resulting file is not playable. It is possible only to seek and see a static picture.

Note mp4box crashes on PCM 24-bit audio files.

[jeanlf]

could you upoad your test files ?

[jeanlf]

we indeed has an issue with the wav parser, thanks for the report. I cannot reproduce the MP4Box crash with 24 bit PCM, could you detail further (command line and source file) ?

[slydiman]

I tried different versions of mp4box and a lot of samples. Unfortunately I cannot find now the exact version and the sample which caused a crash.

Note mp4box up to v2.2 imports PCM_S24LE as PCM_S32BE with the tag in32 w/o subtags. The audio stream is totally damaged.
The expected tag is in24, lpcm or ipcm.
The tag in24 and in32 must contain subtags wave and enda with the BE/LE info.
The tag ipcm must contain pcmC with the BE/LE info.
The tag lpcm must contain AtomSoundDesc v2 with flags like kAudioFormatFlagIsBigEndian.

mp4box v2.2.1 does not import 24-bit PCM audio with the error message

[PCMReframe] Invalid or unsupported WAVE header, aborting
Filters not connected:
mp4mx (importer:file=000001951DF34AB0) (idx=1)
[Importer] Error importing audio.wav
Failure while importing media: BitStream Not Compliant
Error importing audio.wav: BitStream Not Compliant

Hope you can fix 24-bit PCM import. Thanks.

[jeanlf]

Now fixed on master, thanks for the report.

[slydiman]

Thanks.
I have tested it. The importing audio streams from MOV works just fine.
But 24-bit and 32-bit WAV files still are imported incorrectly (as pcm_s16le).

ffmpeg -i test.mp4 -c:v copy -an video.mp4
ffmpeg -i test.mp4 -c:a pcm_s16le audio16.wav
ffmpeg -i test.mp4 -c:a pcm_s24le audio24.wav
ffmpeg -i test.mp4 -c:a pcm_s32le audio32.wav
mp4box -add video.mp4 -add audio16.wav -new ok16.mp4
mp4box -add video.mp4 -add audio24.wav -new bad24.mp4
mp4box -add video.mp4 -add audio32.wav -new bad32.mp4

Note .wav files do not support BE format.

[jeanlf]

please upload test files

[jeanlf]

24 and 32 bit to mp4 works fine here (playback tested in gpac and QT on OSX). If you test with ffplay or other players, it's possible that the player does not properly support ipcm and fpcm (my ffplay build always report pcm_s16le).

24 and 32 bit to mov, either audio entry v1 (in24+enda ...) or v2 (lpcm), works fine as well

[slydiman]

mp4box imports MOV just fine, but cannot import 24-bit or 32-bit WAV.
Note I did not try ADPCM, uLaw and float/double PCM WAV files yet.
Here is the complete test

ffmpeg -i test.mp4 -c:v copy -an video.mp4

ffmpeg -i test.mp4 -c:a pcm_s16le audio16le.wav
mp4box -add video.mp4 -add audio16le.wav -new ok_wav16le.mp4
mp4box -add video.mp4 -add audio16le.wav -new ok_wav16le.mov
ffmpeg -i audio16le.wav -c copy audio16le.mov
ffmpeg -i audio16le.wav -c pcm_s16be audio16be.mov
mp4box -add video.mp4 -add audio16le.mov -new ok_mov16le.mp4
mp4box -add video.mp4 -add audio16be.mov -new ok_mov16be.mp4
mp4box -add video.mp4 -add audio16le.mov -new ok_mov16le.mov
mp4box -add video.mp4 -add audio16be.mov -new ok_mov16be.mov

ffmpeg -i test.mp4 -c:a pcm_s24le audio24le.wav
mp4box -add video.mp4 -add audio24le.wav -new bad_wav24le.mp4
mp4box -add video.mp4 -add audio24le.wav -new bad_wav24le.mov
ffmpeg -i audio24le.wav -c copy audio24le.mov
ffmpeg -i audio24le.wav -c pcm_s24be audio24be.mov
mp4box -add video.mp4 -add audio24le.mov -new ok_mov24le.mp4
mp4box -add video.mp4 -add audio24be.mov -new ok_mov24be.mp4
mp4box -add video.mp4 -add audio24le.mov -new ok_mov24le.mov
mp4box -add video.mp4 -add audio24be.mov -new ok_mov24be.mov

ffmpeg -i test.mp4 -c:a pcm_s32le audio32le.wav
mp4box -add video.mp4 -add audio32le.wav -new bad_wav32le.mp4
mp4box -add video.mp4 -add audio32le.wav -new bad_wav32le.mov
ffmpeg -i audio32le.wav -c copy audio32le.mov
ffmpeg -i audio32le.wav -c pcm_s32be audio32be.mov
mp4box -add video.mp4 -add audio32le.mov -new ok_mov32le.mp4
mp4box -add video.mp4 -add audio32be.mov -new ok_mov32be.mp4
mp4box -add video.mp4 -add audio32le.mov -new ok_mov32le.mov
mp4box -add video.mp4 -add audio32be.mov -new ok_mov32be.mov

Samples:
http://slydiman.me/ffmpeg/pcm/test.mp4
http://slydiman.me/ffmpeg/pcm/video.mp4

http://slydiman.me/ffmpeg/pcm/audio16le.wav
http://slydiman.me/ffmpeg/pcm/audio16le.mov # ffmpeg produces sowt
http://slydiman.me/ffmpeg/pcm/audio16be.mov # ffmpeg produces twos
http://slydiman.me/ffmpeg/pcm/ok_mov16le.mp4 # sowt - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov16be.mp4 # twos - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov16le.mov # sowt - correct
http://slydiman.me/ffmpeg/pcm/ok_mov16be.mov # twos - correct
http://slydiman.me/ffmpeg/pcm/ok_wav16le.mp4 # ipcm - correct
http://slydiman.me/ffmpeg/pcm/ok_wav16le.mov # ipcm - WARNING - expected sowt

http://slydiman.me/ffmpeg/pcm/audio24le.wav
http://slydiman.me/ffmpeg/pcm/audio24le.mov # ffmpeg produces in24
http://slydiman.me/ffmpeg/pcm/audio24be.mov # ffmpeg produces in24
http://slydiman.me/ffmpeg/pcm/ok_mov24le.mp4 # in24 - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov24be.mp4 # in24 - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov24le.mov # in24 - correct
http://slydiman.me/ffmpeg/pcm/ok_mov24be.mov # in24 - correct
http://slydiman.me/ffmpeg/pcm/bad_wav24le.mp4 # ipcm pcm_s16le - WRONG! - expected ipcm pcm_s24le
http://slydiman.me/ffmpeg/pcm/bad_wav24le.mov # ipcm pcm_s16le - WRONG! - expected in24 pcm_s24le

http://slydiman.me/ffmpeg/pcm/audio32le.wav
http://slydiman.me/ffmpeg/pcm/audio32le.mov # ffmpeg produces in32
http://slydiman.me/ffmpeg/pcm/audio32be.mov # ffmpeg produces in32
http://slydiman.me/ffmpeg/pcm/ok_mov32le.mp4 # in32 - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov32be.mp4 # in32 - WARNING - expected ipcm
http://slydiman.me/ffmpeg/pcm/ok_mov32le.mov # in32 - correct
http://slydiman.me/ffmpeg/pcm/ok_mov32be.mov # in32 - correct
http://slydiman.me/ffmpeg/pcm/bad_wav32le.mp4 # ipcm pcm_s16le - WRONG! - expected ipcm pcm_s32le
http://slydiman.me/ffmpeg/pcm/bad_wav32le.mov # ipcm pcm_s16le - WRONG! - expected in32 pcm_s32le

bad_wav24le.mp4 and bad_wav32le.mp4 can be played correctly only in VLC player (that doesn't mean VLC is good).
bad_wav24le.mov and bad_wav32le.mov stutter in VLC player.

Try to playback http://slydiman.me/ffmpeg/pcm/bad_wav24le.mp4 or http://slydiman.me/ffmpeg/pcm/bad_wav32le.mp4
online in the latest Google Chrome (I recommend lowering the volume for safety).

MediaPlayerClassic from k-lite codec pack and any Windows player (except VLC) cannot playback these files correctly too.

Note 1: mp4box -info file reports correct 24-bit or 32-bit LE for bad_wav24le.mp4, bad_wav24le.mov, bad_wav32le.mp4, bad_wav32le.mov. But ipcm SoundSampleDescription contains 16-bit.

Note 2: bad_wav24le.mp4 and bad_wav32le.mp4 contain ipcm Sound Sample Description v0.
bad_wav24le.mov and bad_wav32le.mov contain ipcm Sound Sample Description v1.

Note 3: ipcm is pretty new and many players do not support it yet. I recommend to use lpcm instead for more compatibility or sowt, twos, in24, in32 even for mp4 files.

[jeanlf]

Thanks for your thorough tests !

We had an issue keeping the QT brand while remuxing to isom, which made ipcm in mp4 failed in QT.

All ipcm and fpcm files are working on QT - the reports you see saying ipcm pcm_s16lecome from the tool used, check using MP4Box -info or gpac -i SRC inspect instead

Note 1: mp4box -info file reports correct 24-bit or 32-bit LE for bad_wav24le.mp4, bad_wav24le.mov, bad_wav32le.mp4, bad_wav32le.mov. But ipcm SoundSampleDescription contains 16-bit.

Indeed now fixed (also for qtv1) - note that a player should not trust this info for isobmff, as the ipcm/fpcm sample entry gives "definitive info" about the sample size.

Note 2: bad_wav24le.mp4 and bad_wav32le.mp4 contain ipcm Sound Sample Description v0.
bad_wav24le.mov and bad_wav32le.mov contain ipcm Sound Sample Description v1.

yes, we default to v0 in mp4 as much as possible, however for QT we move to v1 most of the time, as v0 only supports 'raw ' and 'twos' and no sample entry child boxes, according to QTFF.

Note 3: ipcm is pretty new and many players do not support it yet. I recommend to use lpcm instead for more compatibility or sowt, twos, in24, in32 even for mp4 files.

Do that at your own risks (I would not recommend it):

• 'lpcm' uses v2 for sample entry which is undefined in isobmff
• the other QT codes use sample entry v1 which is not binary compatible with isobmff sample entry v1 ...

[slydiman]

All ipcm and fpcm files are working on QT - the reports you see saying ipcm pcm_s16lecome from the tool used, check using MP4Box -info or gpac -i SRC inspect instead

Unfortunately I cannot ask Google Chrome to check MP4Box -info for the correct bitness info. It and many other players simply use the base Sound Sample Description.

Indeed now fixed (also for qtv1) - note that a player should not trust this info for isobmff, as the ipcm/fpcm sample entry gives "definitive info" about the sample size.

Thanks. I'm still waiting for nightly builds for tests
https://gpac.wp.imt.fr/downloads/gpac-nightly-builds/
The last build contains 5bd03ed at 5/23/2023.
Can someone trigger a nightly build with latest commits, please?

[aureliendavid]

Thanks. I'm still waiting for nightly builds for tests
https://gpac.wp.imt.fr/downloads/gpac-nightly-builds/
The last build contains 5bd03ed at 5/23/2023.
Can someone trigger a nightly build with latest commits, please?

Hi,

Indeed, some services hadn't restated properly after a power cut this week-end.

I've fixed it, now the system will catch up with the late builds.

I'll update when it's all done.

thanks

[aureliendavid]

nightlies should now be up to date

[slydiman]

Thanks. I have checked it and everything works fine.