Error with ffmpeg config for batch inference

[ayushchopra96]

Hi, @Rudrabha @prajwalkr thanks for the amazing code and congrats on your paper!!

I am trying to run inference but I get the following error message with ffmpeg:

[buffer @ 0x6f2b80] Error setting option pix_fmt to value -1.
[graph 0 input from stream 1:0 @ 0x6f2e20] Error applying options to the filter.
Error opening filters!

If there some config requirement that I am missing?
Looking forward to your response.

[prajwalkr]

Thanks for trying the code out! Could you please copy paste the exact command which you had run?

Also, I am assuming the error occurs in line 223:
command = 'ffmpeg -i {} -i {} -strict -2 {}'.format(args.audio, path.join(args.results_dir, 'result.avi'), path.join(args.results_dir, 'result_voice.mp4'))

[ayushchopra96]

Yes, the error occurs in line 223.

The command I had run:
CUDA_VISIBLE_DEVICES=1 python batch_inference.py --checkpoint_path logs/lipgan_best_residual.h5 --face demo_images/test_sample2.jpg --audio preprocess/temp.wav --results_dir ./results/ --fps 25

[prajwalkr]

What is your FFMPEG version?

[ayushchopra96]

FFmpeg version 2.8.15-0ubuntu0.16.04.1

[prajwalkr]

Please paste the full output log here. I am guessing it is an issue with the audio/video file or a version issue. Hard to say without the full log.

[ayushchopra96]

Error Summary:
Could not find codec parameters for stream 0 (Video: mpeg4 (DIVX / 0x58564944), none, 256x256): unspecified pixel format
[buffer @ 0x1622a00] Unable to parse option value "-1" as pixel format
[buffer @ 0x1622a00] Error setting option pix_fmt to value -1.
[graph 0 input from stream 1:0 @ 0x1622440] Error applying options to the filter.
Error opening filters!

Full Logs

Predictions shape: (169, 96, 96, 3)
100%|###########################################################################################################################################################################################################| 1/1 [00:11<00:00, 11.90s/it]
ffmpeg version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
libavutil 54. 31.100 / 54. 31.100
libavcodec 56. 60.100 / 56. 60.100
libavformat 56. 40.101 / 56. 40.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 40.101 / 5. 40.101
libavresample 2. 1. 0 / 2. 1. 0
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 2.101 / 1. 2.101
libpostproc 53. 3.100 / 53. 3.100
Guessed Channel Layout for Input Stream #0.0 : mono
Input #0, wav, from 'preprocess/temp.wav':
Metadata:
encoder : Lavf56.40.101
Duration: 00:00:07.12, bitrate: 256 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 16000 Hz, 1 channels, s16, 256 kb/s
[avi @ 0x1619460] Could not find codec parameters for stream 0 (Video: mpeg4 (DIVX / 0x58564944), none, 256x256): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
./results/result.avi: could not find codec parameters
Input #1, avi, from './results/result.avi':
Metadata:
encoder : Lavf58.17.101
Duration: N/A, start: 0.000000, bitrate: N/A
Stream #1:0: Video: mpeg4 (DIVX / 0x58564944), none, 256x256, 25 fps, 25 tbr, 25 tbn, 25 tbc
File './results/result_voice.mp4' already exists. Overwrite ? [y/N] y
[buffer @ 0x1622a00] Unable to parse option value "-1" as pixel format
Last message repeated 1 times
[buffer @ 0x1622a00] Error setting option pix_fmt to value -1.
[graph 0 input from stream 1:0 @ 0x1622440] Error applying options to the filter.
Error opening filters!

[prajwalkr]

Please share the generated result.avi here. Also, see if the error gets resolved if you change result.avi in line 223 to result.mp4.

[ayushchopra96]

Please Find Attached
The generated result.avi looks fine though.

result.avi.zip

[prajwalkr]

Does the error get resolved if you change the avi to mp4?

[prajwalkr]

Please Find Attached
The generated result.avi looks fine though.

result.avi.zip

Please also send the wav file so that I can run the ffmpeg command on my system and check

[ayushchopra96]

No, the issue does not get resolved on changing .avi to .mp4

Please find attached the wav file
temp.wav.zip

[prajwalkr]

I was able to merge with FFmpeg 3.4.6 using the exact same command. Merged file attached below. Please upgrade your FFMPEG version.

result_voice.zip

[ayushchopra96]

Thanks!! I'll try it.

[prajwalkr]

Closing due to inactivity.