Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

no sound on video output #299

Closed
sugizo opened this issue Feb 20, 2024 · 18 comments
Closed

no sound on video output #299

sugizo opened this issue Feb 20, 2024 · 18 comments

Comments

@sugizo
Copy link

sugizo commented Feb 20, 2024

environment
google colab

code

!wget -c http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
pip install -U typed-ffmpeg
import ffmpeg
import IPython
(
    ffmpeg
    .input(filename = 'BigBuckBunny.mp4')
    .trim(duration = 3)
    .output(filename = 'trim_duration.mp4')
    .run()
)
IPython.display.Video('trim_duration.mp4', embed = True)

result
no sound on video output

expected result
video output have sound

best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 21, 2024

@sugizo
Thanks your report

In the context of FFmpeg's complex filters, where audio and video streams are processed separately, if you only apply a video filter (like trimming the video) without addressing the audio stream, the audio may indeed be dropped or not processed as intended.

Hence, they must be handled separately before being combined at the end.

in_file = ffmpeg.input("http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4")

out_file = ffmpeg.output(  # combined two stream together
  in_file.trim(duration=3),  # trim the video part
  in_file.atrim(duration=3), # trim the audio part
  filename="output.mp4"
)
out_file.run()

However, when it comes to trimming, using a complex filter is not necessary.

# Direct trimming without complex filters
ffmpeg.input(
    "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"
).output(
    "out.mp4", t=3  # Apply trimming to both video and audio
).run()

@sugizo
Copy link
Author

sugizo commented Feb 21, 2024

trying with overlay but no luck

!wget -c http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
!wget -c https://cdn.creatomate.com/demo/logo.png
video = ffmpeg.input('output_t.mp4')
image = ffmpeg.input('logo.png')

ffmpeg.output(video, 
              image, 
              filename = "overlay.mp4", 
              filter_complex = 'overlay=25:25:enable="between(t,0,20)"', 
              **{'y': True}
).run()

IPython.display.Video('overlay.mp4', embed = True)

result

FFMpegExecuteError                        Traceback (most recent call last)
[<ipython-input-14-339dde2bf51a>](https://localhost:8080/#) in <cell line: 4>()
      7               filter_complex = 'overlay=25:25:enable="between(t,0,20)"',
      8               **{'y': True}
----> 9 ).run()
     10 
     11 IPython.display.Video('overlay.mp4', embed = True)

[/usr/local/lib/python3.10/dist-packages/ffmpeg/dag/nodes.py](https://localhost:8080/#) in run(self, cmd, capture_stdout, capture_stderr, input, quiet, overwrite_output, auto_fix)
    580 
    581         if retcode:
--> 582             raise FFMpegExecuteError(
    583                 retcode=retcode,
    584                 cmd=self.compile_line(cmd, overwrite_output=overwrite_output, auto_fix=auto_fix),

FFMpegExecuteError: ffmpeg -i output_t.mp4 -i logo.png -map 0 -map 1 -filter_complex 'overlay=25:25:enable="between(t,0,20)"' -y overlay.mp4 error (see stderr output for detail) None

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 21, 2024

ffmpeg.output(video,
image,
filename = "overlay.mp4",
filter_complex = 'overlay=25:25:enable="between(t,0,20)"',
**{'y': True}
).run()

The issue here is that filter_complex is not an output options, so it cannot be used in the output function
Try this:

import ffmpeg

video = ffmpeg.input('http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4')
image = ffmpeg.input('https://cdn.creatomate.com/demo/logo.png')

video.overlay(
    image, x=25, y=25, enable='between(t,0,20)'
).output(filename='overlay.mp4).run()

@sugizo
Copy link
Author

sugizo commented Feb 21, 2024

example code above not work,
trying to make it simple with removing enable parameter also not work
e.g.
overlay(image, x=25, y=25)

even trying to download the source first also not work, e.g.

!wget -c http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4
!wget -c https://cdn.creatomate.com/demo/logo.png
video = ffmpeg.input('BigBuckBunny.mp4')
image = ffmpeg.input('logo.png')

test your code above

video = ffmpeg.input('http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4')
image = ffmpeg.input('https://cdn.creatomate.com/demo/logo.png')
video.overlay(
    image, x=25, y=25, enable='between(t,0,20)'
).output(filename='overlay.mp4').run()

result

FFMpegExecuteError                        Traceback (most recent call last)
[<ipython-input-25-793bba9faa03>](https://localhost:8080/#) in <cell line: 4>()
      4 video.overlay(
      5     image, x=25, y=25, enable='between(t,0,20)'
----> 6 ).output(filename='overlay.mp4').run()

[/usr/local/lib/python3.10/dist-packages/ffmpeg/dag/nodes.py](https://localhost:8080/#) in run(self, cmd, capture_stdout, capture_stderr, input, quiet, overwrite_output, auto_fix)
    580 
    581         if retcode:
--> 582             raise FFMpegExecuteError(
    583                 retcode=retcode,
    584                 cmd=self.compile_line(cmd, overwrite_output=overwrite_output, auto_fix=auto_fix),

FFMpegExecuteError: ffmpeg -i http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4 -i https://cdn.creatomate.com/demo/logo.png -filter_complex '[0][1]overlay=x=25:y=25:enable=between(t\,0\,20)[s0]' -map '[s0]' overlay.mp4 error (see stderr output for detail) None

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

That's strange, as it's functioning properly on my end.

>>> ffmpeg -i http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4 -i https://cdn.creatomate.com/demo/logo.png -filter_complex '[0][1]overlay=x=25:y=25:enable=between(t\,0\,20)[s0]' -map '[s0]' overlay.mp4
ffmpeg version 6.1.1 Copyright (c) 2000-2023 the FFmpeg developers
  built with Apple clang version 15.0.0 (clang-1500.1.0.2.5)
  configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/6.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
  libavutil      58. 29.100 / 58. 29.100
  libavcodec     60. 31.102 / 60. 31.102
  libavformat    60. 16.100 / 60. 16.100
  libavdevice    60.  3.100 / 60.  3.100
  libavfilter     9. 12.100 /  9. 12.100
  libswscale      7.  5.100 /  7.  5.100
  libswresample   4. 12.100 /  4. 12.100
  libpostproc    57.  3.100 / 57.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
    creation_time   : 2010-01-10T08:29:06.000000Z
  Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
  Stream #0:0[0x1](und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      creation_time   : 2010-01-10T08:29:06.000000Z
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn (default)
    Metadata:
      creation_time   : 2010-01-10T08:29:06.000000Z
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
Input #1, png_pipe, from 'https://cdn.creatomate.com/demo/logo.png':
  Duration: N/A, bitrate: N/A
  Stream #1:0: Video: png, rgba(pc, gbr/unknown/unknown), 390x100 [SAR 11811:11811 DAR 39:10], 25 fps, 25 tbr, 25 tbn
Stream mapping:
  Stream #0:1 (h264) -> overlay
  Stream #1:0 (png) -> overlay
  overlay:default -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
...

Could you please share the FFmpeg stderr message as well?

Screenshot 2024-02-22 at 8 27 07 AM

@sugizo
Copy link
Author

sugizo commented Feb 22, 2024

how to get FFmpeg stderr message
from typed-ffmpeg ?

already tried run it from shell in google colab,
it can run well,
but the output video have no sound
!ffmpeg -y -t 9 -i http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4 -i https://cdn.creatomate.com/demo/logo.png -filter_complex '[0][1]overlay=x=25:y=25:enable=between(t\,0\,20)[s0]' -map '[s0]' overlay.mp4

a little modified code to put parameter
-y (replace output video)
and
-t 9 for faster execution time

here's the log from shell execution above (overlay video have no sound)

ffmpeg version 4.4.2-0ubuntu0.22.04.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.2.0-19ubuntu1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.22.04.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
    creation_time   : 2010-01-10T08:29:06.000000Z
  Duration: 00:09:56.47, start: 0.000000, bitrate: 2119 kb/s
  Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 125 kb/s (default)
    Metadata:
      creation_time   : 2010-01-10T08:29:06.000000Z
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
  Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1991 kb/s, 24 fps, 24 tbr, 24k tbn, 48 tbc (default)
    Metadata:
      creation_time   : 2010-01-10T08:29:06.000000Z
      handler_name    : (C) 2007 Google Inc. v08.13.2007.
      vendor_id       : [0][0][0][0]
Input #1, png_pipe, from 'https://cdn.creatomate.com/demo/logo.png':
  Duration: N/A, bitrate: N/A
  Stream #1:0: Video: png, rgba(pc), 390x100 [SAR 11811:11811 DAR 39:10], 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:1 (h264) -> overlay:main
  Stream #1:0 (png) -> overlay:overlay
  overlay -> Stream #0:0 (libx264)
Press [q] to stop, [?] for help
[libx264 @ 0x56c037743b00] using SAR=1/1
[libx264 @ 0x56c037743b00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0x56c037743b00] profile High, level 3.1, 4:2:0, 8-bit
[libx264 @ 0x56c037743b00] 264 - core 163 r3060 5db6aa6 - H.264/MPEG-4 AVC codec - Copyleft 2003-2021 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=24 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'overlay.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isomavc1mp42
    encoder         : Lavf58.76.100
  Stream #0:0: Video: h264 (avc1 / 0x31637661), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 24 fps, 12288 tbn (default)
    Metadata:
      encoder         : Lavc58.134.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
frame=  216 fps= 18 q=-1.0 Lsize=    1564kB time=00:00:08.87 bitrate=1443.9kbits/s speed=0.748x    
video:1561kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.213018%
[libx264 @ 0x56c037743b00] frame I:1     Avg QP:12.64  size:  3677
[libx264 @ 0x56c037743b00] frame P:64    Avg QP:19.05  size: 18901
[libx264 @ 0x56c037743b00] frame B:151   Avg QP:20.85  size:  2546
[libx264 @ 0x56c037743b00] consecutive B-frames:  3.7%  6.5%  8.3% 81.5%
[libx264 @ 0x56c037743b00] mb I  I16..4:  2.4% 95.4%  2.2%
[libx264 @ 0x56c037743b00] mb P  I16..4:  3.7% 22.6%  0.3%  P16..4: 30.9%  8.7%  7.7%  0.0%  0.0%    skip:26.0%
[libx264 @ 0x56c037743b00] mb B  I16..4:  3.8%  2.7%  0.0%  B16..8: 29.6%  1.8%  0.2%  direct: 3.7%  skip:58.2%  L0:53.2% L1:41.4% BI: 5.4%
[libx264 @ 0x56c037743b00] 8x8 transform intra:70.1% inter:72.8%
[libx264 @ 0x56c037743b00] coded y,uvDC,uvAC intra: 9.4% 20.6% 3.2% inter: 7.1% 17.1% 2.2%
[libx264 @ 0x56c037743b00] i16 v,h,dc,p: 62% 23% 12%  4%
[libx264 @ 0x56c037743b00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 40% 17% 40%  0%  0%  0%  0%  0%  0%
[libx264 @ 0x56c037743b00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 24% 16% 20%  6%  7%  7%  7%  7%  7%
[libx264 @ 0x56c037743b00] i8c dc,h,v,p: 65% 20% 14%  2%
[libx264 @ 0x56c037743b00] Weighted P-Frames: Y:26.6% UV:17.2%
[libx264 @ 0x56c037743b00] ref P L0: 76.2%  7.1% 13.5%  3.3%  0.0%
[libx264 @ 0x56c037743b00] ref B L0: 94.0%  5.4%  0.6%
[libx264 @ 0x56c037743b00] ref B L1: 98.0%  2.0%
[libx264 @ 0x56c037743b00] kb/s:1420.24

objective
have typed-ffmpeg video output with sound

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

  1. No sound issues: Remember to include the audio stream in the output when using complex filters.
  2. Typed-ffmpeg triggers FFMpegExecuteError because FFmpeg reports the output file already exists. To resolve this, add overwrite_output=True to overwrite the existing file. I'll work on making this error easier to diagnose.

Please review the updated script.

import ffmpeg

video = ffmpeg.input('http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4')
image = ffmpeg.input('https://cdn.creatomate.com/demo/logo.png')

video.overlay(
    image, x=25, y=25, enable='between(t,0,20)'
).output(video.audio, filename='overlay.mp4', t=10).run(overwrite_output=True)

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

how to get FFmpeg stderr message from typed-ffmpeg ?

The problem is that when ffmpeg is executed from Python, Google Colab does not display the stderr messages generated by ffmpeg.

@sugizo
Copy link
Author

sugizo commented Feb 22, 2024

overlay sound work well with latest code above, thanks

objective
combine mp4 muted video (video with no sound)
with audio file (
.mp3)

this command work well on shell, video have sound
!ffmpeg -y -i https://cdn.creatomate.com/demo/mountains.mp4 -i http://www.jsayles.com/music/sheep.mp3 -c:v copy -map 0:v:0 -map 1:a:0 -shortest ffmpeg_add_audio.mp4

but no sound output in typed-ffmpeg

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

video_file.output(audio_file, filename = "add_audio.mp4", 
                  **{'codec:v': 'libx264', # libx265 (smallest), libx264 (biggest), copy (faster because not reencoding) 
                     'codec:a': 'aac', # aac (smallest), libmp3lame, ac3 (no sound), libtwolame (biggest), copy (faster because not reencoding)
                  'map': '0:v:0', 'map': '1:a:0', 'shortest': True, }, ).run(overwrite_output = True)

IPython.display.Video('add_audio.mp4', embed = True)

and

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

ffmpeg.output(video_file, audio_file, filename = "add_audio.mp4", 
                  **{'codec:v': 'libx264', # libx265 (smallest), libx264 (biggest), copy (faster because not reencoding) 
                     'codec:a': 'aac', # aac (smallest), libmp3lame, ac3 (no sound), libtwolame (biggest), copy (faster because not reencoding)
                  'map': '0:v:0', 'map': '1:a:0', 'shortest': True, }, ).run(overwrite_output = True)

IPython.display.Video('add_audio.mp4', embed = True)

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

  1. When using typed-ffmpeg, the -map option is automatically included, so it should not be manually added in the output function. This will be addressed in the upcoming major release.
  2. To specifically choose the video or audio stream from the source file, utilize video_file.video and audio_file.audio.
import ffmpeg

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

f = video_file.video.output(audio_file.audio, filename = "add_audio.mp4", 
                  **{'codec:v': 'libx264', # libx265 (smallest), libx264 (biggest), copy (faster because not reencoding) 
                     'codec:a': 'aac', # aac (smallest), libmp3lame, ac3 (no sound), libtwolame (biggest), copy (faster because not reencoding)
                  'shortest': True, }, )

@sugizo
Copy link
Author

sugizo commented Feb 22, 2024

objective
concat or merge 2 videos with audio
into 1 video with sound

code for created 2 sources videos with audio, no problem with this code

in_file = ffmpeg.input('http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4')
ffmpeg.output(in_file.trim(start = 0, end = 3),
              in_file.atrim(start = 0, end = 3),
              filename = "1st_3s.mp4", ).overwrite_output().run()
ffmpeg.output(in_file.trim(start = 5, end = 8),
              in_file.atrim(start = 5, end = 8),
              filename = "2nd_3s.mp4", ).overwrite_output().run()

this one return an error

video_0 = ffmpeg.input('1st_3s.mp4')
video_1 = ffmpeg.input('2nd_3s.mp4')

(
    ffmpeg
    .concat(
        video_0,
        video_1,
    )
    .output(filename = "concat.mp4")
).overwrite_output().run()

result

AttributeError                            Traceback (most recent call last)
[<ipython-input-12-6f45c61e9a06>](https://localhost:8080/#) in <cell line: 5>()
      4 (
      5     ffmpeg
----> 6     .concat(
      7         video_0,
      8         video_1,

AttributeError: module 'ffmpeg' has no attribute 'concat'

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

import ffmpeg

video_0 = ffmpeg.input('1st_3s.mp4')
video_1 = ffmpeg.input('2nd_3s.mp4')

(
    ffmpeg
    .filters.concat(
        video_0,
        video_1,
    ).video(0)
    .output(filename="concat.mp4")
).overwrite_output().run()
  • The concat filter is a dynamic input filter, which is why it's located in the filters module and can be accessed through ffmpeg.filters.concat.
  • As the concat filter also dynamically outputs, the result is a FilterNode. You need to specify the desired output by using video(0).

@sugizo
Copy link
Author

sugizo commented Feb 22, 2024

trying code above,
got result video have no sound (no error occured but result not expected)

trying to learn from source :
typed-ffmpeg/src/ffmpeg/filters.py
not sure how to pass argument for audio

objective
concat or merge 2 videos with audio
into 1 video with sound

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

Typed-ffmpeg serves primarily as a builder for FFmpeg CLI commands, meaning familiarity with FFmpeg filters is still required.

For understanding the concat filter specifically, consider reviewing this Stack Overflow article: Understanding FFmpeg Concat.

Typically, the concat demuxer is preferred for concatenation tasks as it tends to be more straightforward and faster.

For concat filtering, an FFmpeg command might appear as follows:

ffmpeg -i opening.mkv -i episode.mkv \
-filter_complex "[0:v] [0:a] [1:v] [1:a] \
concat=n=2:v=1:a=1 [v] [a]" \
-map "[v]" -map "[a]" output.mkv

Using typed-ffmpeg, this translates to:

import ffmpeg

video_0 = ffmpeg.input('opening.mkv')
video_1 = ffmpeg.input('episode.mkv')

concated = ffmpeg.filters.concat(
    video_0.video,
    video_0.audio,
    video_1.video,
    video_1.audio,
    n=2, v=1, a=1
)

ffmpeg.output(
    concated.video(0), 
    concated.audio(0), 
    filename="concat.mp4"
).overwrite_output().run()

@sugizo
Copy link
Author

sugizo commented Feb 22, 2024

concat filter complex work with code above (video have sound), thanks

does typed-ffmpeg support concat demuxer ?
e.g.

f = open('files.txt', "w")
f.write("""
file 'output_t.mp4'
file 'trim_atrim_duration.mp4'
""")
f.close()

run well on google colab shell

# concat demuxer
# Use this method when you want to avoid a re-encode and your format does not support file-level concatenation (most files used by general users do not support file-level concatenation).
!ffmpeg -y -f concat -safe 0 -i files.txt -c copy ffmpeg_concat_demuxer.mp4

trying to translate into typed-ffmpeg, but face an error

(
    ffmpeg
    .input('files.txt')
    .output(filename = 'concat_demuxer.mp4',
            f = 'concat', 
            save = 0, 
            c = 'copy', )
    .overwrite_output()
    .run()
)

result

FFMpegExecuteError                        Traceback (most recent call last)
[<ipython-input-22-0d91cf5f70e3>](https://localhost:8080/#) in <cell line: 2>()
      7             c = 'copy', )
      8     .overwrite_output()
----> 9     .run()
     10 )
     11 

[/usr/local/lib/python3.10/dist-packages/ffmpeg/dag/nodes.py](https://localhost:8080/#) in run(self, cmd, capture_stdout, capture_stderr, input, quiet, overwrite_output, auto_fix)
    580 
    581         if retcode:
--> 582             raise FFMpegExecuteError(
    583                 retcode=retcode,
    584                 cmd=self.compile_line(cmd, overwrite_output=overwrite_output, auto_fix=auto_fix),

FFMpegExecuteError: ffmpeg -y -i files.txt -map 0 -f concat -save 0 -c copy concat_demuxer.mp4 error (see stderr output for detail) None

thanks and best regards

@lucemia
Copy link
Contributor

lucemia commented Feb 22, 2024

Typed-ffmpeg supports the concat demuxer, but it's crucial to correctly specify input and output options.

In FFmpeg CLI, the position of options matters:

ffmpeg -y -f concat -safe 0 -i files.txt -c copy ffmpeg_concat_demuxer.mp4
             ^^^^^^^^^^^^^^^^  input     ^^^^^^ output

Options before -i are considered as input or global options (depending on the specific option), whereas options after -i are treated as output options.

  • -y: global option
  • -f concat, -safe 0: input options for files.txt
  • -c copy: output option for ffmpeg_concat_demuxer.mp4

In typed-ffmpeg, this translates to:

import ffmpeg
(
    ffmpeg
    .input('files.txt', f='concat', safe=0)
    .output('concat_demuxer.mp4', c='copy')
    .overwrite_output()
    .run()
)

@lucemia
Copy link
Contributor

lucemia commented Feb 23, 2024

@sugizo

I'll be closing this issue for now. If you have any questions or further concerns, please feel free to use the discussion section or reopen the issue.

@lucemia lucemia closed this as completed Feb 23, 2024
@sugizo
Copy link
Author

sugizo commented Mar 4, 2024

import ffmpeg

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

f = video_file.video.output(audio_file.audio, filename = "add_audio.mp4", 
                  **{'codec:v': 'libx264', # libx265 (smallest), libx264 (biggest), copy (faster because not reencoding) 
                     'codec:a': 'aac', # aac (smallest), libmp3lame, ac3 (no sound), libtwolame (biggest), copy (faster because not reencoding)
                  'shortest': True, }, )

recently test,
but not work anymore,
2 weeks ago it's work
did recent update make this code broken ?

code

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

f = video_file.video.output(audio_file.audio, filename = "add_audio.mp4", 
                  **{'codec:v': 'libx264', # libx265 (smallest), libx264 (biggest), copy (faster because not reencoding) 
                     'codec:a': 'aac', # aac (smallest), libmp3lame, ac3 (no sound), libtwolame (biggest), copy (faster because not reencoding)
                  'shortest': True, }, )

f.overwrite_output().run()

and

video_file = ffmpeg.input('https://cdn.creatomate.com/demo/mountains.mp4')
audio_file = ffmpeg.input('http://www.jsayles.com/music/sheep.mp3')

video_file.video.output(audio_file.audio, filename = "add_audio.mp4", vcodec = 'libx264', acodec = 'aac', 
                        **{'shortest': True, }, ).overwrite_output().run()

result
no sound on video output

expected result
have sound on video output

best regards

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants