Missing validation of audio format when merging the video and audio of two formats into a single file using -f <video-format>+<audio-format>

[Pr0xym3]

Checklist

• I'm reporting a broken site support issue
• I've verified that I'm running youtube-dl version 2021.12.17
• I've checked that all provided URLs are alive and playable in a browser
• I've checked that all URLs and arguments with special characters are properly quoted or escaped
• I've searched the bugtracker for similar bug reports including closed ones
• I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', 'mp4+webm', '-o', '%(title)s-%(id)s-%(format_id)s.%(ext)s', 'https://www.youtube.com/watch?v=j-3nl88O7uI']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2021.12.17
[debug] Python 3.11.4 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 - OpenSSL 1.1.1u  30 May 2023
[debug] exe versions: ffmpeg 2023-07-02-git-50f34172e0-full_build-www.gyan.dev, ffprobe 2023-07-02-git-50f34172e0-full_build-www.gyan.dev
[debug] Proxy map: {}
[youtube] j-3nl88O7uI: Downloading webpage
[debug] [youtube] Decrypted nsig JfTIlrUHP5_7OvR => 617U8OxogCB59Q
[debug] [youtube] Decrypted nsig yEKCnamuQO8mrax => W86NEdaNJUxadg
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on 'https://rr5---sn-u5a3u5a3-cimd.googlevideo.com/videoplayback?expire=1688595425&ei=gJelZJ3gPNmvxN8PwuKOyAc&ip=***.***.***.***&id=o-AK-IZllz7F-GSnCaiFpHeBFL0PkELonhmq2UWNVVf804&itag=22&source=youtube&requiressl=yes&mh=X2&mm=31%2C29&mn=sn-u5a3u5a3-cimd%2Csn-aigl6nzs&ms=au%2Crdu&mv=m&mvi=5&pl=21&initcwndbps=1728750&spc=Ul2Sq91QueiJLC5mV7z_FTJ0tG6X-qjVXD4pd0Fi3w&vprv=1&svpuc=1&mime=video%2Fmp4&ns=WHNf16yMhRrW1ew_skjJKu8O&cnr=14&ratebypass=yes&dur=639.315&lmt=1670014548745278&mt=1688573586&fvip=5&fexp=24007246%2C24363391&beids=24350018&c=WEB&txp=5311224&n=W86NEdaNJUxadg&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Ccnr%2Cratebypass%2Cdur%2Clmt&sig=AOq0QJ8wRQIgVLZcCoAyD7l73FJCyZXtZCaVD9ITUu6ohYRZ6SHOE7ACIQDrcSJrOjq2hBMC2jd_c_PD77CVR19TxzF6R5INdEsANQ%3D%3D&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgeuiBs8m39nQ3F_n376CwtMQoTkaCY5VKoxqB5bhSSwICIFy8utzcOWdggKnjyVqboJuI3PmNEXRLK3O0lVQAYtbj'
[download] Destination: 4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-22.f22.mp4
[download] 100% of 134.44MiB in 00:34
[debug] Invoking downloader on 'https://rr5---sn-u5a3u5a3-cimd.googlevideo.com/videoplayback?expire=1688595425&ei=gJelZJ3gPNmvxN8PwuKOyAc&ip=***.***.***.***&id=o-AK-IZllz7F-GSnCaiFpHeBFL0PkELonhmq2UWNVVf804&itag=313&aitags=133%2C134%2C135%2C136%2C137%2C160%2C242%2C243%2C244%2C247%2C248%2C271%2C278%2C313&source=youtube&requiressl=yes&mh=X2&mm=31%2C29&mn=sn-u5a3u5a3-cimd%2Csn-aigl6nzs&ms=au%2Crdu&mv=m&mvi=5&pl=21&initcwndbps=1728750&spc=Ul2Sq91QueiJLC5mV7z_FTJ0tG6X-qjVXD4pd0Fi3w&vprv=1&svpuc=1&mime=video%2Fwebm&ns=udFRX7ON6ZSVXn01_gO2Y44O&gir=yes&clen=1101478410&dur=639.205&lmt=1670014304605954&mt=1688573586&fvip=5&keepalive=yes&fexp=24007246%2C24363391&beids=24350018&c=WEB&txp=531A224&n=617U8OxogCB59Q&sparams=expire%2Cei%2Cip%2Cid%2Caitags%2Csource%2Crequiressl%2Cspc%2Cvprv%2Csvpuc%2Cmime%2Cns%2Cgir%2Cclen%2Cdur%2Clmt&sig=AOq0QJ8wRAIgQgwzUVC_-MQbh5n7KMkZDmJ2xxFJgmN5b9afGKg1CZgCIDn7rXrP-OOEezJPYvCM8T9jy5_7VVJ7-_wWOyhZt7d1&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AG3C_xAwRAIgeuiBs8m39nQ3F_n376CwtMQoTkaCY5VKoxqB5bhSSwICIFy8utzcOWdggKnjyVqboJuI3PmNEXRLK3O0lVQAYtbj'
[dashsegments] Total fragments: 106
[download] Destination: 4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-313.f313.webm
[download] 100% of 1.03GiB in 04:24
[ffmpeg] Merging formats into "4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-22+313.mkv"
[debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-22.f22.mp4" -i "file:4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-313.f313.webm" -c copy -map "0:v:0" -map "1:a:0" "file:4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-22+313.temp.mkv"
[debug] ffmpeg version 2023-07-02-git-50f34172e0-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
  configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-libharfbuzz --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
  libavutil      58. 13.101 / 58. 13.101
  libavcodec     60. 22.100 / 60. 22.100
  libavformat    60. 10.100 / 60. 10.100
  libavdevice    60.  2.101 / 60.  2.101
  libavfilter     9.  8.102 /  9.  8.102
  libswscale      7.  3.100 /  7.  3.100
  libswresample   4. 11.100 /  4. 11.100
  libpostproc    57.  2.100 / 57.  2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-22.f22.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: isommp42
    creation_time   : 2022-12-02T20:47:28.000000Z
  Duration: 00:10:39.22, start: 0.000000, bitrate: 1764 kb/s
  Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], 1633 kb/s, 29.97 fps, 29.97 tbr, 30k tbn (default)
    Metadata:
      creation_time   : 2022-12-02T20:47:28.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 12/02/2022.
      vendor_id       : [0][0][0][0]
  Stream #0:1[0x2](eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2022-12-02T20:47:28.000000Z
      handler_name    : ISO Media file produced by Google Inc. Created on: 12/02/2022.
      vendor_id       : [0][0][0][0]
Input #1, matroska,webm, from 'file:4K Stock Footage _ No Copyright Videos _ Drone Aerial View _ Royalty Free-j-3nl88O7uI-313.f313.webm':
  Metadata:
    encoder         : google/video-file
  Duration: 00:10:39.21, start: 0.000000, bitrate: 13785 kb/s
  Stream #1:0(eng): Video: vp9 (Profile 0), yuv420p(tv, bt709), 3840x2160, SAR 1:1 DAR 16:9, 29.97 fps, 29.97 tbr, 1k tbn (default)
Stream map '1:a:0' matches no streams.
ERROR: To ignore this, add a trailing '?' to the map.
Traceback (most recent call last):
  File "C:\Users\****\Downloads\venv_youtube_dl\Lib\site-packages\youtube_dl\YoutubeDL.py", line 2162, in post_process
    files_to_delete, info = pp.run(info)
                            ^^^^^^^^^^^^
  File "C:\Users\****\Downloads\venv_youtube_dl\Lib\site-packages\youtube_dl\postprocessor\ffmpeg.py", line 527, in run
    self.run_ffmpeg_multiple_files(info['__files_to_merge'], temp_filename, args)
  File "C:\Users\****\Downloads\venv_youtube_dl\Lib\site-packages\youtube_dl\postprocessor\ffmpeg.py", line 239, in run_ffmpeg_multiple_files
    raise FFmpegPostProcessorError(msg)
youtube_dl.postprocessor.ffmpeg.FFmpegPostProcessorError: To ignore this, add a trailing '?' to the map.

Description

When merging the video and audio of two formats into a single file using -f <video-format>+<audio-format>, youtube-dl only validates that the first format contains a video. It does not do so for the second format, which must contain an audio. Here is the reference to the source code where the validation happens:

youtube-dl/youtube_dl/YoutubeDL.py

Lines 1364 to 1370 in fa7f0ef

	# The first format must contain the video and the
	# second the audio
	if formats_info[0].get('vcodec') == 'none':
	self.report_error('The first format must '
	'contain the video, try using '
	'"-f %s+%s"' % (format_2, format_1))
	return

So, if the first format contains a video, but the second format does not contain an audio, the youtube-dl won't throw an error. However, during post processing, when merging the two formats into a single file using ffmpeg, the ffmpeg will throw an error. It is because the ffmpeg expects the first format to contain a video stream and the second format to contain an audio stream. When it does not find an audio stream in the second format, it throws an error and quits. Here is the reference to the post processing source code where the manual mapping of stream selection for the ffmpeg is hardcoded:

youtube-dl/youtube_dl/postprocessor/ffmpeg.py

Line 525 in fa7f0ef

args = ['-c', 'copy', '-map', '0:v:0', '-map', '1:a:0']

It is clear that the youtube-dl intended to enforce the format types when merging, but the missing/incomplete validation fails to sanitize the inputs for the ffmpeg, which hence, throws an error.

Testing:

I'm testing on the youtube video with the id=j-3nl88O7uI, using the format selection -f mp4+webm. For the first format (mp4), the youtube-dl downloads the best quality format of that particular file extension served as a single file, which happens to be a video that contains both the video and audio streams (video+audio with the format code 22). For the second format (webm), the youtube-dl downloads a video that contains only the video stream (video-only with the format code 313).

Even though, the second format does not contain an audio stream, the current validation - enforced for merging two formats into a single file - passes and so, the youtube-dl does not throw an error. However, during post processing, the ffmpeg throws an error during merging, due to the missing audio stream in the second format, as shown in the output above. In the error, it instructs user to add a trailing '?' to the second map in order to ignore the map, if the specified stream is not present.

[dirkf]

The logic could (and probably will) be improved, on the lines of yt-dlp, but fundamentally mp4+webm is a bad choice for a format selection. This is probably why the issue isn't regularly, or ever until now AFAIK, reported.

[pukkandan]

The logic could (and probably will) be improved, on the lines of yt-dlp,

If you do, pls consider yt-dlp/yt-dlp#4846 (multistreams)