Add muxing option; allow --embed-subs to automatically remux if needed

[spongbros]

Problem

A video from Crunchyroll is downloaded with --write-sub, --embed-subs and --merge-output-format mkv enabled. The expected outcome is that the downloaded FLV file will be losslessly muxed with the subtitles into an MKV file. What actually happens is that the two files are downloaded as normal without merging; the message [ffmpeg] Subtitles can only be embedded in mp4 or mkv files is returned.

Possible Solution 1

Add an option to remux the video to a given container format without re-encoding. (i.e. --mux mkv, --mux mp4 etc.)

Possible Solution 2

Change the behaviour of --embed-subs so that it automatically remuxes if the file format doesn’t allow embedded subtitles. Allow the container format (MKV, MP4) to be specified if possible.

Possible Solution 3

Change the behaviour of --merge-output-format to always remux regardless of whether or not a merge is required.

Verbose output follows.

$ youtube-dl --ignore-config -v -n --merge-output-format mkv --write-sub --sub-lang enUS --embed-subs "http://www.crunchyroll.com/symphogear/episode-1-awakening-heartbeat-685195"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--ignore-config', u'-v', u'-n', u'--merge-output-format', u'mkv', u'--write-sub', u'--sub-lang', u'enUS', u'--embed-subs', u'http://www.crunchyroll.com/symphogear/episode-1-awakening-heartbeat-685195']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.09.22
[debug] Python version 2.7.10 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8, ffprobe 2.8, rtmpdump 2.4
[debug] Proxy map: {}
[Crunchyroll] Logging in
[Crunchyroll] 685195: Downloading webpage
[Crunchyroll] 685195: Downloading media info
[Crunchyroll] 685195: Downloading media info for 360p
[Crunchyroll] 685195: Downloading media info for 480p
[Crunchyroll] 685195: Downloading media info for 720p
[Crunchyroll] 685195: Downloading media info for 1080p
[Crunchyroll] 685195: Downloading subtitles for English (US)
[Crunchyroll] 685195: Downloading subtitles for Español
[Crunchyroll] 685195: Downloading subtitles for Español (España)
[Crunchyroll] 685195: Downloading subtitles for Français (France)
[Crunchyroll] 685195: Downloading subtitles for العربية
[info] Writing video subtitles to: Symphogear Episode 1 – Awakening Heartbeat-685195.enUS.ass
[debug] Invoking downloader on 'rtmpe://cp150757.edgefcs.net/ondemand/?auth=daEa.cxcIdqdYaucYc1ciaRc7b7c9dqdLd4-bwb6oB-dHa-nCGyptNDxuu&aifp=0009&slist=c20/s/ve1953099/video.mp4'
[download] Symphogear Episode 1 – Awakening Heartbeat-685195.flv has already been downloaded
[download] 100% of 541.18MiB
[ffmpeg] Subtitles can only be embedded in mp4 or mkv files

[dstftw]

There is already an option for recode: --recode-video mp4.

[spongbros]

To clarify, I’m not trying to re-encode. MKV/MP4 muxing is a lossless action; it keeps the original video and audio streams, does not involve re-encoding and takes much less time than the recode command. In ffmpeg, this can be accomplished through the -map and -c copy commands (as seen in the log below when the subtitles are embedded). --recode-video does not use this and the stream is unnecessarily re-encoded. Please reopen this issue.

Verbose log follows. Not shown is the long wait during the encoding stage before subtitles are embedded.

$ youtube-dl --ignore-config -v -n --recode-video mp4 --write-sub --sub-lang enUS --embed-subs "http://www.crunchyroll.com/symphogear/episode-1-awakening-heartbeat-685195"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--ignore-config', u'-v', u'-n', u'--recode-video', u'mp4', u'--write-sub', u'--sub-lang', u'enUS', u'--embed-subs', u'http://www.crunchyroll.com/symphogear/episode-1-awakening-heartbeat-685195']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.09.22
[debug] Python version 2.7.10 - Darwin-14.5.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8, ffprobe 2.8, rtmpdump 2.4
[debug] Proxy map: {}
[Crunchyroll] Logging in
[Crunchyroll] 685195: Downloading webpage
[Crunchyroll] 685195: Downloading media info
[Crunchyroll] 685195: Downloading media info for 360p
[Crunchyroll] 685195: Downloading media info for 480p
[Crunchyroll] 685195: Downloading media info for 720p
[Crunchyroll] 685195: Downloading media info for 1080p
[Crunchyroll] 685195: Downloading subtitles for English (US)
[Crunchyroll] 685195: Downloading subtitles for Español
[Crunchyroll] 685195: Downloading subtitles for Español (España)
[Crunchyroll] 685195: Downloading subtitles for Français (France)
[Crunchyroll] 685195: Downloading subtitles for العربية
[info] Writing video subtitles to: Symphogear Episode 1 – Awakening Heartbeat-685195.enUS.ass
[debug] Invoking downloader on 'rtmpe://cp150757.edgefcs.net/ondemand/?auth=daEcbdtbzdWdcdydrbbdkcXa2cid4duaRb.-bwb7wZ-dHa-mDMxqtPDzxy&aifp=0009&slist=c21/s/ve1953099/video.mp4'
[download] Symphogear Episode 1 – Awakening Heartbeat-685195.flv has already been downloaded
[download] 100% of 541.18MiB
[ffmpeg] Converting video from flv to mp4, Destination: Symphogear Episode 1 – Awakening Heartbeat-685195.mp4
[debug] ffmpeg command line: ffmpeg -y -i 'file:Symphogear Episode 1 – Awakening Heartbeat-685195.flv' 'file:Symphogear Episode 1 – Awakening Heartbeat-685195.mp4'
Deleting original file Symphogear Episode 1 – Awakening Heartbeat-685195.flv (pass -k to keep)
[ffmpeg] Embedding subtitles in 'Symphogear Episode 1 – Awakening Heartbeat-685195.mp4'
[debug] ffmpeg command line: ffmpeg -y -i 'file:Symphogear Episode 1 – Awakening Heartbeat-685195.mp4' -i 'file:Symphogear Episode 1 – Awakening Heartbeat-685195.enUS.ass' -map 0 -c copy -map -0:s -c:s mov_text -map 1:0 -metadata:s:s:0 language=eng 'file:Symphogear Episode 1 – Awakening Heartbeat-685195.temp.mp4'
Deleting original file Symphogear Episode 1 – Awakening Heartbeat-685195.enUS.ass (pass -k to keep)

[nyuszika7h]

Yeah, this would be nice. Whatever subtitle format is picked (either by default or manually with --sub-format and/or --convert-subs) should be respected and it should be remuxed to a compatible container format instead of overriding the subtitle format choice.

[stephen304]

Just want to add my support to this issue. I was unaware that --embed-subs would fail if the format wasn't already in a compatible format for other reasons. I may delay my archiving tasks until this is fixed because I don't want to have to figure out down the line which ones need to have subtitles redownloaded.

It would be great if the solution allows forcing remux to mkv, because for archiving tasks, I want all of the content to have uniform filenames / container formats.

[Abu3safeer]

same here, I thought I am doing something wrong.

[varenc]

another +1 from me on this old issue!

Right now there seems to be no good solution if the source video's format doesn't support embedding subtitles. The only thing that works is a very slow and unnecessary --recode-video.

"recoding" is so slow because youtube-dl calls ffmpeg without any codec information, so ffmpeg just uses sensible defaults based on the output file's container format.

For example, here's what youtube-dl calls:
ffmpeg -y -loglevel 'repeat+info' -i 'file:video.flv' 'file:output.mkv'

after turning up ffmpeg's verbosity you can see it's made these re-encoding choices

Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> vorbis (libvorbis))

In the meantime, I've hacked up my copy of youtube-dl to always pass -c copy to ffmpeg when recoding video so that it's doesn't spend time transcoding things. We could call this --recontainer-video since it preserves the stream encodings and just changes the container. For my purposes, this is always what I want.

Here's a tiny diff in case others might find this useful:

 --- a/youtube_dl/postprocessor/ffmpeg.py
 +++ b/youtube_dl/postprocessor/ffmpeg.py
 @@ -362,6 +362,11 @@ class FFmpegVideoConvertorPP(FFmpegPostProcessor):
          options = []
          if self._preferedformat == 'avi':
              options.extend(['-c:v', 'libxvid', '-vtag', 'XVID'])
 +        ########## HACK ##########
 +        # This makes sure ffmpeg doesn't transcode anything.  It just changes the container.
 +        # this will fail in obscure situations, but this is probably what you want 90% of the time
 +        options.extend(['-c', 'copy'])
 +        ##########################
          prefix, sep, ext = path.rpartition('.')

[gw711]

This please. I want to download CR shows, which come in as MP4, but to use the .ass subtitles I need MKV format. I thought --merge-output-format was what I needed, but it only takes effect if it thinks merge is required, which it apparently doesn't. Instead, it simply downloads MP4, converts the .ass file to a different format, and merged. Not good