youtube-dl is transcoding a video - video already in mp4 format?

[victorhooi]

This is a followup for #3590. I am attempting to download the following video using youtube-dl:

https://www.youtube.com/watch?v=C49jEXZEqr0

Firstly, why is ffmpeg SSL support required for youtube-dl to download this particular video? And why does ffmpeg now appear to be transcoding it, when I download it?

The video is one of a series, all from the same blog (http://blog.lever.co). Below are three examples - only the middle one appears to be exhibiting this behaviour:

https://www.youtube.com/watch?v=mch6cH0g7i4
https://www.youtube.com/watch?v=C49jEXZEqr0
https://www.youtube.com/watch?v=-X0uOOHVp6s

I have included the console output in case that helps shed any light:

youtube-dl --write-sub C49jEXZEqr0
[youtube] Setting language
[youtube] Confirming age
[youtube] C49jEXZEqr0: Downloading webpage
[youtube] C49jEXZEqr0: Downloading video info webpage
[youtube] C49jEXZEqr0: Extracting video information
WARNING: video doesn't have subtitles
[youtube] C49jEXZEqr0: Downloading formats manifest
[download] Destination: Open Source Office Hours 8_8-C49jEXZEqr0.mp4
ffmpeg version 2.3.3 Copyright (c) 2000-2014 the FFmpeg developers
  built on Aug 29 2014 22:42:48 with Apple LLVM version 5.1 (clang-503.0.40) (based on LLVM 3.4svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/2.3.3 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --enable-avresample --enable-vda --cc=clang --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libxvid --enable-openssl
  libavutil      52. 92.100 / 52. 92.100
  libavcodec     55. 69.100 / 55. 69.100
  libavformat    55. 48.100 / 55. 48.100
  libavdevice    55. 13.102 / 55. 13.102
  libavfilter     4. 11.100 /  4. 11.100
  libavresample   1.  3.  0 /  1.  3.  0
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
Input #0, hls,applehttp, from 'https://manifest.googlevideo.com/api/manifest/hls_playlist/id/C49jEXZEqr0.1/itag/95/source/yt_live_broadcast/requiressl/yes/ratebypass/yes/live/1/cmbypass/yes/gir/yes/dg_shard/C49jEXZEqr0.1_95/hls_chunk_host/r5---sn-u2bpouxgoxu-ntqs.googlevideo.com/pmbypass/yes/gcr/au/maudio/1/playlist_type/CLEAN/fexp/902408,905636,927622,931983,932114,932404,932623,932625,934024,934030,937431,945118,945515,946023,946506/sver/3/upn/0iURi3tvvro/mm/32/ms/lv/mt/1409316129/mv/u/ip/203.206.228.20/ipbits/0/expire/1409337835/sparams/ip,ipbits,expire,id,itag,source,requiressl,ratebypass,live,cmbypass,gir,dg_shard,hls_chunk_host,pmbypass,gcr,maudio,playlist_type/signature/3CC6605D385144367191C3E75C1EFC846C03039E.8952E86BA66020E5787B2744E808F35C52342802/key/dg_yt0/file/index.m3u8':
  Duration: 00:57:55.00, start: 390.001000, bitrate: 1 kb/s
  Program 0
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
    Stream #0:1: Audio: aac ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 256 kb/s
Output #0, mp4, to 'Open Source Office Hours 8_8-C49jEXZEqr0.mp4.part':
  Metadata:
    encoder         : Lavf55.48.100
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 29.97 fps, 90k tbn, 90k tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, 256 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help

If I do a -F to list the formats, the output is different to another video, but it does still appear to be in mp4 format:

youtube-dl -F C49jEXZEqr0
[youtube] Setting language
[youtube] Confirming age
[youtube] C49jEXZEqr0: Downloading webpage
[youtube] C49jEXZEqr0: Downloading video info webpage
[youtube] C49jEXZEqr0: Extracting video information
[youtube] C49jEXZEqr0: Downloading formats manifest
[info] Available formats for C49jEXZEqr0:
format code extension resolution  note
151         mp4       72p         HLS  (worst)
132         mp4       240p        HLS
92          mp4       240p        HLS
93          mp4       360p        HLS
94          mp4       480p        HLS
95          mp4       720p        HLS  (best)

Is there a particular reason it needs to be transcoded?

The -F output is different to another video, but I don't know if any of the extra fields are significant:

youtube-dl -F mch6cH0g7i4
[youtube] Setting language
[youtube] Confirming age
[youtube] mch6cH0g7i4: Downloading webpage
[youtube] mch6cH0g7i4: Downloading video info webpage
[youtube] mch6cH0g7i4: Extracting video information
[info] Available formats for mch6cH0g7i4:
format code extension resolution  note
140         m4a       audio only  DASH audio , audio@128k (worst)
160         mp4       144p        DASH video , video only
133         mp4       240p        DASH video , video only
134         mp4       360p        DASH video , video only
135         mp4       480p        DASH video , video only
136         mp4       720p        DASH video , video only
17          3gp       176x144
36          3gp       320x240
5           flv       400x240
43          webm      640x360
18          mp4       640x360
22          mp4       1280x720    (best)

[phihag]

Look closely:

Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)

This is not a transcoding per se, but a a HLS download (you can also see that in the note column in the -Fdownload). Since youtube-dl does not have an HLS implementation, we use ffmpeg to download it. When HLS is being served via HTTPS, that means you'll need an ffmpeg with HTTPS and therefore SSL/TLS support.

youtube-dl may gain an HLS downloader implementation (patches welcome!), but in the meantime, relying on ffmpeg seems like the best option.