[YouTube] Format misdetection; video download fails with HTTP 404

[joepie91]

• Platform: openSUSE 13.1, x86_64, XFCE
• Version: 2015.07.21
• URL: https://www.youtube.com/watch?v=ku7jezAPDXU
• Resolver: youtube

Format list according to -F:

sven@linux-etoq:~/ytdl-test> youtube-dl -v -F https://www.youtube.com/watch?v=ku7jezAPDXU
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.youtube.com/watch?v=ku7jezAPDXU']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.07.21
[debug] Python version 2.7.6 - Linux-3.11.10-29-desktop-x86_64-with-SuSE-13.1-x86_64
[debug] exe versions: ffmpeg 2.7.1, ffprobe 2.7.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] ku7jezAPDXU: Downloading webpage
[youtube] ku7jezAPDXU: Downloading video info webpage
[youtube] ku7jezAPDXU: Extracting video information
[youtube] ku7jezAPDXU: Downloading DASH manifest
[youtube] ku7jezAPDXU: Downloading DASH manifest
[info] Available formats for ku7jezAPDXU:
format code  extension  resolution note
140          m4a        audio only DASH audio   96k , m4a_dash container, aac  @128k (44100Hz)
134          mp4        288x360    DASH video   52k , avc1.4d4015, 15fps, video only
160          mp4        116x144    DASH video  108k , avc1.420000, 15fps, video only
133          mp4        192x240    DASH video  242k , avc1.4d0000, 30fps, video only
135          mp4        384x480    DASH video 1100k , avc1.4d0000, 30fps, video only
136          mp4        576x720    DASH video 2200k , avc1.4d0000, 30fps, video only
17           3gp        176x144    
36           3gp        320x240    
5            flv        400x240    
43           webm       640x360    
18           mp4        640x360    
22           mp4        1280x720   (best)

The 1280x720 is a misdetection; the actual video player on the YouTube site only shows up to 480p:

... thus, the video download fails, as it attempts to retrieve a non-existent 720p version. Using -f 18+bestaudio made the download work as intended.

Full output of download attempt:

sven@linux-etoq:~/ytdl-test> youtube-dl -v https://www.youtube.com/watch?v=ku7jezAPDXU
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'https://www.youtube.com/watch?v=ku7jezAPDXU']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.07.21
[debug] Python version 2.7.6 - Linux-3.11.10-29-desktop-x86_64-with-SuSE-13.1-x86_64
[debug] exe versions: ffmpeg 2.7.1, ffprobe 2.7.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] ku7jezAPDXU: Downloading webpage
[youtube] ku7jezAPDXU: Downloading video info webpage
[youtube] ku7jezAPDXU: Extracting video information
[youtube] ku7jezAPDXU: Downloading DASH manifest
[youtube] ku7jezAPDXU: Downloading DASH manifest
[debug] Invoking downloader on 'https://r2---sn-mn4vg5aa-5hns.googlevideo.com/videoplayback/id/92eee37b300f0d75/itag/136/source/yt_otf/requiressl/yes/ms/au/mv/m/pl/35/mn/sn-mn4vg5aa-5hns/mm/31/ratebypass/yes/mime/video%2Fmp4/otfp/1/otf/1/lmt/1437293220313214/fexp/901816,9405185,9407465,9407943,9407991,9408093,9408196,9408710,9408940,9409246,9416126,9417285,948607/mt/1437619087/signature/140188E7E07A65C9EEB99BC0DDB036C2EF39A4C8.3B5315E2D97A74B1108CE79BB0859D6986075E48/upn/7pZd0HQxdnI/sver/3/key/dg_yt0/ip/2001:980:a4ca:1:7d51:547f:3138:59a3/ipbits/0/expire/1437640771/sparams/ip,ipbits,expire,id,itag,source,requiressl,ms,mv,pl,mn,mm,ratebypass,mime,otfp,otf,lmt/'
[download] Destination: Cop in Sandra Bland Traffic Stop Now on Desk Duty-ku7jezAPDXU.f136.mp4
[DashSegments] ku7jezAPDXU: Downloading initialization segment
ERROR: unable to download video data: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1429, in process_info
    partial_success = dl(fname, new_info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1377, in dl
    return fd.download(name, info)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/common.py", line 342, in download
    return self.real_download(filename, info_dict)
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/dash.py", line 45, in real_download
    'initialization segment')
  File "/usr/lib/python2.7/site-packages/youtube_dl/downloader/dash.py", line 29, in append_url_to_file
    data = self.ydl.urlopen(req).read()
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1731, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib64/python2.7/urllib2.py", line 410, in open
    response = meth(req, response)
  File "/usr/lib64/python2.7/urllib2.py", line 523, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib64/python2.7/urllib2.py", line 448, in error
    return self._call_chain(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/usr/lib64/python2.7/urllib2.py", line 531, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 404: Not Found

[joepie91]

Another similar failure: https://www.youtube.com/watch?v=C61uJbBJ7JY. However, this one does show 720p in the formats list on the YouTube site.

-F:

format code  extension  resolution note
140          m4a        audio only DASH audio  128k , m4a_dash container, aac  @128k (44100Hz)
160          mp4        256x144    DASH video  108k , avc1.420000, 15fps, video only
133          mp4        426x240    DASH video  242k , avc1.4d0000, 30fps, video only
134          mp4        640x360    DASH video  445k , avc1.4d401e, 30fps, video only
135          mp4        854x480    DASH video 1100k , avc1.4d0000, 30fps, video only
136          mp4        1280x720   DASH video 2200k , avc1.4d0000, 30fps, video only
17           3gp        176x144    
36           3gp        320x240    
5            flv        400x240    
43           webm       640x360    
18           mp4        640x360    
22           mp4        1280x720   (best)

Failed download:

[download] Destination: Sandra Bland arrest captured on video, family doubts suicide death-C61uJbBJ7JY.f136.mp4
[DashSegments] C61uJbBJ7JY: Downloading initialization segment
ERROR: unable to download video data: HTTP Error 404: Not Found

Successful download, when specifying -f 22+bestaudio (which actually grabs the 720p version, but is a different format from 136...):

[download] Destination: Sandra Bland arrest captured on video, family doubts suicide death-C61uJbBJ7JY.f22.mp4
[download] 100% of 18.10MiB in 00:01

[jaimeMF]

Thanks for the report, it will download them in the next version.