bestvideo and/or bestaudio select non-existent format codes for some YouTube videos

[zhuyifei1999]

As in #8955, some videos fail to download with ERROR: giving up after 10 retries, inspecting shows the format code that are attempted to download, 141 is not available on YouTube, yet youtube-dl formats still have the code available:

$ youtube-dl https://www.youtube.com/watch?v=tb-U5694Bhg -Fv
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=tb-U5694Bhg', u'-Fv']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.03.18
[debug] Python version 2.7.9 - Linux-3.19.0-2-amd64-x86_64-with-debian-8.3
[debug] exe versions: ffmpeg 2.8.6-1, ffprobe 2.8.6-1
[debug] Proxy map: {}
[youtube] tb-U5694Bhg: Downloading webpage
[youtube] tb-U5694Bhg: Downloading video info webpage
[youtube] tb-U5694Bhg: Extracting video information
[youtube] tb-U5694Bhg: Downloading MPD manifest
[info] Available formats for tb-U5694Bhg:
format code  extension  resolution note
140          m4a        audio only DASH audio  127k , m4a_dash container, mp4a.40.2@128k (44100Hz), 559.30KiB
141          m4a        audio only DASH audio  255k , m4a_dash container, mp4a.40.2@256k (44100Hz), 1.10MiB
160          mp4        256x144    DASH video  111k , avc1.42c00c, 15fps, video only, 483.89KiB
133          mp4        426x240    DASH video  245k , avc1.4d4015, 25fps, video only, 1.05MiB
134          mp4        640x360    DASH video  562k , avc1.4d401e, 25fps, video only, 2.14MiB
135          mp4        854x480    DASH video 1112k , avc1.4d401e, 25fps, video only, 4.72MiB
136          mp4        1280x720   DASH video 2204k , avc1.4d401f, 25fps, video only, 9.42MiB
137          mp4        1920x1080  DASH video 4135k , avc1.640028, 25fps, video only, 17.67MiB
264          mp4        1920x1080  DASH video 4934k , avc1.4d4028, 25fps, video only, 20.07MiB
17           3gp        176x144    small , mp4v.20.3,  mp4a.40.2@ 24k
36           3gp        320x180    small , mp4v.20.3,  mp4a.40.2
5            flv        400x224    small , h263, mp3  @ 64k
43           webm       640x360    medium , vp8.0,  vorbis@128k
18           mp4        640x360    medium , avc1.42001E,  mp4a.40.2@ 96k
22           mp4        1280x720   hd720 , avc1.64001F,  mp4a.40.2@192k (best)

Thus, downloading format bestaudio fails:

$ youtube-dl https://www.youtube.com/watch?v=tb-U5694Bhg -vf bestaudio
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'https://www.youtube.com/watch?v=tb-U5694Bhg', u'-vf', u'bestaudio']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2016.03.18
[debug] Python version 2.7.9 - Linux-3.19.0-2-amd64-x86_64-with-debian-8.3
[debug] exe versions: ffmpeg 2.8.6-1, ffprobe 2.8.6-1
[debug] Proxy map: {}
[youtube] tb-U5694Bhg: Downloading webpage
[youtube] tb-U5694Bhg: Downloading video info webpage
[youtube] tb-U5694Bhg: Extracting video information
[youtube] tb-U5694Bhg: Downloading MPD manifest
[debug] Invoking downloader on u'https://r2---sn-p5qlsnez.googlevideo.com/videoplayback?id=b5bf94e7af780618&itag=141&source=youtube&requiressl=yes&nh=IgpwcjAxLmlhZDI2KgkxMjcuMC4wLjE&mv=u&ms=au&pl=24&mm=31&mn=sn-p5qlsnez&ratebypass=yes&mime=audio/mp4&gir=yes&clen=1148340&lmt=1381155063707495&dur=35.990&upn=mxUwqVfrW1E&fexp=9405960,9408214,9416126,9419452,9420452,9422342,9422596,9423661,9423662,9426857,9427364,9427902,9429297,9431012,9431115,9431117,9431439,9431530,9431862,9432056,9432684&mt=1458913528&sver=3&key=dg_yt0&signature=3B50EC8BC20440B2B9C0315493E6857E739A047D.048FC7A8B29F9FCB4DFE074973BE102E8B638BA0&ip=208.80.155.255&ipbits=0&expire=1458935225&sparams=ip,ipbits,expire,id,itag,source,requiressl,nh,mv,ms,pl,mm,mn,ratebypass,mime,gir,clen,lmt,dur'
[download] Got server HTTP error. Retrying (attempt 1 of 10)...
[download] Got server HTTP error. Retrying (attempt 2 of 10)...
[download] Got server HTTP error. Retrying (attempt 3 of 10)...
[download] Got server HTTP error. Retrying (attempt 4 of 10)...
[download] Got server HTTP error. Retrying (attempt 5 of 10)...
[download] Got server HTTP error. Retrying (attempt 6 of 10)...
[download] Got server HTTP error. Retrying (attempt 7 of 10)...
[download] Got server HTTP error. Retrying (attempt 8 of 10)...
[download] Got server HTTP error. Retrying (attempt 9 of 10)...
[download] Got server HTTP error. Retrying (attempt 10 of 10)...
ERROR: giving up after 10 retries
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/downloader/http.py", line 58, in real_download
    data = self.ydl.urlopen(request)
  File "/usr/local/lib/python2.7/dist-packages/youtube_dl/YoutubeDL.py", line 1929, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 500: Internal Server Error

And the default bestvideo+bestaudio/best fails with the same error, shown in #8955.

Either non-existent format codes should be removed from the format list or bestvideo & bestaudio should fallback to best existent format

(On a side node, format code 140 exists and is not silent.)

[dstftw]

Duplicate of #3721.
We are well aware of the problem, no need to create duplicates.
To remove non-existent formats each format should be checked thus this will involve lots of additional network I/O that is unnecessary in most cases.