Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected 404 on format 22 of a youtube video #20539

Closed
simon816 opened this issue Apr 3, 2019 · 1 comment
Closed

Unexpected 404 on format 22 of a youtube video #20539

simon816 opened this issue Apr 3, 2019 · 1 comment
Labels

Comments

@simon816
Copy link

@simon816 simon816 commented Apr 3, 2019

Please follow the guide below

  • You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
  • Put an x into all the boxes [ ] relevant to your issue (like this: [x])
  • Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2019.04.01. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

  • I've verified and I assure that I'm running youtube-dl 2019.04.01

Before submitting an issue make sure you have:

  • At least skimmed through the README, most notably the FAQ and BUGS sections
  • Searched the bugtracker for similar issues including closed ones
  • Checked that provided video/audio/playlist URLs (if any) are alive and playable in a browser

What is the purpose of your issue?

  • Bug report (encountered problems with youtube-dl)
  • Site support request (request for adding support for a new site)
  • Feature request (request for a new functionality)
  • Question
  • Other

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '-f', '22', 'https://www.youtube.com/watch?v=FASaRFMx3Hg']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.04.01
[debug] Python version 3.6.7 (CPython) - Linux-5.0.2-050002-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.4, ffprobe 3.4.4, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] FASaRFMx3Hg: Downloading webpage
[youtube] FASaRFMx3Hg: Downloading video info webpage
[debug] Invoking downloader on 'https://r7---sn-8pgbpohxqp5-auoe.googlevideo.com/videoplayback?lmt=1503617725171372&requiressl=yes&sparams=dur%2Cei%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpcm2cms%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&initcwndbps=2396250&expire=1554272795&key=yt6&fvip=2&mv=m&pl=22&mt=1554251074&ms=au%2Crdu&id=o-ALmiltFlAyGvzSb48oR6jhe05d1JsHUKIDJfDuFudltW&ei=u_2jXNPyMqCDmLAP5-CPsAc&mn=sn-8pgbpohxqp5-auoe%2Csn-aigl6ned&source=youtube&mm=31%2C29&ip=*myIP*&c=WEB&ratebypass=yes&itag=22&pcm2cms=yes&dur=1477.810&mime=video%2Fmp4&signature=B9015AE42BEBD188E990268583B6288208933DF5.C01C493A3A00C6E23AC051A5A7AFA72792EDF97E&ipbits=0'
[download] Destination: $8000 PC Upgrade-FASaRFMx3Hg.mp4
[download]   0.5% of 153.64MiB at  2.56MiB/s ETA 00:59[download] Got server HTTP error: Downloaded 755154 bytes, expected 161103523 bytes. Retrying (attempt 1 of 10)...
ERROR: unable to download video data: HTTP Error 404: Not Found
Traceback (most recent call last):
  File "/home/simon/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1913, in process_info
    success = dl(filename, info_dict)
  File "/home/simon/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 1852, in dl
    return fd.download(name, info)
  File "/home/simon/bin/youtube-dl/youtube_dl/downloader/common.py", line 364, in download
    return self.real_download(filename, info_dict)
  File "/home/simon/bin/youtube-dl/youtube_dl/downloader/http.py", line 341, in real_download
    establish_connection()
  File "/home/simon/bin/youtube-dl/youtube_dl/downloader/http.py", line 109, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/home/simon/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2225, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.6/urllib/request.py", line 532, in open
    response = meth(req, response)
  File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.6/urllib/request.py", line 570, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.6/urllib/request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found


Description of your issue, suggested solution and other information

This YouTube video (https://www.youtube.com/watch?v=FASaRFMx3Hg) fails with the above 404 error only on format 22. I have tried several other formats and they work as expected.

It successfully downloads the first chunk, but fails on the second.

List of formats for reference:

$ youtube-dl -F https://www.youtube.com/watch?v=FASaRFMx3Hg
[youtube] FASaRFMx3Hg: Downloading webpage
[youtube] FASaRFMx3Hg: Downloading video info webpage
[info] Available formats for FASaRFMx3Hg:
format code  extension  resolution note
249          webm       audio only DASH audio   55k , opus @ 50k, 7.34MiB
250          webm       audio only DASH audio   68k , opus @ 70k, 8.74MiB
171          webm       audio only DASH audio  100k , vorbis@128k, 14.99MiB
251          webm       audio only DASH audio  124k , opus @160k, 16.08MiB
140          m4a        audio only DASH audio  129k , m4a_dash container, mp4a.40.2@128k, 22.38MiB
278          webm       256x144    144p  100k , webm container, vp9, 25fps, video only, 14.57MiB
160          mp4        256x144    144p  111k , avc1.4d400c, 25fps, video only, 7.48MiB
242          webm       426x240    240p  225k , vp9, 25fps, video only, 23.09MiB
133          mp4        426x240    240p  234k , avc1.4d4015, 25fps, video only, 13.62MiB
243          webm       640x360    360p  410k , vp9, 25fps, video only, 42.66MiB
134          mp4        640x360    360p  634k , avc1.4d401e, 25fps, video only, 34.69MiB
244          webm       854x480    480p  767k , vp9, 25fps, video only, 73.81MiB
135          mp4        854x480    480p 1224k , avc1.4d401e, 25fps, video only, 70.30MiB
247          webm       1280x720   720p 1552k , vp9, 25fps, video only, 159.93MiB
136          mp4        1280x720   720p 2411k , avc1.4d401f, 25fps, video only, 131.35MiB
248          webm       1920x1080  1080p 2689k , vp9, 25fps, video only, 314.74MiB
137          mp4        1920x1080  1080p 4367k , avc1.640028, 25fps, video only, 354.47MiB
271          webm       2560x1440  1440p 8899k , vp9, 25fps, video only, 1.03GiB
313          webm       3840x2160  2160p 18486k , vp9, 25fps, video only, 2.81GiB
43           webm       640x360    medium , vp8.0, vorbis@128k, 111.24MiB
18           mp4        640x360    medium  473k , avc1.42001E, mp4a.40.2@ 96k (44100Hz), 83.34MiB
22           mp4        1280x720   hd720  872k , avc1.64001F, mp4a.40.2@192k (44100Hz) (best)

This is what the HTTP requests are:
(I have replaced my IP address with *myIP*)

$ youtube-dl --print-traffic -f 22 https://www.youtube.com/watch?v=FASaRFMx3Hg
[youtube] FASaRFMx3Hg: Downloading webpage
send: b'GET /watch?v=FASaRFMx3Hg&gl=US&hl=en&has_verified=1&bpctr=9999999999&disable_polymer=true HTTP/1.1\r\nHost: www.youtube.com\r\nCookie: PREF=f1=50000000&hl=en\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: X-Content-Type-Options: nosniff
header: Content-Type: text/html; charset=utf-8
header: Strict-Transport-Security: max-age=31536000
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: Content-Encoding: gzip
header: Cache-Control: no-cache
header: X-Frame-Options: SAMEORIGIN
header: Date: Wed, 03 Apr 2019 00:38:07 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Wed, 03-Apr-2019 01:08:07 GMT
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Wed, 03-Apr-2019 01:08:07 GMT
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Wed, 03-Apr-2019 01:08:07 GMT
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Wed, 03-Apr-2019 01:08:07 GMT
header: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
header: Connection: close
header: Transfer-Encoding: chunked
[youtube] FASaRFMx3Hg: Downloading video info webpage
send: b'GET /get_video_info?video_id=FASaRFMx3Hg&ps=default&eurl=&gl=US&hl=en&el=info&sts=17984&disable_polymer=true HTTP/1.1\r\nHost: www.youtube.com\r\nCookie: PREF=f1=50000000&hl=en; GPS=1; YSC=ydo_Jyv9hlk; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=QBRnjgYv00k\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: Strict-Transport-Security: max-age=31536000
header: X-Frame-Options: SAMEORIGIN
header: Content-Type: application/x-www-form-urlencoded
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: X-Content-Type-Options: nosniff
header: Cache-Control: no-store
header: Content-Length: 50
header: Date: Wed, 03 Apr 2019 00:38:08 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
header: Connection: close
send: b'GET /get_video_info?video_id=FASaRFMx3Hg&ps=default&eurl=&gl=US&hl=en&el=embedded&sts=17984&disable_polymer=true HTTP/1.1\r\nHost: www.youtube.com\r\nCookie: PREF=f1=50000000&hl=en; GPS=1; YSC=ydo_Jyv9hlk; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=QBRnjgYv00k\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Encoding: gzip, deflate\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Strict-Transport-Security: max-age=31536000
header: Content-Encoding: gzip
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Cache-Control: no-store
header: Content-Type: application/x-www-form-urlencoded
header: X-Frame-Options: SAMEORIGIN
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: X-Content-Type-Options: nosniff
header: Date: Wed, 03 Apr 2019 00:38:08 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
header: Connection: close
header: Transfer-Encoding: chunked
send: b'GET /videoplayback?id=o-AFGrgooQb4iWtEKXnOZSxc4M1B2FL0FMELLQwAwzybt9&itag=22&source=youtube&requiressl=yes&mm=31,29&mn=sn-8pgbpohxqp5-auoe,sn-aigzrn7k&ms=au,rdu&mv=m&pcm2cms=yes&pl=22&ei=bwCkXN3yKY7jhAeaqqlg&initcwndbps=2373750&mime=video/mp4&ratebypass=yes&dur=1477.810&lmt=1503617725171372&mt=1554251808&fvip=2&c=WEB&ip=*myIP*&ipbits=0&expire=1554273487&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pcm2cms,pl,ei,initcwndbps,mime,ratebypass,dur,lmt&signature=0D7131B2DAF3C56D7CE96DD2C75BBCBB1821EC45.D8066AB716A51B27ECD566285651AD548BEB4F16&key=yt8 HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: r7---sn-8pgbpohxqp5-auoe.googlevideo.com\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Last-Modified: Thu, 24 Aug 2017 23:35:25 GMT
header: Content-Type: video/mp4
header: Date: Wed, 03 Apr 2019 00:38:08 GMT
header: Expires: Wed, 03 Apr 2019 00:38:08 GMT
header: Cache-Control: private, max-age=21299
header: Accept-Ranges: bytes
header: Content-Length: 161103523
header: Connection: close
header: Alt-Svc: quic=":443"; ma=2592000; v="46,44,43,39"
header: X-Content-Type-Options: nosniff
header: Server: gvs 1.0
[download] Destination: $8000 PC Upgrade-FASaRFMx3Hg.mp4
[download]   0.5% of 153.64MiB at  3.67MiB/s ETA 00:41[download] Got server HTTP error: Downloaded 755154 bytes, expected 161103523 bytes. Retrying (attempt 1 of 10)...
send: b'GET /videoplayback?id=o-AFGrgooQb4iWtEKXnOZSxc4M1B2FL0FMELLQwAwzybt9&itag=22&source=youtube&requiressl=yes&mm=31,29&mn=sn-8pgbpohxqp5-auoe,sn-aigzrn7k&ms=au,rdu&mv=m&pcm2cms=yes&pl=22&ei=bwCkXN3yKY7jhAeaqqlg&initcwndbps=2373750&mime=video/mp4&ratebypass=yes&dur=1477.810&lmt=1503617725171372&mt=1554251808&fvip=2&c=WEB&ip=*myIP*&ipbits=0&expire=1554273487&sparams=ip,ipbits,expire,id,itag,source,requiressl,mm,mn,ms,mv,pcm2cms,pl,ei,initcwndbps,mime,ratebypass,dur,lmt&signature=0D7131B2DAF3C56D7CE96DD2C75BBCBB1821EC45.D8066AB716A51B27ECD566285651AD548BEB4F16&key=yt8 HTTP/1.1\r\nAccept-Encoding: identity\r\nHost: r7---sn-8pgbpohxqp5-auoe.googlevideo.com\r\nUser-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101 Firefox/59.0\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nAccept-Language: en-us,en;q=0.5\r\nRange: bytes=755154-\r\nConnection: close\r\n\r\n'
reply: 'HTTP/1.1 404 Not Found\r\n'
header: Last-Modified: Wed, 02 May 2007 10:26:10 GMT
header: Content-Type: text/plain
header: Content-Length: 0
header: Connection: close
header: X-Content-Type-Options: nosniff
header: Date: Wed, 03 Apr 2019 00:38:09 GMT
header: Server: gvs 1.0
ERROR: unable to download video data: HTTP Error 404: Not Found
@dstftw
Copy link
Collaborator

@dstftw dstftw commented Apr 3, 2019

Server side Youtube error.

@dstftw dstftw closed this Apr 3, 2019
@dstftw dstftw added the invalid label Apr 3, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.