Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to download webpage: HTTP Error 429: Too Many Requests when using proxy #5060

Closed
pricejn2 opened this issue Feb 25, 2015 · 3 comments
Closed

Comments

@pricejn2
Copy link

@pricejn2 pricejn2 commented Feb 25, 2015

There appears to be an issue when running youtube-dl through a proxy.

youtube-dl -o '~/%(title)s_%(id)s_%(extractor)s.%(ext)s' 'https://www.youtube.com/watch?v=qILFXXJVy60' --proxy '172.16.0.1:8800' -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-o', '~/%(title)s_%(id)s_%(extractor)s.%(ext)s', 'https://www.youtube.com/watch?v=qILFXXJVy60', '--proxy', '172.16.0.1:8800', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.02.24.2
[debug] Python version 3.2.3 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-Debian-7
[debug] exe versions: avconv 1.2.12, avprobe 1.2.12, ffmpeg 1.2.12, ffprobe 1.2.12, rtmpdump 2.4
[debug] Proxy map: {'http': '172.16.0.1:8800', 'https': '172.16.0.1:8800'}
[youtube] qILFXXJVy60: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/lib/python3.2/dist-packages/youtube_dl/extractor/common.py", line 314, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/lib/python3.2/dist-packages/youtube_dl/YoutubeDL.py", line 1665, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python3.2/urllib/request.py", line 376, in open
    response = meth(req, response)
  File "/usr/lib/python3.2/urllib/request.py", line 488, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python3.2/urllib/request.py", line 414, in error
    return self._call_chain(*args)
  File "/usr/lib/python3.2/urllib/request.py", line 348, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.2/urllib/request.py", line 496, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)

The proxy itself is working as expected:

http_proxy=172.16.0.1:8800 curl http://www.realip.info/api/p/realip.php
{"IP":"111.123.567.123"}

However, if I use one of the proxy IP addresses directly, there is no problem (IP generalized):

youtube-dl -o '~/%(title)s_%(id)s_%(extractor)s.%(ext)s' 'https://www.youtube.com/watch?v=qILFXXJVy60' --source-address '111.123.567.123' -v
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-o', '~/%(title)s_%(id)s_%(extractor)s.%(ext)s', 'https://www.youtube.com/watch?v=qILFXXJVy60', '--source-address', '111.123.567.123', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2015.02.24.2
[debug] Python version 3.2.3 - Linux-3.14.32-xxxx-grs-ipv6-64-x86_64-with-Debian-7
[debug] exe versions: avconv 1.2.12, avprobe 1.2.12, ffmpeg 1.2.12, ffprobe 1.2.12, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] qILFXXJVy60: Downloading webpage
[youtube] qILFXXJVy60: Extracting video information
[youtube] qILFXXJVy60: Downloading DASH manifest
[debug] Invoking downloader on 'https://r2---sn-u2oxu-f5f6.googlevideo.com/videoplayback?ip=198.27.102.143&mime=video%2Fmp4&requiressl=yes&key=yt5&signature=25D330D8367F8FF5407D559C81C04E96947DD051.12E29BF60E0E75933D43300BB7C00F85F6F487EA&mm=31&ipbits=0&ratebypass=yes&sparams=dur%2Cid%2Cip%2Cipbits%2Citag%2Cmime%2Cmm%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cupn%2Cexpire&mv=u&id=a882c55d7255cbad&ms=au&source=youtube&pl=23&dur=299.723&expire=1424895988&sver=3&mt=1424873828&fexp=905657%2C907263%2C927622%2C934954%2C9406573%2C9407468%2C943917%2C947225%2C947240%2C948124%2C951703%2C952302%2C952605%2C952612%2C952901%2C955301%2C957201%2C959701&itag=22&upn=axR49iTxs30'
[download] Destination: /root/SHIGATSU WA KIMI NO USO (OPENING 2) - NANAIRO SYMPHONY (Cover)_qILFXXJVy60_youtube.mp4
[download] 100% of 59.74MiB in 00:05

Per #4728, I have tested with Python 2.7.3 and 3.2.3 (Debian Wheezy package versions)

@Soahr
Copy link

@Soahr Soahr commented Feb 25, 2015

Same here ...

@pricejn2
Copy link
Author

@pricejn2 pricejn2 commented Feb 25, 2015

This is now working for me with no changes on my end.

If there are any thoughts as to why this might have happened, I'd love to hear them. If not, this can be closed.

@jaimeMF jaimeMF closed this Feb 25, 2015
@pricejn2
Copy link
Author

@pricejn2 pricejn2 commented Mar 1, 2015

Just a note that this continues to recur. I have switched to --source-address in the meantime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.