Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Configuration problem with youtube-dl on Ubuntu or being blocked by youtube? #9477
Comments
|
same for us |
|
Anything suspicious with Did you try it with another setting for |
|
I'll try that, thanks! |
|
The user agent was normal: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/44.0 (Chrome) However, the problem is gone--though I'm using a new IP address. The next time I can get it to happen, I'll check the user agent. However, I'm going to be extra cautious from here on out to avoid it. Is there no precedence in youtube-dl history for people being temporarily (or permanently?) blocked[/throttled/otherwise attenuated] by youtube? that's surprising! |
|
Not that I'm aware of.. But it is very much possible that some IPs or rather IP ranges are blocked or throttled because of unusual traffic, although only temporarily. |
|
Could you try: And: |
|
looks like it probably is an IP block or throttle. we (huffduff-video) started seeing this a couple days ago too from 52.37.171.81 (AWS EC2 us-west-2), first with 2016.04.13 and now with 2016.05.10. however, i'm able to download that video ok from 67.169.95.105 (Comcast residential, San Francisco). here are the transcripts. first EC2 us-west-2: youtube-dl -v -4 "https://www.youtube.com/watch?v=5qVK7GIZ394"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'-4', u'https://www.youtube.com/watch?v=5qVK7GIZ394']
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.10 - Linux-4.4.8-20.46.amzn1.x86_64-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 2.5.4-, ffprobe 2.5.4-
[debug] Proxy map: {}
[youtube] 5qVK7GIZ394: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 503: Service Unavailable (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
File "/usr/local/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 388, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1945, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 469, in error
result = self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 656, in http_error_302
return self.parent.open(new, timeout=req.timeout)
File "/usr/lib64/python2.7/urllib2.py", line 437, in open
response = meth(req, response)
File "/usr/lib64/python2.7/urllib2.py", line 550, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/lib64/python2.7/urllib2.py", line 475, in error
return self._call_chain(*args)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/lib64/python2.7/urllib2.py", line 558, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
$ youtube-dl -v -6 "https://www.youtube.com/watch?v=5qVK7GIZ394"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'-6', u'https://www.youtube.com/watch?v=5qVK7GIZ394']
[debug] Encodings: locale ANSI_X3.4-1968, fs ANSI_X3.4-1968, out ANSI_X3.4-1968, pref ANSI_X3.4-1968
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.10 - Linux-4.4.8-20.46.amzn1.x86_64-x86_64-with-glibc2.2.5
[debug] exe versions: ffmpeg 2.5.4-, ffprobe 2.5.4-
[debug] Proxy map: {}
[youtube] 5qVK7GIZ394: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 101] Network is unreachable> (caused by URLError(error(101, 'Network is unreachable'),))
File "/usr/local/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 388, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "/usr/local/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 1945, in urlopen
return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib64/python2.7/urllib2.py", line 431, in open
response = self._open(req, data)
File "/usr/lib64/python2.7/urllib2.py", line 449, in _open
'_open', req)
File "/usr/lib64/python2.7/urllib2.py", line 409, in _call_chain
result = func(*args)
File "/usr/local/lib/python2.7/site-packages/youtube_dl/utils.py", line 932, in https_open
req, **kwargs)
File "/usr/lib64/python2.7/urllib2.py", line 1199, in do_open
raise URLError(err)and Comcast SF: $ youtube-dl -v -4 "https://www.youtube.com/watch?v=5qVK7GIZ394"
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'-v', u'-4', u'https://www.youtube.com/watch?v=5qVK7GIZ394']
[debug] Encodings: locale US-ASCII, fs utf-8, out US-ASCII, pref US-ASCII
[debug] youtube-dl version 2016.05.10
[debug] Python version 2.7.10 - Darwin-15.4.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 2.8.6, ffprobe 2.8.6
[debug] Proxy map: {}
[youtube] 5qVK7GIZ394: Downloading webpage
[youtube] 5qVK7GIZ394: Downloading video info webpage
[youtube] 5qVK7GIZ394: Extracting video information
[youtube] 5qVK7GIZ394: Downloading MPD manifest
WARNING: Requested formats are incompatible for merge and will be merged into mkv.
[debug] Invoking downloader on u'https://r3---sn-n4v7sn76.googlevideo.com/videoplayback?id=e6a54aec6219dfde&itag=135&source=youtube&requiressl=yes&nh=IgpwcjAzLnNqYzA3KgkxMjcuMC4wLjE&mn=sn-n4v7sn76&mv=m&ms=au&pl=17&initcwndbps=2321250&mm=31&ratebypass=yes&mime=video/mp4&gir=yes&clen=18666655&lmt=1420363726995111&dur=285.666&fexp=9406984,9416126,9416891,9422596,9428398,9431012,9433096,9433223,9433946,9436447&key=dg_yt0&signature=2296FDC9FA417CE6130702B465B1B1447F023A17.2A384B0B37A915DBD0DFBF38E949E26E7F6BC92B&mt=1463262197&upn=tRQPu42Hm8A&sver=3&ip=67.169.95.105&ipbits=0&expire=1463283981&sparams=ip,ipbits,expire,id,itag,source,requiressl,nh,mn,mv,ms,pl,initcwndbps,mm,ratebypass,mime,gir,clen,lmt,dur'
[download] Destination: Wo y3 me taste-5qVK7GIZ394.f135.mp4
[download] 100% of 17.80MiB in 00:16
[debug] Invoking downloader on u'https://r3---sn-n4v7sn76.googlevideo.com/videoplayback?id=e6a54aec6219dfde&itag=251&source=youtube&requiressl=yes&nh=IgpwcjAzLnNqYzA3KgkxMjcuMC4wLjE&mn=sn-n4v7sn76&mv=m&ms=au&pl=17&initcwndbps=2321250&mm=31&ratebypass=yes&mime=audio/webm&gir=yes&clen=4832926&lmt=1413956629572797&dur=285.781&fexp=9406984,9416126,9416891,9422596,9428398,9431012,9433096,9433223,9433946,9436447&key=dg_yt0&signature=81480B23F888B73B85F94A9D7E8265F84AB97058.5B2B2A85F4833B11E74F66F038050581BEA131B1&mt=1463262197&upn=tRQPu42Hm8A&sver=3&ip=67.169.95.105&ipbits=0&expire=1463283981&sparams=ip,ipbits,expire,id,itag,source,requiressl,nh,mn,mv,ms,pl,initcwndbps,mm,ratebypass,mime,gir,clen,lmt,dur'
[download] Destination: Wo y3 me taste-5qVK7GIZ394.f251.webm
[download] 100% of 4.61MiB in 00:03
[ffmpeg] Merging formats into "Wo y3 me taste-5qVK7GIZ394.mkv"
[debug] ffmpeg command line: ffmpeg -y -i 'file:Wo y3 me taste-5qVK7GIZ394.f135.mp4' -i 'file:Wo y3 me taste-5qVK7GIZ394.f251.webm' -c copy -map 0:v:0 -map 1:a:0 'file:Wo y3 me taste-5qVK7GIZ394.temp.mkv'
Deleting original file Wo y3 me taste-5qVK7GIZ394.f135.mp4 (pass -k to keep)
Deleting original file Wo y3 me taste-5qVK7GIZ394.f251.webm (pass -k to keep) |
|
@snarfed Is the output of |
|
for us, it didn't work within a docker launched on a centOS instance, but worked well directly from the instance. strange |
|
@yan12125 i believe so, yes...but not any more, youtube-dl is working happily on the AWS VM now. looks like the block wasn't permanent. |
|
@nsilberman @eriophora Is the problem still? |
|
Hi @snarfed @yan12125 @junmuzi , Or if so, could anyone provide how to add chrome browser into youtube-dl, so youtube-dl could download the youtube video as normal. In other words, youtube-dl could be faked as a chrome browser. Thank you very much. # youtube-dl -v -6 "https://www.youtube.com/watch?v=5qVK7GIZ394" # curl -v -6 "https://www.youtube.com/watch?v=5qVK7GIZ394"
< HTTP/1.1 302 Found 302 MovedThe document has moved here. \ * Connection #0 to host www.youtube.com left intact# youtube-dl --version |
|
Seems can not work still when I change my IPv6 addr. |
|
FWIW, |
|
# curl -v -L -6 "https://www.youtube.com/watch?v=5qVK7GIZ394"
< HTTP/1.1 302 Found
< HTTP/1.1 503 Service Unavailable <script src="https://www.google.com/recaptcha/api.js" async defer></script> To continue, please type the characters below: <style>.jsonly{display: none}</style>
About this page
Our systems have detected unusual traffic from your computer network. This page checks to see if it's really you sending the requests, and not a robot. Why did this happen?
This page appears when Google automatically detects requests coming from your computer network which appear to be in violation of the Terms of Service. The block will expire shortly after those requests stop. In the meantime, solving the above CAPTCHA will let you continue to use our services.
This traffic may have been sent by malicious software, a browser plug-in, or a script that sends automated requests. If you share your network connection, ask your administrator for help — a different computer using the same IP address may be responsible. Learn more Sometimes you may be asked to solve the CAPTCHA if you are using advanced terms that robots are known to use, or sending requests very quickly. IP address: 2001:da8:203:3201:6e0b:84ff:fe0a:51af |
|
hi @yan12125 , Based on above info "Our systems have detected unusual traffic from your computer network. ", seems it is due to my large num of requests. Seems it still can not work when I just change my IPv6 addr simply. Could you give me some advise to solve this issue? Thank you very much. |
If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Description of your issue, suggested solution and other information
As you can see above, I'm getting a
503: Unavailableerror for any youtube video I try to download on my Ubuntu machine. However, on my mac machine, these videos download fine. Further, when I attempt tocurlthe URL on ubuntu:(which works like normal on the mac)
On the Ubuntu machine, I can
curlyoutube just fine, however. Please help!