Getting error when use Proxy to download from Youtube

[nielsonsantana]

I'm trying use a proxy to download files but getting error on urllib when try download video from youtube.

ydl = youtube_dl.YoutubeDL({'outtmpl': '%(id)s%(ext)s', 
    "proxy": "http://proxyXXXX.appspot.com/", "verbose": True})
# Add all the available extractors
ydl.add_default_info_extractors()

result = ydl.extract_info('http://www.youtube.com/watch?v=BaW_jenozKc', 
    download=False # We just want to extract the info
)

Traceback:

ERROR: Unable to download webpage: ''; please report this issue on https://yt-dl.org/bug . 
Be sure to call youtube-dl with the --verbose flag and include its complete output. Make sure you
are using the latest version; type  youtube-dl -U  to update.
  File "/home/nsantana/Dropbox/workspace/baixatube/youtube_dl/extractor/common.py", line 202, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/home/nsantana/Dropbox/workspace/baixatube/youtube_dl/YoutubeDL.py", line 1228, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 404, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 422, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
    result = func(*args)
  File "/home/nsantana/Dropbox/workspace/baixatube/youtube_dl/utils.py", line 591, in https_open
    return self.do_open(HTTPSConnectionV3, req)
  File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
    h.request(req.get_method(), req.get_selector(), req.data, headers)
  File "/usr/lib/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/usr/lib/python2.7/httplib.py", line 791, in send
    self.connect()
  File "/home/nsantana/Dropbox/workspace/baixatube/youtube_dl/utils.py", line 583, in connect
    self._tunnel()
  File "/usr/lib/python2.7/httplib.py", line 752, in _tunnel
    (version, code, message) = response._read_status()
  File "/usr/lib/python2.7/httplib.py", line 373, in _read_status
    raise BadStatusLine(line)

[ghost]

Hi,
can you try it with port number? Proxy must be running on that port...
For example: proxyXXXX.appspot.com:3128

[nielsonsantana]

I have tried this port, but getting timeout error.
I also tried 443 and 80, but doesn't work. On appengine I can't use another ports.

[nielsonsantana]

I'm emulating a proxy on appengine. Maybe, it shoud be my problem.

[jaimeMF]

Have you verify that the proxy works in a browser?

[nielsonsantana]

Yeah, the proxy work normally on browser.

[nielsonsantana]

The in proxy use is: https://github.com/labnol/google-proxy
In here, it deployed: http://proxy-outubedl.appspot.com

[phihag]

This looks like a defective proxy indeed. Maybe it's a socks proxy? In
any case, can you give us the output you get when you add

'debug_printtraffic': True

?

[nielsonsantana]

@phihag, the traceback:

[youtube] Setting language
send: 'CONNECT www.youtube.com:443 HTTP/1.0\r\n'
send: '\r\n'
WARNING: unable to set language: ''
[youtube] BaW_jenozKc: Downloading webpage
send: u'CONNECT www.youtube.com:443 HTTP/1.0\r\n'
send: '\r\n'
ERROR: Unable to download webpage: ''; please report this issue on https://yt-dl.org/bug . Be sure to 
call youtube-dl with the --verbose flag and include its complete output. Make sure you are using the 
latest version; type  youtube-dl -U  to update.
File "/home/nsantana/Dropbox/workspace/tubedown.com/baixatube/youtube_dl/extractor/common.py", line 202, in _request_webpage
  return self._downloader.urlopen(url_or_request)
File "/home/nsantana/Dropbox/workspace/tubedown.com/baixatube/youtube_dl/YoutubeDL.py", line 1227, in urlopen
  return self._opener.open(req, timeout=self._socket_timeout)
File "/usr/lib/python2.7/urllib2.py", line 404, in open
  response = self._open(req, data)
File "/usr/lib/python2.7/urllib2.py", line 422, in _open
  '_open', req)
File "/usr/lib/python2.7/urllib2.py", line 382, in _call_chain
  result = func(*args)
File "/home/nsantana/Dropbox/workspace/tubedown.com/baixatube/youtube_dl/utils.py", line 591, in https_open
  return self.do_open(HTTPSConnectionV3, req)
File "/usr/lib/python2.7/urllib2.py", line 1181, in do_open
  h.request(req.get_method(), req.get_selector(), req.data, headers)
File "/usr/lib/python2.7/httplib.py", line 973, in request
  self._send_request(method, url, body, headers)
File "/usr/lib/python2.7/httplib.py", line 1007, in _send_request
  self.endheaders(body)
File "/usr/lib/python2.7/httplib.py", line 969, in endheaders
  self._send_output(message_body)
File "/usr/lib/python2.7/httplib.py", line 829, in _send_output
  self.send(msg)
File "/usr/lib/python2.7/httplib.py", line 791, in send
  self.connect()
File "/home/nsantana/Dropbox/workspace/tubedown.com/baixatube/youtube_dl/utils.py", line 583, in connect
  self._tunnel()
File "/usr/lib/python2.7/httplib.py", line 752, in _tunnel
  (version, code, message) = response._read_status()
File "/usr/lib/python2.7/httplib.py", line 373, in _read_status
  raise BadStatusLine(line)

[ghost]

It's a web proxy, so it won't work.

[jaimeMF]

I manage to get it to work on a browser, as long as you don't try to use https://. On youtube-dl --prefer-insecure can be used, but the video cannot be downloaded because you get an infinite redirect:

HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop.

It looks like there's a problem in the proxy.

[jaimeMF]

Actually, I wasn't using the proxy on the browser, @pulpe is right.

[nielsonsantana]

Thanks Guys! I thought it works. There are any alternative to download through other machine?
Per example: grabbervideostreaming.com/?url=<youtube_url>

I will try override the urlopen to add the prefix.
That's a good solution?

[ghost]

@nielsonsantana proxies works, but not with web proxy. Web proxy is special type of proxy used only from web browser. Easiest solution is to use some public proxies, for ex. from http://spys.ru/en/

[nielsonsantana]

@pulpe, Thanks again! I will check!