Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

https proxy self-sign certificate problem #814

Closed
ghost opened this issue May 1, 2013 · 6 comments
Closed

https proxy self-sign certificate problem #814

ghost opened this issue May 1, 2013 · 6 comments

Comments

@ghost
Copy link

@ghost ghost commented May 1, 2013

What output?
$ youtube-dl -v http://www.youtube.com/watch?v=zA92Rw6kNWw
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=zA92Rw6kNWw']
[debug] youtube-dl version 2013.04.31
[debug] Python version 3.3.1 - Linux-3.8.10-1-ARCH-i686-with-arch
[debug] Proxy map: {'https': 'http://127.0.0.1:8118', 'http': 'http://127.0.0.1:8118'}
[youtube] Setting language
[youtube] zA92Rw6kNWw: Downloading video webpage
[youtube] zA92Rw6kNWw: Downloading video info webpage
ERROR: unable to download video info webpage: urlopen error [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:547)
File "/usr/lib/python3.3/site-packages/youtube_dl/InfoExtractors.py", line 121, in _request_webpage
return compat_urllib_request.urlopen(url_or_request)
File "/usr/lib/python3.3/urllib/request.py", line 160, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.3/urllib/request.py", line 473, in open
response = self._open(req, data)
File "/usr/lib/python3.3/urllib/request.py", line 491, in _open
'_open', req)
File "/usr/lib/python3.3/urllib/request.py", line 451, in _call_chain
result = func(*args)
File "/usr/lib/python3.3/urllib/request.py", line 1287, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.3/urllib/request.py", line 1255, in do_open
raise URLError(err)

What problem?
#805 I know in the stage of "Downloading video info webpage" it will connect to a https address via https_proxy. The problem is my proxy server indeed support https, but use a self-sign certificate because it's deployed on Google App Engine which does not support https.

here's the proxy project https://github.com/jvansteirteghem/apjp
or https://github.com/goagent/goagent
( I know in normal it's no safe to use self-sign certificate. But this two projects are both open source and I know it's safe for me.)

So how to config to support self-sign certificate https proxy? or need youtube-dl to be changed some codes?

@phihag
Copy link
Contributor

@phihag phihag commented May 2, 2013

Mmm, we could either switch to HTTP only, or ignore certificate validation errors. I think an option for the latter would solve this problem.

@phihag
Copy link
Contributor

@phihag phihag commented May 2, 2013

Oh, I overlooked something - why is your self-signed certificate coming into play in the first place? Note that in the debug output, the HTTPS proxy URL starts with http://. Therefore, the SSL certificate of the proxy shouldn't come into play in the first place, and youtube-dl should just CONNECT to youtube's HTTPS server.

@ghost
Copy link
Author

@ghost ghost commented May 2, 2013

$ export http_proxy=127.0.0.1:8118 && export https_proxy=127.0.0.1:8118
$ env | grep http
http_proxy=127.0.0.1:8118
https_proxy=127.0.0.1:8118
$ youtube-dl -v http://www.youtube.com/watch?v=skYPWMM9W1A
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=skYPWMM9W1A']
[debug] youtube-dl version 2013.04.31
[debug] Python version 3.3.1 - Linux-3.8.10-1-ARCH-i686-with-arch
[debug] Proxy map: {'https': '127.0.0.1:8118', 'http': '127.0.0.1:8118'}
[youtube] Setting language
[youtube] skYPWMM9W1A: Downloading video webpage
[youtube] skYPWMM9W1A: Downloading video info webpage
ERROR: unable to download video info webpage: urlopen error [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:547)
File "/usr/lib/python3.3/site-packages/youtube_dl/InfoExtractors.py", line 121, in _request_webpage
return compat_urllib_request.urlopen(url_or_request)
File "/usr/lib/python3.3/urllib/request.py", line 160, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.3/urllib/request.py", line 473, in open
response = self._open(req, data)
File "/usr/lib/python3.3/urllib/request.py", line 491, in _open
'_open', req)
File "/usr/lib/python3.3/urllib/request.py", line 451, in _call_chain
result = func(*args)
File "/usr/lib/python3.3/urllib/request.py", line 1287, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.3/urllib/request.py", line 1255, in do_open
raise URLError(err)

$ export http_proxy=http://127.0.0.1:8118 && export https_proxy=https://127.0.0.1:8118
$ env | grep http
http_proxy=http://127.0.0.1:8118
https_proxy=https://127.0.0.1:8118
$ youtube-dl -v http://www.youtube.com/watch?v=skYPWMM9W1A
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=skYPWMM9W1A']
[debug] youtube-dl version 2013.04.31
[debug] Python version 3.3.1 - Linux-3.8.10-1-ARCH-i686-with-arch
[debug] Proxy map: {'https': 'https://127.0.0.1:8118', 'http': 'http://127.0.0.1:8118'}
[youtube] Setting language
[youtube] skYPWMM9W1A: Downloading video webpage
[youtube] skYPWMM9W1A: Downloading video info webpage
ERROR: unable to download video info webpage: urlopen error [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:547)
File "/usr/lib/python3.3/site-packages/youtube_dl/InfoExtractors.py", line 121, in _request_webpage
return compat_urllib_request.urlopen(url_or_request)
File "/usr/lib/python3.3/urllib/request.py", line 160, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.3/urllib/request.py", line 473, in open
response = self._open(req, data)
File "/usr/lib/python3.3/urllib/request.py", line 491, in _open
'_open', req)
File "/usr/lib/python3.3/urllib/request.py", line 451, in _call_chain
result = func(*args)
File "/usr/lib/python3.3/urllib/request.py", line 1287, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.3/urllib/request.py", line 1255, in do_open
raise URLError(err)

$ sudo less /etc/privoxy/config | grep forward
forward / 127.0.0.1:10000

@ghost
Copy link
Author

@ghost ghost commented May 2, 2013

$ export http_proxy=127.0.0.1:10000 && export https_proxy=127.0.0.1:10000
$ env | grep http
http_proxy=127.0.0.1:10000
https_proxy=127.0.0.1:10000
$ youtube-dl -v http://www.youtube.com/watch?v=skYPWMM9W1A
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=skYPWMM9W1A']
[debug] youtube-dl version 2013.04.31
[debug] Python version 3.3.1 - Linux-3.8.10-1-ARCH-i686-with-arch
[debug] Proxy map: {'https': '127.0.0.1:10000', 'http': '127.0.0.1:10000'}
[youtube] Setting language
[youtube] skYPWMM9W1A: Downloading video webpage
[youtube] skYPWMM9W1A: Downloading video info webpage
ERROR: unable to download video info webpage: urlopen error [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:547)
File "/usr/lib/python3.3/site-packages/youtube_dl/InfoExtractors.py", line 121, in _request_webpage
return compat_urllib_request.urlopen(url_or_request)
File "/usr/lib/python3.3/urllib/request.py", line 160, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.3/urllib/request.py", line 473, in open
response = self._open(req, data)
File "/usr/lib/python3.3/urllib/request.py", line 491, in _open
'_open', req)
File "/usr/lib/python3.3/urllib/request.py", line 451, in _call_chain
result = func(*args)
File "/usr/lib/python3.3/urllib/request.py", line 1287, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.3/urllib/request.py", line 1255, in do_open
raise URLError(err)

$ export http_proxy=http://127.0.0.1:10000 && export https_proxy=https://127.0.0.1:10000
$ env | grep http
http_proxy=http://127.0.0.1:10000
https_proxy=https://127.0.0.1:10000
$ youtube-dl -v http://www.youtube.com/watch?v=skYPWMM9W1A
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=skYPWMM9W1A']
[debug] youtube-dl version 2013.04.31
[debug] Python version 3.3.1 - Linux-3.8.10-1-ARCH-i686-with-arch
[debug] Proxy map: {'https': 'https://127.0.0.1:10000', 'http': 'http://127.0.0.1:10000'}
[youtube] Setting language
[youtube] skYPWMM9W1A: Downloading video webpage
[youtube] skYPWMM9W1A: Downloading video info webpage
ERROR: unable to download video info webpage: urlopen error [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:547)
File "/usr/lib/python3.3/site-packages/youtube_dl/InfoExtractors.py", line 121, in _request_webpage
return compat_urllib_request.urlopen(url_or_request)
File "/usr/lib/python3.3/urllib/request.py", line 160, in urlopen
return opener.open(url, data, timeout)
File "/usr/lib/python3.3/urllib/request.py", line 473, in open
response = self._open(req, data)
File "/usr/lib/python3.3/urllib/request.py", line 491, in _open
'_open', req)
File "/usr/lib/python3.3/urllib/request.py", line 451, in _call_chain
result = func(*args)
File "/usr/lib/python3.3/urllib/request.py", line 1287, in https_open
context=self._context, check_hostname=self._check_hostname)
File "/usr/lib/python3.3/urllib/request.py", line 1255, in do_open
raise URLError(err)

phihag added a commit that referenced this issue May 4, 2013
@phihag
Copy link
Contributor

@phihag phihag commented May 4, 2013

Since youtube-dl 2013.05.05, you can now pass in --no-check-certificate. If you always need that option, you can create the directory .config in %USERPROFILE% and add a file youtube-dl.conf there. The whole content of that file is then appended to the parameters on each invocation of youtube-dl.

@ghost
Copy link
Author

@ghost ghost commented May 5, 2013

It works. Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
1 participant
You can’t perform that action at this time.