Possible Issue with New Version: Unable to set language/Unable to do download webpage

[RingoTheDog]

C:\Transmogrifier>youtube-dl.py -v http://www.youtube.com/watch?v=Yl_jhvfCvOE
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=Yl_jhvfCvOE']
[debug] Encodings: locale 'cp1252', fs 'mbcs', out 'cp850', pref: 'cp1252'
[debug] youtube-dl version 2014.01.23.2
[debug] Python version 2.7.1 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
WARNING: unable to set language: <urlopen error [Errno 1] _ssl.c:499: error:1407
70FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol>
[youtube] Yl_jhvfCvOE: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 1] _ssl.c:499: error:14
0770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol>
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\common.py", line 19
0, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\YoutubeDL.py", line 1148, in
urlopen
return self._opener.open(req)
File "C:\Python27\lib\urllib2.py", line 392, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 410, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 370, in _call_chain
result = func(*args)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\utils.py", line 571, in https
_open
return self.do_open(HTTPSConnectionV3, req)
File "C:\Python27\lib\urllib2.py", line 1161, in do_open
raise URLError(err)

[RingoTheDog]

Bug still appears to be in 2014.01.23.3
(I think it showed up in ver 2014.01.23.2)

If I roll back to an old version everything works, but after the upgrade DL fails...

See below:

C:\Transmogrifier>youtube-dl.py -v http://www.youtube.com/watch?v=Yl_jhvfCvOE
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=Yl_jhvfCvOE']
[debug] youtube-dl version 2013.09.20.1
[debug] Python version 2.7.3 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
[youtube] Yl_jhvfCvOE: Downloading video webpage
[youtube] Yl_jhvfCvOE: Downloading video info webpage
[youtube] Yl_jhvfCvOE: Extracting video information
[download] Destination: WEIRDCOW-Yl_jhvfCvOE.mp4
[download] 100% of 4.21MiB in 00:0152MiB/s ETA 00:003

C:\Transmogrifier>youtube-dl.py -U
Updating to version 2014.01.23.3...
Updated youtube-dl. Restart youtube-dl to use the new version.

C:\Transmogrifier>youtube-dl.py -v http://www.youtube.com/watch?v=Yl_jhvfCvOE
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=Yl_jhvfCvOE']
[debug] Encodings: locale 'cp1252', fs 'mbcs', out 'cp850', pref: 'cp1252'
[debug] youtube-dl version 2014.01.23.3
[debug] Python version 2.7.3 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
WARNING: unable to set language: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SS
L23_GET_SERVER_HELLO:unknown protocol>
[youtube] Yl_jhvfCvOE: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:
SSL23_GET_SERVER_HELLO:unknown protocol>
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\common.py", line 191, in _request_webpa
ge
return self._downloader.urlopen(url_or_request)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\YoutubeDL.py", line 1157, in urlopen
return self._opener.open(req)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 418, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 378, in _call_chain
result = func(*args)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\utils.py", line 571, in https_open
return self.do_open(HTTPSConnectionV3, req)
File "C:\Python27\lib\urllib2.py", line 1177, in do_open
raise URLError(err)

[phihag]

Thanks for the report, that's really strange. Can you reproduce this with multiple ISPs? The only explanation that comes to my mind is foulplay by the ISP or another intermediary. Can you capture youtube-dl's traffic with wireshark and post the pcap file somewhere?

[RingoTheDog]

Will test with other ISP...but I think it is just the python, as the EXE works OK on same computer/ISP:

C:\Transmogrifier>youtube-dl.exe -v http://www.youtube.com/watch?v=Yl_jhvfCvOE
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=Yl_jhvfCvOE']
[debug] Encodings: locale 'cp1252', fs 'mbcs', out 'cp850', pref: 'cp1252'
[debug] youtube-dl version 2014.01.23.4
[debug] Python version 2.7.5 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
[youtube] Yl_jhvfCvOE: Downloading webpage
[youtube] Yl_jhvfCvOE: Downloading video info webpage
[youtube] Yl_jhvfCvOE: Extracting video information
[download] Destination: WEIRDCOW-Yl_jhvfCvOE.mp4
[download] 100% of 4.21MiB in 00:01

C:\Transmogrifier>youtube-dl.py -v http://www.youtube.com/watch?v=Yl_jhvfCvOE
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-v', 'http://www.youtube.com/watch?v=Yl_jhvfCvOE']
[debug] Encodings: locale 'cp1252', fs 'mbcs', out 'cp850', pref: 'cp1252'
[debug] youtube-dl version 2014.01.23.4
[debug] Python version 2.7.3 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
WARNING: unable to set language: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SS
L23_GET_SERVER_HELLO:unknown protocol>
[youtube] Yl_jhvfCvOE: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:
SSL23_GET_SERVER_HELLO:unknown protocol>
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\extractor\common.py", line 191, in _request_webpa
ge
return self._downloader.urlopen(url_or_request)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\YoutubeDL.py", line 1157, in urlopen
return self._opener.open(req)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 418, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 378, in _call_chain
result = func(*args)
File "C:\Transmogrifier\youtube-dl.py\youtube_dl\utils.py", line 571, in https_open
return self.do_open(HTTPSConnectionV3, req)
File "C:\Python27\lib\urllib2.py", line 1177, in do_open
raise URLError(err)

[phihag]

Oh, sorry, I did not see that you were running with Python. I'll have a look into the innards then. The reason why I suspected the ISP is that the first phase SSL negotiation should be relatively simple, and what we've changed there is demand only SSL3. I'd assumed buggy interception equipment would be at fault here. But you're right, it's definitely not that, but the system Python.

[RingoTheDog]

NP!...standing by,
Thanks (as always!) for looking into this.....

[seojoohyun]

I'm getting the same thing. The first time it happened I downloaded a few videos and then one gave me the error. I tried downloading the latest version (2014.01.23) and still got the error. After a little while I retried the same video and it worked okay. Then other videos started giving the error.
I tried a few other sites without any problems.

C:\Users\xxxx\Videos\ytdl>youtube-dl.py -f22 -v https://www.youtube.com/watch?v=KgnSTMfPR34
[debug] System config: []
[debug] User config: []
[debug] Command-line args: ['-f22', '-v', 'https://www.youtube.com/watch?v=KgnSTMfPR34']
[debug] Encodings: locale 'cp1252', fs 'mbcs', out 'cp437', pref: 'cp1252'
[debug] youtube-dl version 2014.01.23
[debug] Python version 2.7.3 - Windows-7-6.1.7601-SP1
[debug] Proxy map: {}
[youtube] Setting language
WARNING: unable to set language: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO
:unknown protocol>
[youtube] KgnSTMfPR34: Downloading webpage
ERROR: Unable to download webpage: <urlopen error [Errno 1] _ssl.c:504: error:140770FC:SSL routines:SSL23_GET_SERVER_HEL
LO:unknown protocol>
File "C:\Users\xxxx\Videos\ytdl\youtube-dl.py\youtube_dl\extractor\common.py", line 190, in _request_webpage
return self._downloader.urlopen(url_or_request)
File "C:\Users\xxxx\Videos\ytdl\youtube-dl.py\youtube_dl\YoutubeDL.py", line 1150, in urlopen
return self._opener.open(req)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 418, in _open
'_open', req)
File "C:\Python27\lib\urllib2.py", line 378, in _call_chain
result = func(*args)
File "C:\Users\xxxx\Videos\ytdl\youtube-dl.py\youtube_dl\utils.py", line 571, in https_open
return self.do_open(HTTPSConnectionV3, req)
File "C:\Python27\lib\urllib2.py", line 1177, in do_open
raise URLError(err)

[phihag]

How did you two generate youtube-dl.py? As far as I'm aware, that file is not part of our distribution.

[seojoohyun]

I download the youtube-dl file from the download page and add the .py extension. I've always done it that way, and the version I first had the problem with was actually 2014.01.05, which had been working fine for over two weeks and dozens of downloads. I'm using Windows 7.

[phihag]

Oh, I can confirm the issue on

• Windows 7 x64, youtube-dl binary, cPython 2.7

The following work fine:

• Windows 7 x64, youtube-dl.exe
• Windows 8 x64, youtube-dl.exe
• Windows 8 x64, youtube-dl binary, cPython 2.7
• Windows 8 x64, youtube-dl binary, cPython 3.3
• Windows 8 x64, python -m youtube_dl from git , cPython 2.7
• Windows 8 x64, python -m youtube_dl from git , cPython 3.3

I get a different error,

WARNING: unable to set language: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:550)>

on Windows 7 x64, youtube-dl binary, cPython 3.3

[phihag]

I'll try to diagnose it better when I have the time, probably not before the weekend.

@seojoohyun As a workaround, you can use our prebuilt exe file that comes with its own copy of Python's ssl libraries and therefore works.

[seojoohyun]

Thanks, I'll give that a go.

[phihag]

Minimum demonstration code:

import socket
import ssl
s = socket.create_connection(('www.youtube.com', 443))
wrap = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_SSLv3)

Replacing the ssl version with SSLv23 makes the code work.

Our code is something like

s = socket.create_connection(('www.youtube.com', 443))
try:
    wrap = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_SSLv3)
except ssl.SSLError:
    wrap = ssl.wrap_socket(s, ssl_version=ssl.PROTOCOL_SSLv23)

This seems to bring the socket in an invalid state. On Windows < 8 with Python < 3, we should therefore directly go for SSLv23.

[RingoTheDog]

JFYI...seems that updating from python 2.7.3 to 2.7.5 (or newer) also fixes the issue.

[phihag]

Oh, I had an old Python version on Windows 7, that explains it. I'm closing this issue then - updating Python is the solution, the exe a workaround.

[seojoohyun]

Thanks, both solutions work for me. I didn't even think about the python version, it's been working with that one for so long I guess I just took it for granted.