PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds

[ncoghlan]

BPO	21462
Nosy	@loewis, @ncoghlan, @tjguk, @benjaminp, @zware, @zooba
Files	issue21462.diff 224ca86e3919_backport.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2014-05-31.18:31:39.470>
created_at = <Date 2014-05-09.12:12:06.948>
labels = ['type-feature', 'OS-windows', 'build']
title = 'PEP 466: upgrade OpenSSL in the Python 2.7 Windows builds'
updated_at = <Date 2014-06-02.21:24:03.316>
user = 'https://github.com/ncoghlan'

bugs.python.org fields:

activity = <Date 2014-06-02.21:24:03.316>
actor = 'zach.ware'
assignee = 'none'
closed = True
closed_date = <Date 2014-05-31.18:31:39.470>
closer = 'benjamin.peterson'
components = ['Build', 'Windows']
creation = <Date 2014-05-09.12:12:06.948>
creator = 'ncoghlan'
dependencies = []
files = ['35200', '35415']
hgrepos = []
issue_num = 21462
keywords = ['patch']
message_count = 16.0
messages = ['218154', '218171', '218176', '218180', '218193', '218315', '219359', '219396', '219403', '219407', '219422', '219463', '219464', '219465', '219466', '219467']
nosy_count = 7.0
nosy_names = ['loewis', 'ncoghlan', 'tim.golden', 'benjamin.peterson', 'python-dev', 'zach.ware', 'steve.dower']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue21462'
versions = ['Python 2.7']

[ncoghlan]

PEP-466 includes updating to a newer version of OpenSSL. This may be needed for the ssl module feature backports in bpo-21308.

[zware]

Here's all the patch that should be necessary to do the upgrade. Running test_ssl on 2.7 with 1.0.1g I do have a failure:

======================================================================
ERROR: test_socketserver (test.test_ssl.ThreadedTests)
Using a SocketServer to create and manage SSL connections.
----------------------------------------------------------------------

Traceback (most recent call last):
  File "P:\ath\to\2.7\cpython\lib\test\test_ssl.py", line 1179, in test_socketserver
    f = urllib.urlopen(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 87, in urlopen
    return opener.open(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 208, in open
    return getattr(self, name)(url)
  File "P:\ath\to\2.7\cpython\lib\urllib.py", line 437, in open_https
    h.endheaders(data)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 829, in _send_output
    self.send(msg)
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 791, in send
    self.connect()
  File "P:\ath\to\2.7\cpython\lib\httplib.py", line 1176, in connect
    self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 392, in wrap_socket
    ciphers=ciphers)
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 148, in __init__
    self.do_handshake()
  File "P:\ath\to\2.7\cpython\lib\ssl.py", line 310, in do_handshake
    self._sslobj.do_handshake()
IOError: [Errno socket error] [Errno 1] _ssl.c:510: error:140770FC:SSL routines:
SSL23_GET_SERVER_HELLO:unknown protocol

---

But this looks closely related to a failure that I have on this machine using 3.4 (which is probably the fault of the way this network is set up):

======================================================================
ERROR: test_socketserver (test.test_ssl.ThreadedTests)
Using a SocketServer to create and manage SSL connections.
----------------------------------------------------------------------

Traceback (most recent call last):
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1189, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1090, in request
    self._send_request(method, url, body, headers)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1128, in _send_request
    self.endheaders(body)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1086, in endheaders
    self._send_output(message_body)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 924, in _send_output
    self.send(msg)
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 859, in send
    self.connect()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 1221, in connect
    super().connect()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 839, in connect
    self._tunnel()
  File "P:\ath\to\3.4\cpython\lib\http\client.py", line 822, in _tunnel
    message.strip()))
OSError: Tunnel connection failed: 403 Forbidden

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "P:\ath\to\3.4\cpython\lib\test\test_ssl.py",line 2315, in test_socketserver
    f = urllib.request.urlopen(url)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 153, in urlopen
    return opener.open(url, data, timeout)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 455, in open
    response = self._open(req, data)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 473, in _open
    '_open', req)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 433, in _call_chain
    result = func(*args)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1230, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "P:\ath\to\3.4\cpython\lib\urllib\request.py", line 1192, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error Tunnel connection failed: 403 Forbidden>

---

As such, I'll leave it to someone else to properly test this before committing.

[zooba]

I just tried it and had no trouble building and running the ssl tests on Windows.

python Lib\test\regrtest.py -u network -v test_ssl
...
Ran 38 tests in 7.700s

OK (skipped=2)

[zware]

Thanks, Steve. Nick, I assume 1.0.1g is the target version?

[tjguk]

Builds & tests ok here on a fresh checkout (of cpython & openssl-1.0.1g)

[ncoghlan]

Yes, since OpenSSL 1.0.2 is still in beta, the target version for 2.7.7 would be 1.0.1g

[zware]

Steve or Tim, did one of you want to commit this since you can run a clean test?

[zooba]

I can commit it, though I don't know how it'll affect Benjamin's release branch?

(Obviously the build will be fine either way - I had the patch applied for 2.7.7rc1.)

[zware]

Go ahead and commit; it will be up to Benjamin to cherry-pick it to his release branch (or to ask you to do it).

[python-dev]

New changeset f6e47d27f67a by Steve Dower in branch '2.7':
Issue bpo-21462 PEP-466: upgrade OpenSSL in the Python 2.7 Windows builds
http://hg.python.org/cpython/rev/f6e47d27f67a

[zware]

Looks like the AMD64 buildbot isn't happy with OpenSSL 1.0.1g: http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%202.7/builds/1223/steps/compile/logs/stdio

Here's my best stab-in-the-dark guess at fixing what's wrong; I don't have the ability to build x64 2.7 handy.

[python-dev]

New changeset e90024db8d6c by Benjamin Peterson in branch '2.7':
openssl requires nasm (bpo-21462)
http://hg.python.org/cpython/rev/e90024db8d6c

[benjaminp]

On Fri, May 30, 2014, at 12:54, Zachary Ware wrote:

Zachary Ware added the comment:

Looks like the AMD64 buildbot isn't happy with OpenSSL 1.0.1g:
http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%202.7/builds/1223/steps/compile/logs/stdio

Here's my best stab-in-the-dark guess at fixing what's wrong; I don't
have the ability to build x64 2.7 handy.

Good call. That seems to have satisfied the AMD64 bot (at least in the
compile stage).

[python-dev]

New changeset 7095a9bfbe76 by Steve Dower in branch '2.7':
Issue bpo-21462 PEP-466: upgrade OpenSSL in the Python 2.7 Windows builds
http://hg.python.org/cpython/rev/7095a9bfbe76

New changeset 8e2cda42b466 by Benjamin Peterson in branch '2.7':
openssl requires nasm (bpo-21462)
http://hg.python.org/cpython/rev/8e2cda42b466

[ncoghlan]

Will this be in 2.7.7? If so, I'll update the draft What's New changes in issue bpo-21569 accordingly (as well as the implementation status in PEP-466)

[benjaminp]

Yes.

On Sat, May 31, 2014, at 11:30, Nick Coghlan wrote:

Nick Coghlan added the comment:

Will this be in 2.7.7? If so, I'll update the draft What's New changes in
issue bpo-21569 accordingly (as well as the implementation status in PEP
466)

----------

---

Python tracker <report@bugs.python.org>
<http://bugs.python.org/issue21462\>

---