SSL not working

Originally reported by: Anonymous

---

Using a recent checkout of CherryPy 3.2.5 and a recent nightly build of PyPy, SSL support doesn't seem to work.

I have tried pyOpenSSL 0.12 (pypy version), pyOpenSSL 0.14, and CherryPy "builtin". In all cases, CherryPy starts up and runs fine, throws no errors, but I can't connect via HTTPS.

In production I'm using an older (circa 9 months) nightly build of PyPy, CherryPy 3.2.4 and pyOpenSSL 0.12 and it works fine.

My connection code:

#!python

server2 = cherrypy._cpserver.Server()
            server2.socket_port = 443
            server2.socket_host = LISTEN_HOST
            server2.thread_pool = 30
            server2.ssl_module = 'pyopenssl'
            server2.ssl_certificate = SSL_CERTIFICATE_PATH
            server2.ssl_private_key = SSL_KEY_PATH
            server2.subscribe()

---

• Bitbucket: https://bitbucket.org/cherrypy/cherrypy/issue/1298

Original comment by zoomorph (Bitbucket: zoomorph, GitHub: zoomorph):

---

Best I can get...

Using "pyopenssl" module:

When sending HTTP to mysite:443 - "The client sent a plain HTTP request, but this server only speaks HTTPS on this port."

When sending HTTPS - Nothing... browser fails to connect, CherryPy doesn't log anything.

When using "builtin" module, I get this error when sending HTTP to mysite:443:
SSLError: [Errno 0] The handshake operation timed out

Again nothing when connecting via HTTPS.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

This was broken in CherryPy 3.2.5 by commit 4f2ef8d. Specifically, the changes in the init method of the HttpConnection class prevent SSL from working at all. To test, I'm just using the HelloWorld() tutorial with SSL configured, and testing with curl. Here's what the failure looks like:

[nkinder@localhost cherrypy-test]$ curl -v --cert-type pem --cacert ./cacert.pem "https://localhost.localdomain:8443"
* About to connect() to localhost.localdomain port 8443 (#0)
*   Trying ::1...
* connected
* Connected to localhost.localdomain (::1) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: ./cacert.pem
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*   subject: CN=localhost.localdomain,O="test  organization"
*   start date: Mar 19 02:43:11 2014 GMT
*   expire date: Mar 19 02:43:11 2015 GMT
*   common name: localhost.localdomain
*   issuer: CN=test organization
> GET / HTTP/1.1
> User-Agent: curl/7.27.0
> Host: localhost.localdomain:8443
> Accept: */*
> 
* SSL read: errno -12263 (SSL_ERROR_RX_RECORD_TOO_LONG)
* SSL received a record that exceeded the maximum permissible length.
* Closing connection #0
curl: (56) SSL received a record that exceeded the maximum permissible length.

If I revert the changes in this area, I am able to get SSL to work correctly.

Original comment by Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe):

---

So pypy does not work with the high level socket interface when is just a plain socket but it does when is using SSL? That was the reason of the _sock that was changed on the init of the HTTPConnection.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

I was testing SSL with Python 2.7.3, not PyPy. SSL works fine with the high level socket interface.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

I just tried my SSL test with PyPy 1.9.0 and the default branch of CherryPy, and it works fine with this code change:

#!python

diff -r 080abbbd67ba cherrypy/wsgiserver/wsgiserver2.py
--- a/cherrypy/wsgiserver/wsgiserver2.py    Thu Mar 06 16:23:42 2014 -0500
+++ b/cherrypy/wsgiserver/wsgiserver2.py    Wed Mar 19 07:32:17 2014 -0700
@@ -1324,8 +1324,8 @@
     def __init__(self, server, sock, makefile=CP_fileobject):
         self.server = server
         self.socket = sock
-        self.rfile = makefile(sock._sock, "rb", self.rbufsize)
-        self.wfile = makefile(sock._sock, "wb", self.wbufsize)
+        self.rfile = makefile(sock, "rb", self.rbufsize)
+        self.wfile = makefile(sock, "wb", self.wbufsize)
         self.requests_seen = 0

     def communicate(self):

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

I also tested on another system using PyPy 2.2.1 with the changes from my previous comment, and it appears to work (even using SSL).

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):

---

I'm confused with this. As Joel states, IIRC we had to provide the actual socket because PyPy was being picky. See #1263. So now we have to revert this change?

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

I only had to revert the portion of the change that I mentioned in the diff above. Reverting the whole change breaks PyPy. It seems that PyPy doesn't have a problem with using the high level socket in this section of code. With this change, both Python and PyPy work for me (with and without SSL).

There was also another report of SSL being broken with PyPy in issue #1293. This looks like a duplicate to me. I have asked the reporter to try the above fix to confirm that it works for him as well.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

I've submitted a pull request for this issue:

https://bitbucket.org/cherrypy/cherrypy/pull-request/63/fix-issue-1298-ssl-not-working

The reporter of SSL not working with PyPy in issue #1293 has also confirmed that this fix works for him using PyPy.

Original comment by zoomorph (Bitbucket: zoomorph, GitHub: zoomorph):

---

Hi Nathan,

Changing those 2 lines, I received the following error using a nightly build of PyPy, and the latest CherryPy from the repo:

File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1966, in start
self.tick()
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 2057, in tick
conn = self.ConnectionClass(self, s, makefile)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1327, in init
self.rfile = makefile(sock, "rb", self.rbufsize)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1004, in init
socket._fileobject.init(self, _args, *_kwargs)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/lib-python/2.7/socket.py", line 300, in init
sock._reuse()
AttributeError: '_socketobject' object has no attribute '_reuse'

I tried this twice on 2 builds and both failed. Reverting those 2 lines, the error is gone, but SSL silently fails to work. Please try using a nightly build of PyPy.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

@zoomorph

You are correct. I get the same failures using a local build of PyPy with source that I pulled from their repo today.

It looks like ssl.wrap_socket() will only work with a socket.socket object. When SSL/TLS is not being used, PyPy requires the underlying internal socket (the _sock attribute of socket.socket). Fortunately, a SSL wrapped socket (ssl.SSLSocket) implements the _reuse() and _drop() methods that PyPy expects.

I'm testing a patch that checks if we are dealing with a ssl.SSLSocket or not, and passes the SSL wrapped socket or the low-level socket (_sock) to CP_fileobject depending on what we have. This seems to work nicely with PyPy for both SSL/TLS and plain HTTPS cases using the "builtin" SSL module. I still want to run some tests with the "pyopenssl" SSL module as well as C Python before submitting a new pull request.

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):

---

It turns out that the "pyopenssl" module doesn't work the same as "builtin" here. That is, the OpenSSL.SSL.Connection class does not have _reuse() and _drop() methods, yet we must pass this high level object to CP_fileobject() for SSL/TLS to work.

I'm beginning to think the easier route to take is to simply run CherryPy with Apache HTTPD since it has better SSL support, including client certificate authentication.

Original comment by Paul Brown (Bitbucket: paul_brown_, GitHub: Unknown):

---

I'm having this issue too:
"When sending HTTP to mysite:443 - "The client sent a plain HTTP request, but this server only speaks HTTPS on this port."

When sending HTTPS - Nothing... browser fails to connect, CherryPy doesn't log anything."

I'm using:
CherryPy==3.3.0
Windows Server 2008
pyOpenSSL==0.14

Original comment by Paul Brown (Bitbucket: paul_brown_, GitHub: Unknown):

---

I switched to cherrypy==3.2.3 and it worked again.

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):

---

Unfortunately, as far as I can tell, SSL support is broken with 3.3 indeed. It just doesn't work anymore.

What's more is that, there are strange behaviors with buffering and the SSL wrapper in previous releases as far as I can tell.