Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL not working #1298

Closed
ghost opened this issue Mar 6, 2014 · 25 comments
Closed

SSL not working #1298

ghost opened this issue Mar 6, 2014 · 25 comments

Comments

@ghost
Copy link

ghost commented Mar 6, 2014

Originally reported by: Anonymous


Using a recent checkout of CherryPy 3.2.5 and a recent nightly build of PyPy, SSL support doesn't seem to work.

I have tried pyOpenSSL 0.12 (pypy version), pyOpenSSL 0.14, and CherryPy "builtin". In all cases, CherryPy starts up and runs fine, throws no errors, but I can't connect via HTTPS.

In production I'm using an older (circa 9 months) nightly build of PyPy, CherryPy 3.2.4 and pyOpenSSL 0.12 and it works fine.

My connection code:

#!python

server2 = cherrypy._cpserver.Server()
            server2.socket_port = 443
            server2.socket_host = LISTEN_HOST
            server2.thread_pool = 30
            server2.ssl_module = 'pyopenssl'
            server2.ssl_certificate = SSL_CERTIFICATE_PATH
            server2.ssl_private_key = SSL_KEY_PATH
            server2.subscribe()

@ghost
Copy link
Author

ghost commented Mar 11, 2014

Original comment by zoomorph (Bitbucket: zoomorph, GitHub: zoomorph):


Best I can get...

Using "pyopenssl" module:

When sending HTTP to mysite:443 - "The client sent a plain HTTP request, but this server only speaks HTTPS on this port."

When sending HTTPS - Nothing... browser fails to connect, CherryPy doesn't log anything.

When using "builtin" module, I get this error when sending HTTP to mysite:443:
SSLError: [Errno 0] The handshake operation timed out

Again nothing when connecting via HTTPS.

@ghost
Copy link
Author

ghost commented Mar 19, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


This was broken in CherryPy 3.2.5 by commit 4f2ef8d. Specifically, the changes in the init method of the HttpConnection class prevent SSL from working at all. To test, I'm just using the HelloWorld() tutorial with SSL configured, and testing with curl. Here's what the failure looks like:

[nkinder@localhost cherrypy-test]$ curl -v --cert-type pem --cacert ./cacert.pem "https://localhost.localdomain:8443"
* About to connect() to localhost.localdomain port 8443 (#0)
*   Trying ::1...
* connected
* Connected to localhost.localdomain (::1) port 8443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: ./cacert.pem
  CApath: none
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
*   subject: CN=localhost.localdomain,O="test  organization"
*   start date: Mar 19 02:43:11 2014 GMT
*   expire date: Mar 19 02:43:11 2015 GMT
*   common name: localhost.localdomain
*   issuer: CN=test organization
> GET / HTTP/1.1
> User-Agent: curl/7.27.0
> Host: localhost.localdomain:8443
> Accept: */*
> 
* SSL read: errno -12263 (SSL_ERROR_RX_RECORD_TOO_LONG)
* SSL received a record that exceeded the maximum permissible length.
* Closing connection #0
curl: (56) SSL received a record that exceeded the maximum permissible length.

If I revert the changes in this area, I am able to get SSL to work correctly.

@ghost
Copy link
Author

ghost commented Mar 19, 2014

Original comment by Joel Rivera (Bitbucket: cyraxjoe, GitHub: cyraxjoe):


So pypy does not work with the high level socket interface when is just a plain socket but it does when is using SSL? That was the reason of the _sock that was changed on the init of the HTTPConnection.

@ghost
Copy link
Author

ghost commented Mar 19, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


I was testing SSL with Python 2.7.3, not PyPy. SSL works fine with the high level socket interface.

@ghost
Copy link
Author

ghost commented Mar 19, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


I just tried my SSL test with PyPy 1.9.0 and the default branch of CherryPy, and it works fine with this code change:

#!python

diff -r 080abbbd67ba cherrypy/wsgiserver/wsgiserver2.py
--- a/cherrypy/wsgiserver/wsgiserver2.py    Thu Mar 06 16:23:42 2014 -0500
+++ b/cherrypy/wsgiserver/wsgiserver2.py    Wed Mar 19 07:32:17 2014 -0700
@@ -1324,8 +1324,8 @@
     def __init__(self, server, sock, makefile=CP_fileobject):
         self.server = server
         self.socket = sock
-        self.rfile = makefile(sock._sock, "rb", self.rbufsize)
-        self.wfile = makefile(sock._sock, "wb", self.wbufsize)
+        self.rfile = makefile(sock, "rb", self.rbufsize)
+        self.wfile = makefile(sock, "wb", self.wbufsize)
         self.requests_seen = 0

     def communicate(self):

@ghost
Copy link
Author

ghost commented Mar 19, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


I also tested on another system using PyPy 2.2.1 with the changes from my previous comment, and it appears to work (even using SSL).

@ghost
Copy link
Author

ghost commented Mar 20, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


I'm confused with this. As Joel states, IIRC we had to provide the actual socket because PyPy was being picky. See #1263. So now we have to revert this change?

@ghost
Copy link
Author

ghost commented Mar 20, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


I only had to revert the portion of the change that I mentioned in the diff above. Reverting the whole change breaks PyPy. It seems that PyPy doesn't have a problem with using the high level socket in this section of code. With this change, both Python and PyPy work for me (with and without SSL).

There was also another report of SSL being broken with PyPy in issue #1293. This looks like a duplicate to me. I have asked the reporter to try the above fix to confirm that it works for him as well.

@ghost
Copy link
Author

ghost commented Mar 26, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


I've submitted a pull request for this issue:

https://bitbucket.org/cherrypy/cherrypy/pull-request/63/fix-issue-1298-ssl-not-working

The reporter of SSL not working with PyPy in issue #1293 has also confirmed that this fix works for him using PyPy.

@ghost
Copy link
Author

ghost commented Mar 26, 2014

Original comment by zoomorph (Bitbucket: zoomorph, GitHub: zoomorph):


Hi Nathan,

Changing those 2 lines, I received the following error using a nightly build of PyPy, and the latest CherryPy from the repo:

File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1966, in start
self.tick()
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 2057, in tick
conn = self.ConnectionClass(self, s, makefile)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1327, in init
self.rfile = makefile(sock, "rb", self.rbufsize)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/site-packages/CherryPy-3.2.6-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1004, in init
socket._fileobject.init(self, _args, *_kwargs)
File "/root/pypy-c-jit-69737-b03d8d46d83d-linux/lib-python/2.7/socket.py", line 300, in init
sock._reuse()
AttributeError: '_socketobject' object has no attribute '_reuse'

I tried this twice on 2 builds and both failed. Reverting those 2 lines, the error is gone, but SSL silently fails to work. Please try using a nightly build of PyPy.

@ghost
Copy link
Author

ghost commented Mar 27, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


@zoomorph

You are correct. I get the same failures using a local build of PyPy with source that I pulled from their repo today.

It looks like ssl.wrap_socket() will only work with a socket.socket object. When SSL/TLS is not being used, PyPy requires the underlying internal socket (the _sock attribute of socket.socket). Fortunately, a SSL wrapped socket (ssl.SSLSocket) implements the _reuse() and _drop() methods that PyPy expects.

I'm testing a patch that checks if we are dealing with a ssl.SSLSocket or not, and passes the SSL wrapped socket or the low-level socket (_sock) to CP_fileobject depending on what we have. This seems to work nicely with PyPy for both SSL/TLS and plain HTTPS cases using the "builtin" SSL module. I still want to run some tests with the "pyopenssl" SSL module as well as C Python before submitting a new pull request.

@ghost
Copy link
Author

ghost commented Mar 27, 2014

Original comment by Nathan Kinder (Bitbucket: nkinder, GitHub: nkinder):


It turns out that the "pyopenssl" module doesn't work the same as "builtin" here. That is, the OpenSSL.SSL.Connection class does not have _reuse() and _drop() methods, yet we must pass this high level object to CP_fileobject() for SSL/TLS to work.

I'm beginning to think the easier route to take is to simply run CherryPy with Apache HTTPD since it has better SSL support, including client certificate authentication.

@ghost
Copy link
Author

ghost commented Apr 26, 2014

Original comment by Paul Brown (Bitbucket: paul_brown_, GitHub: Unknown):


I'm having this issue too:
"When sending HTTP to mysite:443 - "The client sent a plain HTTP request, but this server only speaks HTTPS on this port."

When sending HTTPS - Nothing... browser fails to connect, CherryPy doesn't log anything."

I'm using:
CherryPy==3.3.0
Windows Server 2008
pyOpenSSL==0.14

@ghost
Copy link
Author

ghost commented Apr 26, 2014

Original comment by Paul Brown (Bitbucket: paul_brown_, GitHub: Unknown):


I switched to cherrypy==3.2.3 and it worked again.

@ghost
Copy link
Author

ghost commented Apr 26, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Unfortunately, as far as I can tell, SSL support is broken with 3.3 indeed. It just doesn't work anymore.

What's more is that, there are strange behaviors with buffering and the SSL wrapper in previous releases as far as I can tell.

@ghost
Copy link
Author

ghost commented Sep 9, 2014

Original comment by helix7 (Bitbucket: helix7, GitHub: helix7):


After searching the Internet and finally checking out the code and stepping through the history, I came to the same conclusion as @nkinder and ended up here. For what it's worth, replacing sock._sock with just sock within HTTPConnection.__init__() appears to make it work again for me (Python 2.7, ssl_module = 'builtin', Linux).

The errors I have seen include:

  • HTTP requires CRLF terminators
  • EOF occurred in violation of protocol
  • error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number

@ghost
Copy link
Author

ghost commented Sep 16, 2014

Original comment by mikeazo (Bitbucket: mikeazo, GitHub: mikeazo):


I have had the same SSL error with 3.5.0 and 3.6.0. To fix it, all I have to do is apply the patch @nkinder recommends and it works. Any idea when this fix will be released? Are there other issues that changing those lines might create? When I disable SSL, everything still works.

@ghost
Copy link
Author

ghost commented Dec 1, 2014

Original comment by Jesper Reenberg (Bitbucket: jesper_reenberg, GitHub: Unknown):


I also have this issue with 3.6.0 from pypi. After fixing sock._sock to sock as per above sugested, it works with ssl_module = 'pyopenssl'

Please take this bug report seriously and apply the fix.

@ghost
Copy link
Author

ghost commented Dec 3, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Hello there,

I apologise for the lack of response here these days. I do recognise this is a very critical bug that has annoyed us for too long. If people are happy with the suggested fix, so am I. However, we always need to ensure that we don't break the 3.x Python support whilst fixing for 2.x.

I will definitely look at this over the weekend (sooner if I can).

Thanks guys.

@ghost
Copy link
Author

ghost commented Dec 7, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Issue #1293 was marked as a duplicate of this issue.

@ghost
Copy link
Author

ghost commented Dec 7, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Fix #1298 for CPython 2.7 with the builtin ssl module and pyOpenSSL 0.14. However, this will break pyOpenSSL on Pypy for now. Until we find the most appropriate solution that is

@ghost
Copy link
Author

ghost commented Dec 7, 2014

Original comment by Sylvain Hellegouarch (Bitbucket: Lawouach, GitHub: Lawouach):


Here's my findings:

  • CPython 2.7.6 => reversing PyPy support broken #1263 to remove ._sock does fix the ssl issue with both the ssl builtin module and pyOpenSSL 0.14
  • Pypy 2.4 => reversing it breaks pyOpenSSL support as per PyPy support broken #1263. I thought it fixed the builtin ssl module but, even though the request goes through, CP will eventually raise the following error:
#!python

Traceback (most recent call last):
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1343, in communicate
    req.parse_request()
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 610, in parse_request
    success = self.read_request_line()
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 643, in read_request_line
    request_line = self.rfile.readline()
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 301, in readline
    data = self.rfile.readline(256)
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1163, in readline
    data = self.recv(self._rbufsize)
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/site-packages/CherryPy-3.6.1-py2.7.egg/cherrypy/wsgiserver/wsgiserver2.py", line 1030, in recv
    data = self._sock.recv(size)
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/lib-python/2.7/ssl.py", line 243, in recv
    return self.read(buflen)
  File "/home/sylvain/Dev/3rdparties/pypy-2.4.0-linux64/lib-python/2.7/ssl.py", line 162, in read
    return self._sslobj.read(len)
SSLError: [Errno 0] The read operation timed out

If I do keep the ._sock attribute as per #1263, on Pypy, the server never answers HTTPS requests but only plain HTTP when using pyOpenSSL. Go figure.

All in all, I have decided to reverse #1263 because we have so many users expecting ssl to work on CPython. However, this will break CP/SSL/PyPy/pyOpenSSL again. We will have to look into an appropriate solution that makes sense and doesn't add extra branching in a rather busy code.

@ghost
Copy link
Author

ghost commented Mar 21, 2015

Original comment by mar10 (Bitbucket: mar10, GitHub: mar10):


This issue is marked 'resolved' with milestone 3.3, but it seems that the patch was committed for the (yet unreleased) 3.6.1. Is this correct? Is there an ETA for 3.7?

@ghost
Copy link
Author

ghost commented May 14, 2015

Original comment by Donald Adu-Poku (Bitbucket: dnldd, GitHub: dnldd):


Using CherryPy 3.7 and still having this issue.

@ghost
Copy link
Author

ghost commented Jun 30, 2015

Original comment by sandeep balouria (Bitbucket: sbalouria, GitHub: Unknown):


curl -sSk https://sm-west.usw1.aws.tidemark.net:8081/login -H'Accept: application/x-yaml' -d username=saltdev -d password=saltdev -d eauth=auto
curl: (56) SSL read: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number, errno 0

using CherryPy==3.8.0

jaraco pushed a commit that referenced this issue Apr 30, 2016
….14. However, this will break pyOpenSSL on Pypy for now. Until we find the most appropriate solution that is
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

0 participants