Connecting to HTTPS port using HTTP when using 'builtin' SSL

[Safihre]

We (SABnzbd) are investigating if we need pyOpenSSL at all, therefore we tried running with server.ssl_module: 'builtin' on CherryPy 8.1.0.
However, when connecting to the HTTPS port via HTTP we get an ugly error that doesn't happen when using pyOpenSSL as ssl_module.

2016-09-07 16:25:57,895::ERROR::[_cplogging:216] [07/Sep/2016:16:25:57] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\wsgiserver2.py", line 1948, in start
    self.tick()
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\wsgiserver2.py", line 2015, in tick
    s, ssl_env = self.ssl_adapter.wrap(s)
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\ssl_builtin.py", line 54, in wrap
    ssl_version=ssl.PROTOCOL_SSLv23)
  File "C:\Python27\lib\ssl.py", line 933, in wrap_socket
    ciphers=ciphers)
  File "C:\Python27\lib\ssl.py", line 601, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:590)

Using pyOpenSSL the browser will show:

The client sent a plain HTTP request, but this server only speaks HTTPS on this port.

[jaraco]

I think you may have a bug in your installation. CherryPy 8.1.0 doesn't have a wsgiserver2 module. I'd check your site-packages directory and make sure you have only one installation of cherrypy 8.1.0. Then report back if the issue persists.

[jaraco]

I think you may have a bug in your installation. CherryPy 8.1.0 doesn't have a wsgiserver2 module. I'd check your site-packages directory and make sure you have only one installation of cherrypy 8.1.0. Then report back if the issue persists.

[Safihre]

Sorry about that, old log.
Retested and used print cherrypy.__version__ to confirm the version when starting the server:

V: 8.1.0

2016-09-07 20:40:02,651::ERROR::[_cplogging:217] [07/Sep/2016:20:40:02] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\__init__.py", line 2019, in start
    self.tick()
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\__init__.py", line 2086, in tick
    s, ssl_env = self.ssl_adapter.wrap(s)
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\ssl_builtin.py", line 67, in wrap
    server_side=True)
  File "C:\Python27\lib\ssl.py", line 353, in wrap_socket
    _context=self)
  File "C:\Python27\lib\ssl.py", line 601, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:590)

[Safihre]

Sorry about that, old log.
Retested and used print cherrypy.__version__ to confirm the version when starting the server:

V: 8.1.0

2016-09-07 20:40:02,651::ERROR::[_cplogging:217] [07/Sep/2016:20:40:02] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\__init__.py", line 2019, in start
    self.tick()
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\__init__.py", line 2086, in tick
    s, ssl_env = self.ssl_adapter.wrap(s)
  File "C:\Python27\Lib\site-packages\cherrypy\wsgiserver\ssl_builtin.py", line 67, in wrap
    server_side=True)
  File "C:\Python27\lib\ssl.py", line 353, in wrap_socket
    _context=self)
  File "C:\Python27\lib\ssl.py", line 601, in __init__
    self.do_handshake()
  File "C:\Python27\lib\ssl.py", line 830, in do_handshake
    self._sslobj.do_handshake()
SSLError: [SSL: HTTP_REQUEST] http request (_ssl.c:590)

[Safihre]

Actually, my previous (removed) comment was wrong. The error is here because on Windows/Ubuntu the error message doesn't end with http request (see the error above), but this text is in the middle.
https://github.com/cherrypy/cherrypy/blob/master/cherrypy/wsgiserver/ssl_builtin.py#L82

It should be:

if 'http request' in e.args[1]:

I can submit a PR, but I can image you feel this change is too small to warrant that!

@jaraco We were also wondering why it just doesn't do a redirect to the HTTPS version of the URL? Since it knows that the HTTPS version exists?
This is one of the last reasons we had to include a modified copy of CherryPy in our application, instead of using the distribution.

[Safihre]

Actually, my previous (removed) comment was wrong. The error is here because on Windows/Ubuntu the error message doesn't end with http request (see the error above), but this text is in the middle.
https://github.com/cherrypy/cherrypy/blob/master/cherrypy/wsgiserver/ssl_builtin.py#L82

It should be:

if 'http request' in e.args[1]:

I can submit a PR, but I can image you feel this change is too small to warrant that!

@jaraco We were also wondering why it just doesn't do a redirect to the HTTPS version of the URL? Since it knows that the HTTPS version exists?
This is one of the last reasons we had to include a modified copy of CherryPy in our application, instead of using the distribution.

[Safihre]

Feel this should be handled also by the _handle_no_ssl() function you made 58511cd for #1401?

[Safihre]

Feel this should be handled also by the _handle_no_ssl() function you made 58511cd for #1401?

[Safihre]

Interesting how the pyOpenSSL wrapper doesn't actually throw NoSSLError in the tick() when a HTTP client connects to HTTPS port, how could that be?
Using the builtin it does throw that error.
It just continues to create a socket and continue to make a HTTPConnection.

[Safihre]

Interesting how the pyOpenSSL wrapper doesn't actually throw NoSSLError in the tick() when a HTTP client connects to HTTPS port, how could that be?
Using the builtin it does throw that error.
It just continues to create a socket and continue to make a HTTPConnection.

[Safihre]

In my work on this I encountered another error.

When Firefox connects for the first time to a server that has a self-signed certificate, it will show the unsafe-server page, but on the server this happens (using unmodified CherryPy 8.1.0):

Traceback (most recent call last):
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1398, in communicate
    req.parse_request()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 601, in parse_request
    success = self.read_request_line()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 634, in read_request_line
    request_line = self.rfile.readline()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 303, in readline
    data = self.rfile.readline(256)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1215, in readline
    data = self.recv(self._rbufsize)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1069, in recv
    data = self._sock.recv(size)
  File "C:\Python27\lib\ssl.py", line 756, in recv
    return self.read(buflen)
  File "C:\Python27\lib\ssl.py", line 643, in read
    v = self._sslobj.read(len)
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1754)

So it seems the connection goes past the handshake, but then gets somehow gets treated differently by Firefox.

[Safihre]

In my work on this I encountered another error.

When Firefox connects for the first time to a server that has a self-signed certificate, it will show the unsafe-server page, but on the server this happens (using unmodified CherryPy 8.1.0):

Traceback (most recent call last):
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1398, in communicate
    req.parse_request()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 601, in parse_request
    success = self.read_request_line()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 634, in read_request_line
    request_line = self.rfile.readline()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 303, in readline
    data = self.rfile.readline(256)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1215, in readline
    data = self.recv(self._rbufsize)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1069, in recv
    data = self._sock.recv(size)
  File "C:\Python27\lib\ssl.py", line 756, in recv
    return self.read(buflen)
  File "C:\Python27\lib\ssl.py", line 643, in read
    v = self._sslobj.read(len)
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1754)

So it seems the connection goes past the handshake, but then gets somehow gets treated differently by Firefox.

[Safihre]

No idea if anyone is reading this, but for the sake of thoroughness and making sure it's not the fault of the OS, Python version or certificates:

• Tested on Windows and Ubuntu using Python 2.7 and 3.5, using self-signed certificates generated using the guide on the docs.
• No problems when using pyopenssl as ssl_module, but when using builtin all the problems described above occur both in Python 2.7 and 3.5.
• Also on Python 3.5 it will throw the TLSV1_ALERT_UNKNOWN_CA when Firefox connects.
• I created a patch (link) for that error as well, which will work on Python 2.7, but not on Python 3.5 since it doesn't use the CP_makefile_PY2.

[Safihre]

No idea if anyone is reading this, but for the sake of thoroughness and making sure it's not the fault of the OS, Python version or certificates:

• Tested on Windows and Ubuntu using Python 2.7 and 3.5, using self-signed certificates generated using the guide on the docs.
• No problems when using pyopenssl as ssl_module, but when using builtin all the problems described above occur both in Python 2.7 and 3.5.
• Also on Python 3.5 it will throw the TLSV1_ALERT_UNKNOWN_CA when Firefox connects.
• I created a patch (link) for that error as well, which will work on Python 2.7, but not on Python 3.5 since it doesn't use the CP_makefile_PY2.

[jaraco]

We do read these, but the issues come in faster than we can handle them. If you can come up with a sound solution, send it as a PR. Thanks. I'll hope to look at #1499 soon.

[jaraco]

We do read these, but the issues come in faster than we can handle them. If you can come up with a sound solution, send it as a PR. Thanks. I'll hope to look at #1499 soon.