"tlsv1 alert unknown ca" for builtin SSL

[Safihre]

As described in #1497, there is still 1 bug left when using server.ssl_module: 'builtin'.

When Firefox connects for the first time to a server that has a self-signed certificate, FF will show the unsafe-server page, but on the server this happens (using unmodified CherryPy 8.1.0):

Traceback (most recent call last):
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1398, in communicate
    req.parse_request()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 601, in parse_request
    success = self.read_request_line()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 634, in read_request_line
    request_line = self.rfile.readline()
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 303, in readline
    data = self.rfile.readline(256)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1215, in readline
    data = self.recv(self._rbufsize)
  File "C:\Users\user\Documents\GitHub\sabnzbd\cherrypy\wsgiserver\__init__.py", line 1069, in recv
    data = self._sock.recv(size)
  File "C:\Python27\lib\ssl.py", line 756, in recv
    return self.read(buflen)
  File "C:\Python27\lib\ssl.py", line 643, in read
    v = self._sslobj.read(len)
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1754)

This happens on both Python 2.7 and 3.5 on Win10 and Ubuntu.
I created a patch (link) for to catch this error, which will work on Python 2.7, but not on Python 3.5 since it doesn't use the CP_makefile_PY2.

Maybe you know a better solution which will also work on Python 3?
Maybe wrap the self._sock.recv(size) function for the builtin SSL module to catch such errors?

[webknjaz]

It looks SSLError should've been trapped somewhere inside HTTPConnection.communicate(). There's try/except inside of it. I think the exception should be caught there.

@jaraco what's your opinion?

[ghost]

I'm not sure when I'll get the time to look at this. Feel free to devise a solution and we can evaluate it or roll it out for comment.

[webknjaz]

Similar/duplicate: #1155

[Safihre]

@webknjaz indeed, seems the problem is deeper..

[webknjaz]

@Safihre so did you find a cure?

[Safihre]

I created a patch (link) for to catch this error, which will work on Python 2.7, but not on Python 3.5 since it doesn't use the CP_makefile_PY2.
sabnzbd/sabnzbd@1e72304#diff-c93ce94e61ff166711e8b913b7b20b72

And probably doesn't fix the problem in #1155.

[Safihre]

Turns out Safari on macOS also throws this error, but during the handshake when self-signed certificates are used:

2016-12-15 09:20:15,586::ERROR::[_cplogging:219] [15/Dec/2016:09:20:15] ENGINE Error in HTTPServer.tick
Traceback (most recent call last):
  File "/Users/xxx/Documents/sabsupport/builder/src/cherrypy/wsgiserver/__init__.py", line 2024, in start
    self.tick()
  File "/Users/xxx/Documents/sabsupport/builder/src/cherrypy/wsgiserver/__init__.py", line 2091, in tick
    s, ssl_env = self.ssl_adapter.wrap(s)
  File "/Users/xxx/Documents/sabsupport/builder/src/cherrypy/wsgiserver/ssl_builtin.py", line 67, in wrap
    server_side=True)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 363, in wrap_socket
    _context=self)
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 611, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 840, in do_handshake
    self._sslobj.do_handshake()
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:661)

Requires another catch of the exception:
sabnzbd/sabnzbd@84de0fa

[webknjaz]

@Safihre is this still happening with the newest Cheroot? (master)

[Safihre]

Well, yes and no. We are back to this one:

2019-06-23 11:45:27,812::WARNING::[_cplogging:216] [23/Jun/2019:11:45:27] ENGINE socket.error 1
Traceback (most recent call last):
  File "C:\Users\Saf\Documents\GitHub\tests\py3\lib\site-packages\cheroot\server.py", line 1263, in communicate
    req.parse_request()
  File "C:\Users\Saf\Documents\GitHub\tests\py3\lib\site-packages\cheroot\server.py", line 719, in parse_request
    success = self.read_request_line()
  File "C:\Users\Saf\Documents\GitHub\tests\py3\lib\site-packages\cheroot\server.py", line 760, in read_request_line
    request_line = self.rfile.readline()
  File "C:\Users\Saf\Documents\GitHub\tests\py3\lib\site-packages\cheroot\server.py", line 302, in readline
    data = self.rfile.readline(256)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\_pyio.py", line 513, in readline
    b = self.read(nreadahead())
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\_pyio.py", line 492, in nreadahead
    readahead = self.peek(1)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\_pyio.py", line 1076, in peek
    return self._peek_unlocked(size)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\_pyio.py", line 1083, in _peek_unlocked
    current = self.raw.read(to_read)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\ssl.py", line 1052, in recv_into
    return self.read(nbytes, buffer)
  File "C:\Users\Saf\AppData\Local\Programs\Python\Python37\Lib\ssl.py", line 911, in read
    return self._sslobj.read(len, buffer)
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:2488)

[webknjaz]

@Safihre even with the latest urllib3?

[Safihre]

This is when I connect using Chrome and Firefox with a self-signed certificate on the server.

[Safihre]

Seems like another one to add to the _block_errors?

[webknjaz]

@Safihre not exactly. According to the traceback you posted this exception happens on the later stage (after the TLS handshake), so changing this variable won't change anything.
We likely need another try/except in another place.

I assume your example uses TLS 1.3 which allows a post-handshake certificate validation. One of the instances of this problem is described @ urllib3/urllib3#1537.

I think you can check this by disabling post_handshake_auth TLS extension in the client you use.

P.S. A while back I've discovered this wonderful library http://trustme.readthedocs.io/ which allows you to test things without manually generating self-signed certificates which is closed to what you have IRL. You can use that and try modifying SSLContext. Maybe play with https://docs.python.org/3/library/ssl.html#ssl.SSLContext.post_handshake_auth.

Ref: https://tools.ietf.org/html/rfc8446#section-4.2.6 / https://tools.ietf.org/html/rfc8446#section-4.6.2

Based on this, I think we can close this issue and create another one under Cheroot to address specifically the TLS 1.3 post-handshake auth case.

[Safihre]

Based on this, I think we can close this issue and create another one under Cheroot to address specifically the TLS 1.3 post-handshake auth case.

Agreed, sounds good! 👍

[webknjaz]

@Safihre: cherrypy/cheroot#209