Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SSL tests fail with uncaught SSL validation #173

Closed
jaraco opened this issue Feb 4, 2019 · 25 comments

Comments

@jaraco
Copy link
Member

commented Feb 4, 2019

❓ I'm submitting a ...

  • 🐞 bug report
  • 🐣 feature request
  • ❓ question about the decisions made in the repository

🐞 Describe the bug. What is the current behavior?
Running tests on macOS, two tests fail:

       2 failed
         - cheroot/test/test_ssl.py:192 test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl]
         - cheroot/test/test_ssl.py:192 test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-pyopenssl]

❓ What is the motivation / use case for changing the behavior?

Tests should pass on master.

πŸ’‘ To Reproduce

Steps to reproduce the behavior:

  1. Run tox -r on macOS

πŸ’‘ Expected behavior

Tests should pass.

πŸ“‹ Details

Failures look like:

――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――

mocker = <pytest_mock.MockFixture object at 0x103995ba8>, tls_http_server = <generator object tls_http_server.<locals>.start_srv at 0x10392c228>, adapter_type = 'pyopenssl'
ca = <trustme.CA object at 0x103995fd0>, tls_certificate = <trustme.LeafCert object at 0x10396bb70>
tls_certificate_chain_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpg0wjfzji.pem'
tls_certificate_private_key_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpii4yp07w.pem'
tls_ca_certificate_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpyv5h7s8l.pem', is_trusted_cert = False, tls_client_identity = 'localhost'
tls_verify_mode = <VerifyMode.CERT_OPTIONAL: 1>

    @pytest.mark.parametrize(
        'adapter_type',
        (
            'builtin',
            'pyopenssl',
        ),
    )
    @pytest.mark.parametrize(
        'is_trusted_cert,tls_client_identity',
        (
            (True, 'localhost'), (True, '127.0.0.1'),
            (True, '*.localhost'), (True, 'not_localhost'),
            (False, 'localhost'),
        ),
    )
    @pytest.mark.parametrize(
        'tls_verify_mode',
        (
            ssl.CERT_NONE,  # server shouldn't validate client cert
            ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
            ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
        ),
    )
    def test_tls_client_auth(
        # FIXME: remove twisted logic, separate tests
        mocker,
        tls_http_server, adapter_type,
        ca,
        tls_certificate,
        tls_certificate_chain_pem_path,
        tls_certificate_private_key_pem_path,
        tls_ca_certificate_pem_path,
        is_trusted_cert, tls_client_identity,
        tls_verify_mode,
    ):
        """Verify that client TLS certificate auth works correctly."""
        test_cert_rejection = (
            tls_verify_mode != ssl.CERT_NONE
            and not is_trusted_cert
        )
        interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
    
        client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
        with mocker.mock_module.patch(
            'idna.core.ulabel',
            return_value=ntob(tls_client_identity),
        ):
            client_cert = client_cert_root_ca.issue_server_cert(
                # FIXME: change to issue_cert once new trustme is out
                ntou(tls_client_identity),
            )
            del client_cert_root_ca
    
        with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
            tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
            tls_adapter = tls_adapter_cls(
                tls_certificate_chain_pem_path,
                tls_certificate_private_key_pem_path,
            )
            if adapter_type == 'pyopenssl':
                tls_adapter.context = tls_adapter.get_context()
                tls_adapter.context.set_verify(
                    _stdlib_to_openssl_verify[tls_verify_mode],
                    lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
                )
            else:
                tls_adapter.context.verify_mode = tls_verify_mode
    
            ca.configure_trust(tls_adapter.context)
            tls_certificate.configure_cert(tls_adapter.context)
    
            tlshttpserver = tls_http_server.send(
                (
                    (interface, port),
                    tls_adapter,
                ),
            )
    
            interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
    
            make_https_request = functools.partial(
                requests.get,
                'https://' + interface + ':' + str(port) + '/',
    
                # Server TLS certificate verification:
                verify=tls_ca_certificate_pem_path,
    
                # Client TLS certificate verification:
                cert=cl_pem,
            )
    
            if not test_cert_rejection:
                resp = make_https_request()
                assert resp.status_code == 200
                assert resp.text == 'Hello world!'
                return
    
            with pytest.raises(requests.exceptions.SSLError) as ssl_err:
>               make_https_request()

cheroot/test/test_ssl.py:290: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/python/lib/python3.7/site-packages/requests/api.py:75: in get
    return request('get', url, params=params, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/api.py:60: in request
    return session.request(method=method, url=url, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:533: in request
    resp = self.send(prep, **send_kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:646: in send
    r = adapter.send(request, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/adapters.py:449: in send
    timeout=timeout
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:600: in urlopen
    chunked=chunked)
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:384: in _make_request
    six.raise_from(e, None)
<string>:2: in raise_from
    ???
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:380: in _make_request
    httplib_response = conn.getresponse()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:1321: in getresponse
    response.begin()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:296: in begin
    version, status, reason = self._read_status()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:257: in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py:589: in readinto
    return self._sock.recv_into(b)
.tox/python/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py:294: in recv_into
    return self.connection.recv_into(*args, **kwargs)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1822: in recv_into
    self._raise_ssl_error(self._ssl, result)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.
    
        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []
    
        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))
    
>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]

.tox/python/lib/python3.7/site-packages/OpenSSL/_util.py:54: Error

 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_OPTIONAL-False-localhost-pyopenssl] β¨―                                                                      84% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ– 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-localhost-builtin] βœ“                                                                         85% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Œ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-localhost-pyopenssl] βœ“                                                                       86% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-127.0.0.1-builtin] βœ“                                                                         87% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‹ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-127.0.0.1-pyopenssl] βœ“                                                                       88% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Š 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-*.localhost-builtin] βœ“                                                                       89% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-*.localhost-pyopenssl] βœ“                                                                     90% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‰ 
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-not_localhost-builtin] βœ“                                                                     90% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-True-not_localhost-pyopenssl] βœ“                                                                   91% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž
 cheroot/test/test_ssl.py::test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-builtin] βœ“                                                                        92% β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–Ž

――――――――――――――――――――――――――――――――――――――――――――――――――――――― test_tls_client_auth[VerifyMode.CERT_REQUIRED-False-localhost-pyopenssl] ――――――――――――――――――――――――――――――――――――――――――――――――――――――――

mocker = <pytest_mock.MockFixture object at 0x10395c6d8>, tls_http_server = <generator object tls_http_server.<locals>.start_srv at 0x103a1ae58>, adapter_type = 'pyopenssl'
ca = <trustme.CA object at 0x103ee1160>, tls_certificate = <trustme.LeafCert object at 0x103c2f630>
tls_certificate_chain_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmp78479eyf.pem'
tls_certificate_private_key_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpwdexlrl7.pem'
tls_ca_certificate_pem_path = '/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/tmpf3obdeu1.pem', is_trusted_cert = False, tls_client_identity = 'localhost'
tls_verify_mode = <VerifyMode.CERT_REQUIRED: 2>

    @pytest.mark.parametrize(
        'adapter_type',
        (
            'builtin',
            'pyopenssl',
        ),
    )
    @pytest.mark.parametrize(
        'is_trusted_cert,tls_client_identity',
        (
            (True, 'localhost'), (True, '127.0.0.1'),
            (True, '*.localhost'), (True, 'not_localhost'),
            (False, 'localhost'),
        ),
    )
    @pytest.mark.parametrize(
        'tls_verify_mode',
        (
            ssl.CERT_NONE,  # server shouldn't validate client cert
            ssl.CERT_OPTIONAL,  # same as CERT_REQUIRED in client mode, don't use
            ssl.CERT_REQUIRED,  # server should validate if client cert CA is OK
        ),
    )
    def test_tls_client_auth(
        # FIXME: remove twisted logic, separate tests
        mocker,
        tls_http_server, adapter_type,
        ca,
        tls_certificate,
        tls_certificate_chain_pem_path,
        tls_certificate_private_key_pem_path,
        tls_ca_certificate_pem_path,
        is_trusted_cert, tls_client_identity,
        tls_verify_mode,
    ):
        """Verify that client TLS certificate auth works correctly."""
        test_cert_rejection = (
            tls_verify_mode != ssl.CERT_NONE
            and not is_trusted_cert
        )
        interface, _host, port = _get_conn_data(ANY_INTERFACE_IPV4)
    
        client_cert_root_ca = ca if is_trusted_cert else trustme.CA()
        with mocker.mock_module.patch(
            'idna.core.ulabel',
            return_value=ntob(tls_client_identity),
        ):
            client_cert = client_cert_root_ca.issue_server_cert(
                # FIXME: change to issue_cert once new trustme is out
                ntou(tls_client_identity),
            )
            del client_cert_root_ca
    
        with client_cert.private_key_and_cert_chain_pem.tempfile() as cl_pem:
            tls_adapter_cls = get_ssl_adapter_class(name=adapter_type)
            tls_adapter = tls_adapter_cls(
                tls_certificate_chain_pem_path,
                tls_certificate_private_key_pem_path,
            )
            if adapter_type == 'pyopenssl':
                tls_adapter.context = tls_adapter.get_context()
                tls_adapter.context.set_verify(
                    _stdlib_to_openssl_verify[tls_verify_mode],
                    lambda conn, cert, errno, depth, preverify_ok: preverify_ok,
                )
            else:
                tls_adapter.context.verify_mode = tls_verify_mode
    
            ca.configure_trust(tls_adapter.context)
            tls_certificate.configure_cert(tls_adapter.context)
    
            tlshttpserver = tls_http_server.send(
                (
                    (interface, port),
                    tls_adapter,
                ),
            )
    
            interface, _host, port = _get_conn_data(tlshttpserver.bind_addr)
    
            make_https_request = functools.partial(
                requests.get,
                'https://' + interface + ':' + str(port) + '/',
    
                # Server TLS certificate verification:
                verify=tls_ca_certificate_pem_path,
    
                # Client TLS certificate verification:
                cert=cl_pem,
            )
    
            if not test_cert_rejection:
                resp = make_https_request()
                assert resp.status_code == 200
                assert resp.text == 'Hello world!'
                return
    
            with pytest.raises(requests.exceptions.SSLError) as ssl_err:
>               make_https_request()

cheroot/test/test_ssl.py:290: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.tox/python/lib/python3.7/site-packages/requests/api.py:75: in get
    return request('get', url, params=params, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/api.py:60: in request
    return session.request(method=method, url=url, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:533: in request
    resp = self.send(prep, **send_kwargs)
.tox/python/lib/python3.7/site-packages/requests/sessions.py:646: in send
    r = adapter.send(request, **kwargs)
.tox/python/lib/python3.7/site-packages/requests/adapters.py:449: in send
    timeout=timeout
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:600: in urlopen
    chunked=chunked)
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:384: in _make_request
    six.raise_from(e, None)
<string>:2: in raise_from
    ???
.tox/python/lib/python3.7/site-packages/urllib3/connectionpool.py:380: in _make_request
    httplib_response = conn.getresponse()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:1321: in getresponse
    response.begin()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:296: in begin
    version, status, reason = self._read_status()
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py:257: in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py:589: in readinto
    return self._sock.recv_into(b)
.tox/python/lib/python3.7/site-packages/urllib3/contrib/pyopenssl.py:294: in recv_into
    return self.connection.recv_into(*args, **kwargs)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1822: in recv_into
    self._raise_ssl_error(self._ssl, result)
.tox/python/lib/python3.7/site-packages/OpenSSL/SSL.py:1647: in _raise_ssl_error
    _raise_current_error()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

exception_type = <class 'OpenSSL.SSL.Error'>

    def exception_from_error_queue(exception_type):
        """
        Convert an OpenSSL library failure into a Python exception.
    
        When a call to the native OpenSSL library fails, this is usually signalled
        by the return value, and an error code is stored in an error queue
        associated with the current thread. The err library provides functions to
        obtain these error codes and textual error messages.
        """
        errors = []
    
        while True:
            error = lib.ERR_get_error()
            if error == 0:
                break
            errors.append((
                text(lib.ERR_lib_error_string(error)),
                text(lib.ERR_func_error_string(error)),
                text(lib.ERR_reason_error_string(error))))
    
>       raise exception_type(errors)
E       OpenSSL.SSL.Error: [('SSL routines', 'ssl3_read_bytes', 'tlsv1 alert unknown ca')]

.tox/python/lib/python3.7/site-packages/OpenSSL/_util.py:54: Error

πŸ“‹ Environment

  • Cheroot version: master (e616c25)
  • CherryPy version: n/a
  • Python version: 3.7.2
  • OS: macOS 10.14.3
  • Browser: n/a

πŸ“‹ Additional context

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

It appears that those tests are attempting to catch the SSL Exception, but requests.exceptions.SSLError doesn't include OpenSSL error. Maybe requests switched to OpenSSL... or will rely on it in certain environments?

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

@jaraco jaraco changed the title SSL tests fail with uncaught SSL validation on macOS SSL tests fail with uncaught SSL validation Feb 4, 2019

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Looks like tests passed 17 days ago. At the time, these were the deps installed:

apipkg==1.5
argh==0.26.2
asn1crypto==0.24.0
atomicwrites==1.2.1
attrs==18.2.0
backports.functools-lru-cache==1.5
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
-e git+https://github.com/cherrypy/cheroot.git@d179b71c9a8c43d2595b6a892f41f44ccfd72bb4#egg=cheroot
codecov==2.0.15
colorama==0.4.1
coverage==4.5.2
cryptography==2.4.2
ddt==1.2.0
docopt==0.6.2
execnet==1.5.0
idna==2.8
more-itertools==5.0.0
packaging==18.0
pathtools==0.1.2
pluggy==0.8.1
py==1.7.0
pycparser==2.19
pyOpenSSL==18.0.0
pyparsing==2.3.1
pytest==4.1.1
pytest-cov==2.6.1
pytest-forked==1.0.1
pytest-mock==1.10.0
pytest-sugar==0.9.2
pytest-testmon==0.9.14
pytest-watch==4.2.0
pytest-xdist==1.26.0
PyYAML==3.13
requests==2.21.0
requests-unixsocket==0.1.5
six==1.12.0
termcolor==1.1.0
trustme==0.4.0
urllib3==1.24.1
watchdog==0.9.0
@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

Yeah... About that. It looks like different combos of python runtime (cpython/pypy) + underlying ssl lib (openssl/libressl) + it's version (0.9/1.0/1.1) produce weirdly different results.

I was trying to identify certain patterns but it's not usually obvious what's going on. Sometime test would behave differently under the seemingly similar env (like real macbook vs Travis CI vs Circle CI with the same version of macOS and openssl lib).

So currently I'm trying to figure this out time to time when possible and then give up periodically...

I also noticed that upgrade of pyOpenSSL affected the behavior.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Here we have one of the early failures where here are the deps installed:

apipkg==1.5
argh==0.26.2
asn1crypto==0.24.0
atomicwrites==1.2.1
attrs==18.2.0
backports.functools-lru-cache==1.5
certifi==2018.11.29
cffi==1.11.5
chardet==3.0.4
-e git+https://github.com/cherrypy/cheroot.git@d179b71c9a8c43d2595b6a892f41f44ccfd72bb4#egg=cheroot
codecov==2.0.15
colorama==0.4.1
coverage==4.5.2
cryptography==2.5
ddt==1.2.0
docopt==0.6.2
execnet==1.5.0
idna==2.8
more-itertools==5.0.0
packaging==19.0
pathtools==0.1.2
pluggy==0.8.1
py==1.7.0
pycparser==2.19
pyOpenSSL==19.0.0
pyparsing==2.3.1
pytest==4.1.1
pytest-cov==2.6.1
pytest-forked==1.0.1
pytest-mock==1.10.0
pytest-sugar==0.9.2
pytest-testmon==0.9.14
pytest-watch==4.2.0
pytest-xdist==1.26.0
PyYAML==3.13
requests==2.21.0
requests-unixsocket==0.1.5
six==1.12.0
termcolor==1.1.0
trustme==0.5.0
urllib3==1.24.1
watchdog==0.9.0
@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

One of the issues is that currently code catches and ignores "safe" exceptions during the initial TLS handshake (i.e. on connect) but I saw those happenning on the later stage (basically when we're trying to read from the network stream) as well in some cases.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

I tried downgrading to pyopenssl 18, but that didn't help.

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

maybe some transitive dependency upgrade + it also depends on the actual openssl backend

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

Oh and this "combo" issue thing is kinda complicated: the stdlib ssl module can be linked against different version of openssl, also pyopenssl (cryptography really) can be linked against something different

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Installing the full set of deps from that successful build does work around the issue:

Installing collected packages: atomicwrites, cryptography, packaging, pytest, pytest-mock, pytest-xdist, trustme
  Found existing installation: atomicwrites 1.3.0
    Uninstalling atomicwrites-1.3.0:
      Successfully uninstalled atomicwrites-1.3.0
  Found existing installation: cryptography 2.5
    Uninstalling cryptography-2.5:
      Successfully uninstalled cryptography-2.5
  Found existing installation: packaging 19.0
    Uninstalling packaging-19.0:
      Successfully uninstalled packaging-19.0
  Found existing installation: pytest 4.2.0
    Uninstalling pytest-4.2.0:
      Successfully uninstalled pytest-4.2.0
  Found existing installation: pytest-mock 1.10.1
    Uninstalling pytest-mock-1.10.1:
      Successfully uninstalled pytest-mock-1.10.1
  Found existing installation: pytest-xdist 1.26.1
    Uninstalling pytest-xdist-1.26.1:
      Successfully uninstalled pytest-xdist-1.26.1
  Found existing installation: trustme 0.5.0
    Uninstalling trustme-0.5.0:
      Successfully uninstalled trustme-0.5.0
Successfully installed atomicwrites-1.2.1 cryptography-2.4.2 packaging-18.0 pytest-4.1.1 pytest-mock-1.10.0 pytest-xdist-1.26.0 trustme-0.4.0
@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

And PyPy has its own CFFI based wrapper for OpenSSL while CPython has that in pure C and cryptography also does pure C thing I think.

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

Okay, from diff it seems that it's cryptography's fault... But I'm curious what it's build against now and whether it matches a shared lib actually present on the machine.

jaraco added a commit that referenced this issue Feb 4, 2019

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

According to the changelog, cryptography 2.5 introduced this change:

Updated Windows, macOS, and manylinux1 wheels to be compiled with OpenSSL 1.1.1a.

Based on your other assertions above, this sounds like a likely proximate cause.

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

@jaraco you can try this to check actual versions of stuff:

python -m OpenSSL.debug
@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Really annoying that requests would raise different exceptions depending on the openssl version present in cryptography.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

I wonder if there's a way to test this behavior outside of cheroot - using just requests and cryptography versions against some known SSL endpoint with an unknown CA.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Running with simple requests and cryptography 2.5, I see that 'requests' raises the expected requests.exceptions.SSLError:

cheroot master $ pip-run requests 'cryptography>=2.5' -- -c "import requests; requests.get('https://untrusted-root.badssl.com/')"                                                        
Collecting cryptography>=2.5
  Using cached https://files.pythonhosted.org/packages/d7/9e/12bb10fd009b0146935c169cc0e1e86221eacf8dc207990d54b783c47a7d/cryptography-2.5-cp34-abi3-macosx_10_6_intel.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting cffi!=1.11.3,>=1.8 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/0b/ba/32835c9965d8a0090723e1d0b47373365525c4bd08c807b5efdc9fecbc99/cffi-1.11.5-cp37-cp37m-macosx_10_9_x86_64.whl
Collecting six>=1.4.1 (from cryptography>=2.5)
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography>=2.5)
Installing collected packages: asn1crypto, pycparser, cffi, six, cryptography
cheroot 6.5.5.dev7+ge616c254.d20190204 requires backports.functools_lru_cache, which is not installed.
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.5 pycparser-2.19 six-1.12.0
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 849, in _validate_conn
    conn.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connection.py", line 356, in connect
    ssl_context=context)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 359, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 412, in wrap_socket
    session=session
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 853, in _create
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ssl.py", line 1117, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/adapters.py", line 445, in send
    timeout=timeout
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/urllib3/util/retry.py", line 398, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='untrusted-root.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/requests/adapters.py", line 511, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='untrusted-root.badssl.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1056)')))

So the problem seems to have something to do with cheroot.

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

Well, it's actually not requests itself, but underlying libraries behaving like this. I've integrated trustme to have our own fake CA and it's very easy to use + I'm pretty sure it works correctly.

OTOH because implementation differences and a lot of env factors (like I mentioned above) it's kinda hard to cover all cases. In our tests I saw for one "logical" case of wrong setup the underlying library would sometimes return an alert from TLS or SSL, for example, which have different codes because they are from different namespaces.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Yeah, I'm pretty sure my test above isn't accurately exercising the behavior. I'm not passing verify or cert to requests.

@jaraco

This comment has been minimized.

Copy link
Member Author

commented Feb 4, 2019

Well, I'm stumped. I can't tell what factors are leading to requests not wrapping the underlying error... to implicate requests, or what factors cheroot could possibly do to mitigate the error.

I did try trapping both requests.exceptions.SSLError and OpenSSL.SSL.Error thus:

diff --git a/cheroot/test/test_ssl.py b/cheroot/test/test_ssl.py
index 4708e23f..b628c36c 100644
--- a/cheroot/test/test_ssl.py
+++ b/cheroot/test/test_ssl.py
@@ -286,7 +286,11 @@ def test_tls_client_auth(
             assert resp.text == 'Hello world!'
             return
 
-        with pytest.raises(requests.exceptions.SSLError) as ssl_err:
+        expected = (
+            requests.exceptions.SSLError,
+            OpenSSL.SSL.Error,
+        )
+        with pytest.raises(expected) as ssl_err:
             make_https_request()
 
         err_text = ssl_err.value.args[0].reason.args[0].args[0]

But that didn't work as the follow-up assertions on err_text fail.

I need to punt on this error as I'm trying to investigate other issues. I suggest we either mark these tests as xfail or proceed with #174 (pin cryptography) until someone has time to investigate further (or the problem is discovered and fixed elsewhere).

@webknjaz Any preference on those two options ^?

@webknjaz

This comment has been minimized.

Copy link
Member

commented Feb 4, 2019

I'd go for xfail. Regarding requests wrapping an error, it actually puts the underlying error inside of its own exception and err_text = ssl_err.value.args[0].reason.args[0].args[0] thing tries to extract it back for comparison.

jaraco added a commit that referenced this issue Feb 4, 2019

@jaraco jaraco referenced this issue Feb 4, 2019
10 of 15 tasks complete

jaraco added a commit that referenced this issue Feb 4, 2019

@stale

This comment has been minimized.

Copy link

commented Apr 5, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Apr 5, 2019

@webknjaz webknjaz added the bug label Apr 6, 2019

@stale stale bot removed the stale label Apr 6, 2019

@webknjaz

This comment has been minimized.

Copy link
Member

commented Apr 8, 2019

It looks like urllib3 > 1.24.1 should fix this via urllib3/urllib3#1496. So we have to either wait or point to our test env to their git repo directly...

@webknjaz

This comment has been minimized.

Copy link
Member

commented Apr 21, 2019

urllib==1.24.2 has been released 4 days ago.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.