TestOpenSSL.test_openssl_assert_error_on_stack is failing on Fedora rawhide

[tiran]

TestOpenSSL.test_openssl_assert_error_on_stack is failing on our build farm.

• platform: s390x (big endian)
• cryptography: 2.3.0
• cffi: 1.11.5
• python: 3.6.6
• openssl: 1.1.1-pre8

________________ TestOpenSSL.test_openssl_assert_error_on_stack ________________
self = <test_openssl.TestOpenSSL object at 0x3ffd08e7160>
    def test_openssl_assert_error_on_stack(self):
        b = Binding()
        b.lib.ERR_put_error(
            b.lib.ERR_LIB_EVP,
            b.lib.EVP_F_EVP_ENCRYPTFINAL_EX,
            b.lib.EVP_R_DATA_NOT_MULTIPLE_OF_BLOCK_LENGTH,
            b"",
            -1
        )
        with pytest.raises(InternalError) as exc_info:
            _openssl_assert(b.lib, False)
    
        error = exc_info.value.err_code[0]
>       assert error.code == 101183626
E       AssertionError: assert 337146240 == 101183626
E        +  where 337146240 = _OpenSSLErrorWithText(code=337146240, lib=20, func=391, reason=384, reason_text=b'error:14187180:SSL routines:ssl_do_config:bad value').code
tests/hazmat/bindings/test_openssl.py:98: AssertionError

The code value is plausible:

>>> (20 << 24) + (391 << 12) + 384
337146240

but the lib, func, and reason code are off. They have a different value in OpenSSL's header files.

# define ERR_LIB_EVP             6
# define EVP_F_EVP_ENCRYPTFINAL_EX                        127
# define EVP_R_DATA_NOT_MULTIPLE_OF_BLOCK_LENGTH          138

[alex]

huh, interesting; we don't have access to a big-endian box, is this something you'll be able to track down?

[tiran]

Could it be possible that there are some values left on the error stack? The numbers 20, 391, 384 correspond to ERR_LIB_SSL, SSL_F_SSL_DO_CONFIG, SSL_R_BAD_VALUE.

Build: https://koji.fedoraproject.org/koji/taskinfo?taskID=28794831, see build.log

[alex]

Possibly! It'd certainly be a bug in that case, but this test passes on all other platforms, so it's something specific to s390x

[tiran]

Big endian was a red herring. The s390x build just happened to get tested first. I can now reproduce the issue on all F29 / Rawhide architectures. The issue is caused by a combination of additional security hardening in Fedora and bad tests in cryptography. Fedora builds of OpenSSL use crypto policies to disable bad crypto. This causes the tests test_ssl_ctx_options and test_ssl_options to leave ERR_LIB_SSL, SSL_F_SSL_DO_CONFIG, SSL_R_BAD_VALUE on the error stack. The tests use low-level APIs and never check the error stack. The test test_openssl_assert_error_on_stack does not fail on its own:

.tox/py37/bin/pytest -vv 'tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack'
=============================================== test session starts ===============================================
platform linux -- Python 3.7.0, pytest-3.7.1, py-1.5.4, pluggy-0.7.1 -- /home/heimes/cryptography/.tox/py37/bin/python3.7
cachedir: .pytest_cache
OpenSSL: OpenSSL 1.1.1-pre8 (beta) FIPS 20 Jun 2018
rootdir: /home/heimes/cryptography, inifile: tox.ini
plugins: hypothesis-3.67.0
collected 1 item                                                                                                  

tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack PASSED               [100%]

============================================ 1 passed in 0.01 seconds =============================================

but it fails in combination with another test:

tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_ssl_ctx_options PASSED                             [ 50%]
tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack FAILED               [100%]

==================================================== FAILURES =====================================================
_________________________________ TestOpenSSL.test_openssl_assert_error_on_stack __________________________________

self = <test_openssl.TestOpenSSL object at 0x7f4cce291630>

    def test_openssl_assert_error_on_stack(self):
        b = Binding()
        b.lib.ERR_put_error(
            b.lib.ERR_LIB_EVP,
            b.lib.EVP_F_EVP_ENCRYPTFINAL_EX,
            b.lib.EVP_R_DATA_NOT_MULTIPLE_OF_BLOCK_LENGTH,
            b"",
            -1
        )
        with pytest.raises(InternalError) as exc_info:
            _openssl_assert(b.lib, False)
    
        error = exc_info.value.err_code[0]
>       assert error.code == 101183626
E       AssertionError: assert 337146240 == 101183626
E        +  where 337146240 = _OpenSSLErrorWithText(code=337146240, lib=20, func=391, reason=384, reason_text=b'error:14187180:SSL routines:ssl_do_config:bad value').code

tests/hazmat/bindings/test_openssl.py:98: AssertionError
======================================= 1 failed, 1 passed in 0.07 seconds ========================================

In fact the test case test_ssl_ctx_options leaves error code 337146240 on the stack:

    def test_ssl_ctx_options(self):
        # Test that we're properly handling 32-bit unsigned on all platforms.
        b = Binding()
        assert b.lib.SSL_OP_ALL > 0
        ctx = b.lib.SSL_CTX_new(b.lib.TLSv1_method())
        ctx = b.ffi.gc(ctx, b.lib.SSL_CTX_free)
        current_options = b.lib.SSL_CTX_get_options(ctx)
        resp = b.lib.SSL_CTX_set_options(ctx, b.lib.SSL_OP_ALL)
        expected_options = current_options | b.lib.SSL_OP_ALL
        assert resp == expected_options
        assert b.lib.SSL_CTX_get_options(ctx) == expected_options
>       assert b.lib.ERR_get_error() == 0
E       AssertionError: assert 337146240 == 0

[tiran]

On Fedora rawhide, SSL_CTX_new(TLSv1_method()) leaves an error on the error stack. I opened downstream bug https://bugzilla.redhat.com/show_bug.cgi?id=1615098.

[alex]

I suspect the fix for both of these is to switch from TLSv1_method to TLS_method

…

On Sun, Aug 12, 2018 at 4:24 AM Christian Heimes ***@***.***> wrote: Big endian was a red herring. The s390x build just happened to get tested first. I can now reproduce the issue on all F29 / Rawhide architectures. The issue is caused by a combination of additional security hardening in Fedora and bad tests in cryptography. Fedora builds of OpenSSL use crypto policies to disable bad crypto. This causes the tests test_ssl_ctx_options and test_ssl_options to leave ERR_LIB_SSL, SSL_F_SSL_DO_CONFIG, SSL_R_BAD_VALUE on the error stack. The tests use low-level APIs and never check the error stack. The test test_openssl_assert_error_on_stack does not fail on its own: .tox/py37/bin/pytest -vv 'tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack' =============================================== test session starts =============================================== platform linux -- Python 3.7.0, pytest-3.7.1, py-1.5.4, pluggy-0.7.1 -- /home/heimes/cryptography/.tox/py37/bin/python3.7 cachedir: .pytest_cache OpenSSL: OpenSSL 1.1.1-pre8 (beta) FIPS 20 Jun 2018 rootdir: /home/heimes/cryptography, inifile: tox.ini plugins: hypothesis-3.67.0 collected 1 item tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack PASSED [100%] ============================================ 1 passed in 0.01 seconds ============================================= but it fails in combination with another test: tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_ssl_ctx_options PASSED [ 50%] tests/hazmat/bindings/test_openssl.py::TestOpenSSL::test_openssl_assert_error_on_stack FAILED [100%] ==================================================== FAILURES ===================================================== _________________________________ TestOpenSSL.test_openssl_assert_error_on_stack __________________________________ self = <test_openssl.TestOpenSSL object at 0x7f4cce291630> def test_openssl_assert_error_on_stack(self): b = Binding() b.lib.ERR_put_error( b.lib.ERR_LIB_EVP, b.lib.EVP_F_EVP_ENCRYPTFINAL_EX, b.lib.EVP_R_DATA_NOT_MULTIPLE_OF_BLOCK_LENGTH, b"", -1 ) with pytest.raises(InternalError) as exc_info: _openssl_assert(b.lib, False) error = exc_info.value.err_code[0] > assert error.code == 101183626 E AssertionError: assert 337146240 == 101183626 E + where 337146240 = _OpenSSLErrorWithText(code=337146240, lib=20, func=391, reason=384, reason_text=b'error:14187180:SSL routines:ssl_do_config:bad value').code tests/hazmat/bindings/test_openssl.py:98: AssertionError ======================================= 1 failed, 1 passed in 0.07 seconds ======================================== In fact the test case test_ssl_ctx_options leaves error code 337146240 on the stack: def test_ssl_ctx_options(self): # Test that we're properly handling 32-bit unsigned on all platforms. b = Binding() assert b.lib.SSL_OP_ALL > 0 ctx = b.lib.SSL_CTX_new(b.lib.TLSv1_method()) ctx = b.ffi.gc(ctx, b.lib.SSL_CTX_free) current_options = b.lib.SSL_CTX_get_options(ctx) resp = b.lib.SSL_CTX_set_options(ctx, b.lib.SSL_OP_ALL) expected_options = current_options | b.lib.SSL_OP_ALL assert resp == expected_options assert b.lib.SSL_CTX_get_options(ctx) == expected_options > assert b.lib.ERR_get_error() == 0 E AssertionError: assert 337146240 == 0 — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#4380 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAADBGiE72_WVBtc8_txbXlGPN0x33r4ks5uP-YtgaJpZM4VsrCB> .

-- All that is necessary for evil to succeed is for good people to do nothing.