Empty InternalErrors from latest OpenSSL 1.1.1

[hroncok]

In Fedora, the openssl package was updated with bugfixes form the 1.1.1 branch.

https://src.fedoraproject.org/rpms/openssl/c/5c7382cd79a3ba0832c4c77f875c0feb4ea1b13d?branch=master

cryptography started to fail with empty InternalErrors in tests/hazmat/primitives/test_aead.py:

E           InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([])
src/cryptography/hazmat/bindings/openssl/binding.py:78: InternalError

I'm trying to debug the problem with the Fedora's openssl maintainer, but we don't actually see the OpenSSL errors.

Here is a reproducer with Docker:

$ docker run -ti fedora:rawhide /bin/bash

# dnf install openssl-devel
...
Installed:
  openssl-devel-1:1.1.1b-8.fc31.x86_64     <--- verify it's at least 1:1.1.1b-6

# dnf install /usr/bin/git tox gcc
# git clone https://github.com/pyca/cryptography.git
# cd cryptography
# tox
...
InternalError: Unknown OpenSSL error...

Tox output:

• py27
• py35
• py36
• py37

Original Fedora report: https://bugzilla.redhat.com/show_bug.cgi?id=1708353

[alex]

Please include the complete stack trace, right now there isn't enough information to debug.

[hroncok]

How do i get it from tox?

[alex]

Simply copy the complete stdout/stderr from tox instead of truncating it.

[hroncok]

As said: Will attach full output of this reproducer in a few minutes. It is still running.

Here comes the partial log so far: toxlog.txt

[alex]

There are a lot of failing tests here -- I'm pretty confident these are OpenSSL bugs, not pyca/cryptography ones.

Maybe related to openssl/openssl#8636

[hroncok]

I have no idea how to debug the failures, especially since the errors are empty ([]).

[alex]

If I had to guess, I'd say it's the changes to crypto/evp/evp_enc.c in OpenSSL.

[hroncok]

As a side note, tests/hazmat/backends/test_openssl_memleak.py hangs on PyPy, so running tox for specific environments now instead.

[hroncok]

Tox output:

• py27
• py35
• py36
• py37

[reaperhulk]

This looks suspiciously like this issue: openssl/openssl#8636 (comment)

If that sample program fails fedora should immediately revert their patches.

[t8m]

It is. The problem is that this is upstream fix for another issue :( I am going to revert it for now however there is a more fundamental problem.

[hroncok]

The issue is now fixed and that fix is reverted. Thanks for help.

There is one subsequent problem, but I'll open a separate issue:

=================================== FAILURES ===================================
_________________ test_buffer_protocol_alternate_modes[mode5] __________________

mode = <cryptography.hazmat.primitives.ciphers.modes.XTS object at 0x7f6d75211690>
backend = <cryptography.hazmat.backends.openssl.backend.Backend object at 0x7f6d7efeb650>

    @pytest.mark.parametrize(
        "mode",
        [
            modes.CBC(bytearray(b"\x00" * 16)),
            modes.CTR(bytearray(b"\x00" * 16)),
            modes.OFB(bytearray(b"\x00" * 16)),
            modes.CFB(bytearray(b"\x00" * 16)),
            modes.CFB8(bytearray(b"\x00" * 16)),
            modes.XTS(bytearray(b"\x00" * 16)),
        ]
    )
    @pytest.mark.requires_backend_interface(interface=CipherBackend)
    def test_buffer_protocol_alternate_modes(mode, backend):
        data = bytearray(b"sixteen_byte_msg")
        cipher = base.Cipher(
            algorithms.AES(bytearray(b"\x00" * 32)), mode, backend
        )
>       enc = cipher.encryptor()

tests/hazmat/primitives/test_aes.py:495: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
src/cryptography/hazmat/primitives/ciphers/base.py:121: in encryptor
    self.algorithm, self.mode
src/cryptography/hazmat/backends/openssl/backend.py:295: in create_symmetric_encryption_ctx
    return _CipherContext(self, cipher, mode, _CipherContext._ENCRYPT)
src/cryptography/hazmat/backends/openssl/ciphers.py:116: in __init__
    self._backend.openssl_assert(res != 0)
src/cryptography/hazmat/backends/openssl/backend.py:125: in openssl_assert
    return binding._openssl_assert(self._lib, ok)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

lib = <module 'lib' (built-in)>, ok = False

    def _openssl_assert(lib, ok):
        if not ok:
            errors = _consume_errors(lib)
            errors_with_text = []
            for err in errors:
                buf = ffi.new("char[]", 256)
                lib.ERR_error_string_n(err.code, buf, len(buf))
                err_text_reason = ffi.string(buf)
    
                errors_with_text.append(
                    _OpenSSLErrorWithText(
                        err.code, err.lib, err.func, err.reason, err_text_reason
                    )
                )
    
            raise InternalError(
                "Unknown OpenSSL error. This error is commonly encountered when "
                "another library is not cleaning up the OpenSSL error stack. If "
                "you are using cryptography with another library that uses "
                "OpenSSL try disabling it before reporting a bug. Otherwise "
                "please file an issue at https://github.com/pyca/cryptography/"
                "issues with information on how to reproduce "
                "this. ({0!r})".format(errors_with_text),
>               errors_with_text
            )
E           InternalError: Unknown OpenSSL error. This error is commonly encountered when another library is not cleaning up the OpenSSL error stack. If you are using cryptography with another library that uses OpenSSL try disabling it before reporting a bug. Otherwise please file an issue at https://github.com/pyca/cryptography/issues with information on how to reproduce this. ([_OpenSSLErrorWithText(code=101617856L, lib=6, func=233, reason=192, reason_text='error:060E90C0:digital envelope routines:aesni_xts_init_key:xts duplicated keys')])

src/cryptography/hazmat/bindings/openssl/binding.py:78: InternalError
=========== 1 failed, 100943 passed, 6032 skipped in 323.44 seconds ============

@t8m says:

It should not try AES-XTS with both halves of the key being the same.

[hroncok]

#4885