pyOpenSSL 17.1.0 test failure on ARM

[mbakke]

=================================== FAILURES ===================================
______________________ TestRand.test_insufficient_memory _______________________

self = <tests.test_rand.TestRand object at 0xb588c130>

    def test_insufficient_memory(self):
        """
            `OpenSSL.rand.bytes` raises `MemoryError` if more bytes are requested
            than will fit in memory.
            """
        with pytest.raises(MemoryError):
>           rand.bytes(sys.maxsize)

tests/test_rand.py:39: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

num_bytes = 2147483647

    def bytes(num_bytes):
        """
        Get some random bytes from the PRNG as a string.
    
        This is a wrapper for the C function ``RAND_bytes``.
    
        :param num_bytes: The number of bytes to fetch.
    
        :return: A string of random bytes.
        """
        if not isinstance(num_bytes, _integer_types):
            raise TypeError("num_bytes must be an integer")
    
        if num_bytes < 0:
            raise ValueError("num_bytes must not be negative")
    
        result_buffer = _ffi.new("unsigned char[]", num_bytes)
        result_code = _lib.RAND_bytes(result_buffer, num_bytes)
        if result_code == -1:
            # TODO: No tests for this code path.  Triggering a RAND_bytes failure
            # might involve supplying a custom ENGINE?  That's hard.
            _raise_current_error()
    
>       return _ffi.buffer(result_buffer)[:]
E       OverflowError: byte string is too large

/gnu/store/np01vrm9rqycjmrfc3jk7z9kd3z80fjh-python-pyopenssl-17.1.0/lib/python3.5/site-packages/OpenSSL/rand.py:62: OverflowError
============================== 2 tests deselected ==============================
======= 1 failed, 488 passed, 14 skipped, 2 deselected in 570.96 seconds =======

Full log (with dependency versions) at: https://hydra.gnu.org/build/2166355/nixlog/2/raw

[alex]

Is this a 32-bit or 64-bit ARM? What's the value of sys.maxsize?

[mbakke]

It's 32-bit. Unfortunately I don't have shell access to this machine, but will try to obtain sys.maxsize later/tomorrow.

[alex]

Oh, hah, it's in the traceback num_bytes = 2147483647.

So... why the heck is _ffi.new succeeding with an allocation that large...

Can you test what the following script does:

import sys

import cffi


print(cffi.FFI().new("unsigned char[]", sys.maxsize))

[mbakke]

It failed on 32-bit x86 as well.

guix environment --system=i686-linux --ad-hoc python python-cffi -- python3 -c 'import sys;import cffi;print(cffi.FFI().new("unsigned char[]", sys.maxsize))'
<cdata 'unsigned char[]' owning 2147483647 bytes>

On amd64 it prints MemoryError and exits.

[alex]

so the allocation succeeds on 32-bit x86, but does it raise the OverflowError later there as well?

Given that I want to kill this entire module, I'm inclined to just say the test should accept OverflowError as well -- assuming that makes it pass on 32-bit x86.

Opinions @reaperhulk @Lukasa ?

[reaperhulk]

on 32-bit x86 you can get an OverflowError by allocating sys.maxsize + 1 (or we could just always allocate 2 ** 63 rather than using sys.maxsize at all). Catching OverflowError here seems appropriate given the purpose of the test.