pytest 3.3.0 breaks some of our tests

[mattsb42-aws]

Problem

With version 3.3.0, pytest behavior appears to have changed. It is now setting a PYTEST_CURRENT_TEST environment variable with the pytest name of each test. This is problematic if you are using the default pytest behavior of auto-generated names with for parametrized with binary values, as os.environ.set chokes on null bytes:

self = environ({'PATH': '/Users/bullocm/git/aws-encryption-sdk-cli/.tox/py36/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin...ncryption_sdk_cli', 'COV_CORE_CONFIG': '', 'COV_CORE_DATAFILE': '/Users/bullocm/git/aws-encryption-sdk-cli/.coverage'})
key = b'PYTEST_CURRENT_TEST'
value = b'test/unit/test_encoding.py::test_base64io_decode_with_whitespace[x\\x83H\\x91\\xb3\\xce\\xc2\\xb8\\xde\x12\\xef\x00\...FT82EpX9d36bYAs+lj\n+fmQZO1r5hVR4lMl4uCN\nT42b199G+ZyjK5pcP5Xh\nEJiAOpH/U10KO/FKg+UJ\nwVw5f3K8Eozh8g==-100] (teardown)'

    def __setitem__(self, key, value):
        key = self.encodekey(key)
        value = self.encodevalue(value)
>       self.putenv(key, value)
E       ValueError: embedded null byte

Solution

TBD

[mattsb42-aws]

Issue identified and raised with pytest pytest-dev/pytest#2957

[mattsb42-aws]

I think the best short-term solution will be to set pytest<3.3.0 in tox.ini. Once this bug is fixed, we can update to either require the new minimum version or blacklist 3.3.0.

[mattsb42-aws]

Also relevant to us on a related note: pytest-dev/pytest#2951

A lot of our tests have long parametrized values, some of which might exceed 32kB.

[mattsb42-aws]

Closing this for now. If pytest#2951 becomes an issue, we'll address that separately.