test_buffer altered the execution environment

[serhiy-storchaka]

BPO	31792
Nosy	@vstinner, @ezio-melotti, @voidspace, @skrah, @serhiy-storchaka
PRs	bpo-31792: Restore os.environ in test_buffer when import numpy. #4007 [3.6] bpo-31792: Restore os.environ in test_buffer when import numpy. (GH-4007) #4009

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-10-16.08:22:33.422>
created_at = <Date 2017-10-15.09:35:57.853>
labels = ['3.7', 'tests', 'performance']
title = 'test_buffer altered the execution environment'
updated_at = <Date 2017-10-16.08:22:33.369>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2017-10-16.08:22:33.369>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2017-10-16.08:22:33.422>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2017-10-15.09:35:57.853>
creator = 'serhiy.storchaka'
dependencies = []
files = []
hgrepos = []
issue_num = 31792
keywords = ['patch']
message_count = 14.0
messages = ['304426', '304428', '304430', '304432', '304434', '304435', '304436', '304437', '304438', '304441', '304442', '304455', '304460', '304461']
nosy_count = 5.0
nosy_names = ['vstinner', 'ezio.melotti', 'michael.foord', 'skrah', 'serhiy.storchaka']
pr_nums = ['4007', '4009']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue31792'
versions = ['Python 3.6', 'Python 3.7']

[serhiy-storchaka]

test_buffer adds two environment variables: 'OPENBLAS_MAIN_FREE': '1', 'GOTOBLAS_MAIN_FREE': '1'.

$ ./python -m test -v test_buffer
...
Warning -- os.environ was modified by test_buffer
  Before: (140174267697792, environ({...})
  After:  (140174267697792, environ({..., 'OPENBLAS_MAIN_FREE': '1', 'GOTOBLAS_MAIN_FREE': '1'}) 
test_buffer failed (env changed)

1 test altered the execution environment:
test_buffer

Total duration: 8 sec
Tests result: SUCCESS

[vstinner]

Why the env changed wasn't noticed before? Is it a recent change?

[serhiy-storchaka]

I just have installed NumPy. It temporary sets these two environment variables for importing the multiarray numpy extension module, but forgot to remove them if the import is failed.

test_buffer imports numpy.

try:
with warnings.catch_warnings():
from numpy import ndarray as numpy_array
except ImportError:
numpy_array = None

[skrah]

So it is a numpy issue? Which numpy version is it?

[serhiy-storchaka]

It is particularly a NumPy issue (I have opened a bug numpy/numpy#9861). It is exposed because I have installed NumPy with non-debug build of CPython and run the test with debug build of CPython. The test is passes with non-debug build of CPython. I am wondering if we can make a workaround and if we should do this.

[skrah]

Thanks. If it only occurs when mixing debug/non-debug I don't think we have to do anything about it.

[vstinner]

I suggest to always save/restore env vars on numpy import, since we don't
control 3rd party code.

[skrah]

That's fine.

[skrah]

I can't figure out how to make the damn button green on GitHub,
so LGTM. :)

[serhiy-storchaka]

The issue is fixed on NumPy side (will be included in the next release). Is the workaround on our side still needed?