testEnviron (test.test_wsgiref.HandlerTests) fails when environment variable X is set

[hroncok]

BPO	37411
Nosy	@vstinner, @hroncok, @miss-islington, @tirkarthi
PRs	bpo-37411: Rewrite test_wsgiref.testEnviron() #14394 [3.8] bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394) #14402 [3.7] bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394) #14403 [2.7] bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394) #14404

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-06-26.20:55:23.757>
created_at = <Date 2019-06-26.12:05:51.551>
labels = ['3.8', '3.7', 'tests', '3.9']
title = 'testEnviron (test.test_wsgiref.HandlerTests) fails when environment variable X is set'
updated_at = <Date 2019-07-03.09:31:21.760>
user = 'https://github.com/hroncok'

bugs.python.org fields:

activity = <Date 2019-07-03.09:31:21.760>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-06-26.20:55:23.757>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-06-26.12:05:51.551>
creator = 'hroncok'
dependencies = []
files = []
hgrepos = []
issue_num = 37411
keywords = ['patch']
message_count = 7.0
messages = ['346605', '346615', '346649', '346670', '346671', '346679', '346680']
nosy_count = 4.0
nosy_names = ['vstinner', 'hroncok', 'miss-islington', 'xtreak']
pr_nums = ['14394', '14402', '14403', '14404']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue37411'
versions = ['Python 2.7', 'Python 3.7', 'Python 3.8', 'Python 3.9']

[hroncok]

In Fedora CI, we use the environment variable X to set tests to skip:

https://src.fedoraproject.org/tests/python/blob/bd3ec9505cd37d80fe47fbb8234928abcfc0c658/f/selftest/parallel.sh#_9 - lines 9 and 21

However, I'Ve realized that testEnviron (test.test_wsgiref.HandlerTests) fails.

Here's a reproducer:

$ python3.7 -m test test_wsgiref 
Run tests sequentially
0:00:00 load avg: 0.75 [1/1] test_wsgiref

== Tests result: SUCCESS ==

1 test OK.

Total duration: 77 ms
Tests result: SUCCESS

$ X=boom python3.7 -m test test_wsgiref 
Run tests sequentially
0:00:00 load avg: 0.71 [1/1] test_wsgiref
test test_wsgiref failed -- Traceback (most recent call last):
  File "/usr/lib64/python3.7/test/test_wsgiref.py", line 567, in testEnviron
    self.checkOSEnviron(h)
  File "/usr/lib64/python3.7/test/test_wsgiref.py", line 559, in checkOSEnviron
    self.assertEqual(env[k],v)
AssertionError: 'Y' != 'boom'
- Y
+ boom

test_wsgiref failed

== Tests result: FAILURE ==

1 test failed:
test_wsgiref

Total duration: 73 ms
Tests result: FAILURE

I believe that such tests should not be so easily fooled - it should be properly isolated or a less common variable name should be used if that is not possible, such as PYTHON_TEST_WSGIREF_TMP instead of X.

[vstinner]

I wrote PR 14394 to fix the test.

[vstinner]

New changeset 5150d32 by Victor Stinner in branch 'master':
bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394)
5150d32

[miss-islington]

New changeset 814c7ae by Miss Islington (bot) in branch '3.7':
bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394)
814c7ae

[miss-islington]

New changeset bdbd5e8 by Miss Islington (bot) in branch '3.8':
bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394)
bdbd5e8

[vstinner]

New changeset dfa9499 by Victor Stinner (Miss Islington (bot)) in branch '2.7':
[2.7] bpo-37411: Rewrite test_wsgiref.testEnviron() (GH-14394) (GH-14404)
dfa9499

[vstinner]

Ok, the bug is now fixed in 2.7, 3.7, 3.8 and master branches. Thanks Miro for the report. I also enhanced the test to test all variables of handler.environ.