test_multiprocessing_spawn fails when ran with -Werror

[serhiy-storchaka]

BPO	21779
Nosy	@serhiy-storchaka, @applio
Superseder	bpo-25654: test_multiprocessing_spawn ResourceWarning with -Werror
Files	issue_21779_py34_and_py35.patch: Patch for 3.4 and default/3.5 (both)

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-11-19.17:58:27.154>
created_at = <Date 2014-06-16.15:35:04.055>
labels = ['type-bug', 'tests']
title = 'test_multiprocessing_spawn fails when ran with -Werror'
updated_at = <Date 2015-11-19.17:58:27.153>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2015-11-19.17:58:27.153>
actor = 'serhiy.storchaka'
assignee = 'sbt'
closed = True
closed_date = <Date 2015-11-19.17:58:27.154>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2014-06-16.15:35:04.055>
creator = 'serhiy.storchaka'
dependencies = []
files = ['38427']
hgrepos = []
issue_num = 21779
keywords = ['patch']
message_count = 6.0
messages = ['220732', '237452', '237793', '237797', '237802', '254916']
nosy_count = 5.0
nosy_names = ['jnoller', 'BreamoreBoy', 'sbt', 'serhiy.storchaka', 'davin']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '25654'
type = 'behavior'
url = 'https://bugs.python.org/issue21779'
versions = ['Python 3.4', 'Python 3.5']

[serhiy-storchaka]

$ ./python -Werror -m test.regrtest -v -m test_sys_exit test_multiprocessing_spawn
== CPython 3.5.0a0 (default:149cc6364180+, Jun 12 2014, 15:45:54) [GCC 4.6.3]
==   Linux-3.8.0-36-generic-i686-with-debian-wheezy-sid little-endian
==   hash algorithm: siphash24 32bit
==   /home/serhiy/py/cpython/build/test_python_9425
Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0, hash_randomization=1, isolated=0)
[1/1] test_multiprocessing_spawn
test_sys_exit (test.test_multiprocessing_spawn.WithProcessesTestSubclassingProcess) ... Exception ignored in: <_io.FileIO name='@test_9425_tmp' mode='wb'>
ResourceWarning: unclosed file <_io.TextIOWrapper name='@test_9425_tmp' mode='w' encoding='UTF-8'>
FAIL

======================================================================
FAIL: test_sys_exit (test.test_multiprocessing_spawn.WithProcessesTestSubclassingProcess)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/serhiy/py/cpython/Lib/test/_test_multiprocessing.py", line 483, in test_sys_exit
    self.assertEqual(f.read().rstrip(), str(reason))
AssertionError: "[1, 2, 3]\nException ignored in: <_io.Fi[123 chars]-8'>" != '[1, 2, 3]'
- [1, 2, 3]
?          -
+ [1, 2, 3]- Exception ignored in: <_io.FileIO name='/dev/null' mode='rb'>
- ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>

---

Ran 1 test in 1.247s

FAILED (failures=1)
test test_multiprocessing_spawn failed
1 test failed:
test_multiprocessing_spawn

[BreamoreBoy]

This can be reproduced on Windows 8.1.

[applio]

Attaching patch for both 3.4 and default/3.5 (single file works for both) which fixes the test to properly adjust for running under -Werror.

Whether -Werror is set or otherwise the equivalent of 'warnings.simplefilter("error", Warning)' has been set, any warnings will trigger messages being sent to stderr (that's good). This problematic test, in particular, reassigns sys.stderr to point at a file in an attempt to capture exit codes from a Process into that file (that's okay); but any warning messages sent to stderr may also end up being captured inside that same file depending upon the timings of the threads involved (that's bad). To support the established behavior inside multiprocessing for capturing non-int exit codes (see bpo-13854), the test has been made more robust to detect when it is being run with -Werror (or similar) and allow for extra messages potentially appearing on stderr, otherwise it will continue to use the existing test to validate the output seen on stderr (captured in that file).

In the suite of tests for multiprocessing, test_multiprocessing_spawn currently triggers a bunch of warnings due to differences in how spawn works versus fork -- one example of this common pattern:

test_abort (test.test_multiprocessing_spawn.WithProcessesTestBarrier) ... Exception ignored in: <_io.FileIO name='/dev/null' mode='rb' closefd=True>
ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='r' encoding='UTF-8'>

How to better handle the triggering of these warnings in the first place is left as a topic for consideration in the future.

This patch's updated test has been tested with and without "-Werror" against 3.4 and default/3.5 on OS X 10.10.

[serhiy-storchaka]

Wouldn't the test fail with -Werror::ResourceWarning or like?

[applio]

Yes, setting -Werror::ResourceWarning would indeed cause the test to fail again.

A couple of options we could potentially adopt:

1. Decide to ignore anything extra in the file anytime any kind of "error" filterwarning is present.
2. Decide to ignore anything extra in the file anytime "-Werror" is used because that is arguably the dominant, most common scenario where this issue comes up.
3. Decide that certain kinds of -Werror::SomeParticularWarning are worth paying attention to and others are not.
4. Decide that the ResourceWarning from _io itself needs to be properly dealt with.

Of those options, I do not like #1 and I think #3 is a slippery slope to step onto -- this patch pragmatically offers us #2 with the potential to pursue #4 (which is ultimately where I think we want this to go). I kind of like the thought of someone specifying -Werror::ResourceWarning and getting to see this test fail.

[serhiy-storchaka]

Proposed patch cures only the symptom, the failure of one rigid test. A warning still is emitted in applications that use multiprocessing.

Closed in favor of duplicate bpo-25654 with patches that are intended to fix the cause of warnings. In any case thank you for your contribution Davin.