intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception

[applio]

BPO	23713
Nosy	@berkerpeksag, @serhiy-storchaka, @applio
Files	issue_23713_fix_fragile_imap_unordered_test_py35_and_py34.patch: Patch for default/3.5 and 3.4 (both). issue_23713_fix_fragile_imap_unordered_test_py27.patch: Patch for 2.7 branch.

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = <Date 2015-04-23.08:37:49.049>
created_at = <Date 2015-03-19.20:05:03.183>
labels = ['type-bug', 'tests']
title = 'intermittent failure of multiprocessing unit test test_imap_unordered_handle_iterable_exception'
updated_at = <Date 2015-04-23.08:37:49.049>
user = 'https://github.com/applio'

bugs.python.org fields:

activity = <Date 2015-04-23.08:37:49.049>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = True
closed_date = <Date 2015-04-23.08:37:49.049>
closer = 'serhiy.storchaka'
components = ['Tests']
creation = <Date 2015-03-19.20:05:03.183>
creator = 'davin'
dependencies = []
files = ['38567', '38568']
hgrepos = []
issue_num = 23713
keywords = ['patch']
message_count = 9.0
messages = ['238553', '238554', '238555', '239481', '240285', '241801', '241826', '241851', '241852']
nosy_count = 6.0
nosy_names = ['jnoller', 'python-dev', 'sbt', 'berker.peksag', 'serhiy.storchaka', 'davin']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue23713'
versions = ['Python 2.7', 'Python 3.4', 'Python 3.5']

[applio]

test_imap_unordered_handle_iterable_exception fails intermittently on some platforms due to the test being too rigid (fragile) about the order of the items being iterated over which, as the name would imply, is not guaranteed.

Of the multiprocessing module's test modules, test_multiprocessing_fork, test_multiprocessing_forkserver, and test_multiprocessing_spawn all leverage the unit test test_imap_unordered_handle_iterable_exception in multiple different contexts. There are thus multiple opportunities for test_imap_unordered_handle_iterable_exception to fail due to this fragility -- it's maybe a little bit surprising the test doesn't fail more often than it does.

A patch is forthcoming.

[applio]

Attaching patch for this fragile test for default/3.5 and 3.4 branches.

Blame for implementing the fragile test falls to @davin. In my own defense, @davin also discovered it in testing on FreeBSD 10.1 i386-RELEASE.

[applio]

Attaching patch for this fragile test for 2.7 branch.

Patches have been put through test.regrtest on OS X 10.10 (64-bit), FreeBSD 10.1 i386 (32-bit), and Raspbian Debian (ARMv6) thus far.

[serhiy-storchaka]

Why not use just

self.assertEqual(sorted(it), list(map(sqr, list(range(10)))))

or

self.assertCountEqual(list(it), list(map(sqr, list(range(10)))))

?

[applio]

Serhiy: If I understand correctly what you suggest, calling sorted(it) or list(it) would run the iterator all the way until it raises the SayWhenError exception, triggering the self.assertRaises before it ever actually gets to call the self.assertEqual or self.assertCountEqual you suggest. It would indeed test the raising of the exception but would never test individual values returned by the iterator to see that they belong.

Or were you suggesting something different?

[berkerpeksag]

Just saw this on x86 Tiger 3.x:

======================================================================
FAIL: test_imap_unordered_handle_iterable_exception (test.test_multiprocessing_fork.WithProcessesTestPool)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/Users/db3l/buildarea/3.x.bolen-tiger/build/Lib/test/_test_multiprocessing.py", line 1805, in test_imap_unordered_handle_iterable_exception
    self.assertEqual(next(it), i*i)
AssertionError: 1 != 0

http://buildbot.python.org/all/builders/x86%20Tiger%203.x/builds/9559/steps/test/logs/stdio

The patch looks good to me. I'm not familiar with the test code, but Serhiy's suggestion also looks good to me.

[applio]

Serhiy: If my comments on your questions make sense, should we go ahead with the patch as it is?

[serhiy-storchaka]

Thanks for your explanation and sorry for the delay Davin. Yes, it makes sense.

[python-dev]

New changeset 0ac30526c208 by Serhiy Storchaka in branch '2.7':
Issue bpo-23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0ac30526c208

New changeset 0eb5968c15ad by Serhiy Storchaka in branch '3.4':
Issue bpo-23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/0eb5968c15ad

New changeset f60f65507d8e by Serhiy Storchaka in branch 'default':
Issue bpo-23713: Fixed fragility of test_imap_unordered_handle_iterable_exception.
https://hg.python.org/cpython/rev/f60f65507d8e