test_multiprocessing_spawn: test_queue_feeder_donot_stop_onexc() failed on x86 Windows7 3.x

[vstinner]

BPO	38212
Nosy	@vstinner, @pablogsal, @miss-islington
PRs	bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout #16348 [3.8] bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348) #16350 [3.7] bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348) #16351

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2019-09-24.12:21:40.185>
created_at = <Date 2019-09-18.09:21:37.685>
labels = ['tests', '3.9']
title = 'test_multiprocessing_spawn: test_queue_feeder_donot_stop_onexc() failed on x86 Windows7 3.x'
updated_at = <Date 2019-09-24.12:21:40.185>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2019-09-24.12:21:40.185>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2019-09-24.12:21:40.185>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2019-09-18.09:21:37.685>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 38212
keywords = ['patch']
message_count = 4.0
messages = ['352712', '353072', '353073', '353074']
nosy_count = 3.0
nosy_names = ['vstinner', 'pablogsal', 'miss-islington']
pr_nums = ['16348', '16350', '16351']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38212'
versions = ['Python 3.9']

[vstinner]

x86 Windows7 3.x:
https://buildbot.python.org/all/#/builders/58/builds/3015

If I recall correctly, this buildbot is known to be slow.

It sounds like a race condition.

ERROR: test_queue_feeder_donot_stop_onexc (test.test_multiprocessing_spawn.WithProcessesTestQueue)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\test\_test_multiprocessing.py", line 1132, in test_queue_feeder_donot_stop_onexc
    self.assertTrue(q.get(timeout=1.0))
  File "D:\cygwin\home\db3l\buildarea\3.x.bolen-windows7\build\lib\multiprocessing\queues.py", line 108, in get
    raise Empty
_queue.Empty

Extract of the test:

    def test_queue_feeder_donot_stop_onexc(self):
        # bpo-30414: verify feeder handles exceptions correctly
        if self.TYPE != 'processes':
            self.skipTest('test not appropriate for {}'.format(self.TYPE))

        class NotSerializable(object):
            def __reduce__(self):
                raise AttributeError
        with test.support.captured_stderr():
            q = self.Queue()
            q.put(NotSerializable())
            q.put(True)
            # bpo-30595: use a timeout of 1 second for slow buildbots
            self.assertTrue(q.get(timeout=1.0))   # <===== FAIL HERE =====
            close_queue(q)

    ...

Hum, "bpo-30595: use a timeout of 1 second for slow buildbots" sounds like this test depends on time which is bad for reliable tests. Would it be possible to use a more reliable synchronization primitive?

If no solution is found, the workaround is to use a timeout of 60 seconds instead of just 1 second...

[vstinner]

New changeset 99799c7 by Victor Stinner in branch 'master':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
99799c7

[miss-islington]

New changeset c13a4d6 by Miss Islington (bot) in branch '3.7':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
c13a4d6

[miss-islington]

New changeset fea9ca1 by Miss Islington (bot) in branch '3.8':
bpo-38212: Increase MP test_queue_feeder_donot_stop_onexc() timeout (GH-16348)
fea9ca1