test_multiprocessing_fork failures

[skrah]

BPO	19432
Nosy	@skrah

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2013-10-29.13:33:19.002>
created_at = <Date 2013-10-29.12:00:44.074>
labels = ['extension-modules', 'type-bug']
title = 'test_multiprocessing_fork failures'
updated_at = <Date 2013-10-29.13:33:19.000>
user = 'https://github.com/skrah'

bugs.python.org fields:

activity = <Date 2013-10-29.13:33:19.000>
actor = 'skrah'
assignee = 'none'
closed = True
closed_date = <Date 2013-10-29.13:33:19.002>
closer = 'skrah'
components = ['Extension Modules']
creation = <Date 2013-10-29.12:00:44.074>
creator = 'skrah'
dependencies = []
files = []
hgrepos = []
issue_num = 19432
keywords = []
message_count = 4.0
messages = ['201624', '201630', '201632', '201634']
nosy_count = 2.0
nosy_names = ['skrah', 'sbt']
pr_nums = []
priority = 'normal'
resolution = 'works for me'
stage = 'needs patch'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue19432'
versions = ['Python 3.3', 'Python 3.4']

[skrah]

I'm getting the following test_multiprocessing_fork failures on Debian
Wheezy. Perhaps this is related to bpo-19227:

test_default_timeout (test.test_multiprocessing_fork.WithThreadsTestBarrier) ... Exception in thread Thread-344:
Traceback (most recent call last):
  File "/home/stefan/hg/master3/Lib/threading.py", line 921, in _bootstrap_inner
    self.run()
  File "/home/stefan/hg/master3/Lib/threading.py", line 869, in run
    self._target(*self._args, **self._kwargs)
  File "/home/stefan/hg/master3/Lib/test/_test_multiprocessing.py", line 1152, in task
    self.f(*self.args)
  File "/home/stefan/hg/master3/Lib/test/_test_multiprocessing.py", line 1382, in _test_default_timeout_f
    barrier.wait()
  File "/home/stefan/hg/master3/Lib/threading.py", line 616, in wait
    self._wait(timeout)
  File "/home/stefan/hg/master3/Lib/threading.py", line 653, in _wait
    self._break()
  File "/home/stefan/hg/master3/Lib/threading.py", line 702, in _break
    self._cond.notify_all()
  File "/home/stefan/hg/master3/Lib/threading.py", line 359, in notify_all
    self.notify(len(self._waiters))
  File "/home/stefan/hg/master3/Lib/threading.py", line 342, in notify
    waiters_to_notify = _deque(_islice(all_waiters, n))
RuntimeError: deque mutated during iteration

FAIL

======================================================================
FAIL: test_default_timeout (test.test_multiprocessing_fork.WithThreadsTestBarrier)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/stefan/hg/master3/Lib/test/_test_multiprocessing.py", line 1393, in test_default_timeout
    self.assertEqual(len(results), barrier.parties)
AssertionError: 4 != 5

---

[skrah]

The issue is probably different from bpo-19227, since it already occurs
with 4e79c3ae8a12.

[sbt]

This is a test of threading.Barrier rather than anything implemented directly by multiprocessing.

Tests which involve timeouts tend to be a bit flaky. Increasing the length of timeouts usually helps, but makes the tests take even longer.

How often have you seen this failure? Did it happen on a buildbot? Was there a lot of other activity on the system at the time?

[skrah]

I've seen the failure very often, without any particular load.

Now I noticed that during test_tools there were several ksoftirqd
processes consuming 20% CPU on 4 cores. That seemed unusual to
me, so I followed the advice of a certain MIT koan and rebooted.

test_multiprocessing_fork no longer fails.