test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks on x86 Gentoo Refleaks 3.x

[vstinner]

BPO	33984
Nosy	@vstinner
Superseder	bpo-33735: test_multiprocessing_spawn leaked [1, 2, 1] memory blocks on AMD64 Windows8.1 Refleaks 3.7

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-07-03.11:34:44.820>
created_at = <Date 2018-06-27.22:44:57.513>
labels = []
title = 'test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks on x86 Gentoo Refleaks 3.x'
updated_at = <Date 2018-07-03.11:40:06.902>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2018-07-03.11:40:06.902>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2018-07-03.11:34:44.820>
closer = 'vstinner'
components = []
creation = <Date 2018-06-27.22:44:57.513>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 33984
keywords = []
message_count = 4.0
messages = ['320636', '320656', '320965', '320967']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '33735'
type = None
url = 'https://bugs.python.org/issue33984'
versions = []

[vstinner]

http://buildbot.python.org/all/#/builders/1/builds/267

1:47:06 load avg: 4.77 [161/417/1] test_multiprocessing_forkserver failed (16 min 32 sec) -- running: test_support (56 sec 282 ms)
beginning 6 repetitions
123456
......
test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks, sum=4
(...)
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_forkserver' in verbose mode
(...)
OK (skipped=27)

[vstinner]

http://buildbot.python.org/all/#/builders/1/builds/267

Oh, this is: x86 Gentoo Refleaks 3.x.

--

x86 Gentoo Refleaks 3.7 also leaked:
http://buildbot.python.org/all/#/builders/114/builds/150

test_multiprocessing_forkserver leaked [1, 2, 1] memory blocks, sum=4
Re-running failed tests in verbose mode
Re-running test 'test_multiprocessing_forkserver' in verbose mode
test_multiprocessing_forkserver leaked [21, 2, 1] memory blocks, sum=24
test_multiprocessing_forkserver leaked [2, 0, 0] file descriptors, sum=2

[vstinner]

Oh. I have a very good news: this issue is a duplicate of bpo-33735 and I just identified and fixed bpo-33735!

[vstinner]

Hum. Sadly my commit 23401fb is not enough, the test still fails randomly:

pydev@stormageddon ~/cpython $ ./python -m test test_multiprocessing_forkserver -m test_imap_unordered -R 3:3 -F
Run tests sequentially
0:00:00 load avg: 5.79 [ 1] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
0:00:28 load avg: 6.98 [ 2] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
0:00:59 load avg: 7.69 [ 3] test_multiprocessing_forkserver -- test_multiprocessing_forkserver passed in 30 sec 717 ms
beginning 6 repetitions
123456
......
0:01:28 load avg: 8.63 [ 4] test_multiprocessing_forkserver
beginning 6 repetitions
123456
......
test_multiprocessing_forkserver leaked [2, 1, 1] memory blocks, sum=4
test_multiprocessing_forkserver failed

== Tests result: FAILURE ==

3 tests OK.

1 test failed:
test_multiprocessing_forkserver

Total duration: 1 min 55 sec
Tests result: FAILURE

The test failed when the system load was high (8.63).

I hope that in pratice, my change will be enough to get less random failures. The "Re-run in verbose mode" on the "idle" buildbot should make the bug disappear.