test_multiprocessing_fork leaked [0, 2, 0] file descriptors on aarch64 RHEL8 Refleaks 3.7 buildbot

[vstinner]

BPO	39932
Nosy	@vstinner
PRs	[3.7] bpo-39932: Fix multiprocessing test_heap() #19690

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2020-04-23.22:20:36.526>
created_at = <Date 2020-03-11.11:52:10.275>
labels = ['3.7', 'tests']
title = 'test_multiprocessing_fork leaked [0, 2, 0] file descriptors on aarch64 RHEL8 Refleaks 3.7 buildbot'
updated_at = <Date 2020-04-23.22:20:36.526>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2020-04-23.22:20:36.526>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2020-04-23.22:20:36.526>
closer = 'vstinner'
components = ['Tests']
creation = <Date 2020-03-11.11:52:10.275>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 39932
keywords = ['patch']
message_count = 6.0
messages = ['363909', '363925', '364366', '367132', '367133', '367154']
nosy_count = 1.0
nosy_names = ['vstinner']
pr_nums = ['19690']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue39932'
versions = ['Python 3.7']

[vstinner]

aarch64 RHEL8 Refleaks 3.7:
https://buildbot.python.org/all/#/builders/620/builds/20

test_multiprocessing_fork leaked [0, 2, 0] file descriptors, sum=2

0:40:22 load avg: 0.93 Re-running failed tests in verbose mode
0:40:22 load avg: 0.93 Re-running test_multiprocessing_fork in verbose mode
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2

[vstinner]

Another example on PPC64LE RHEL8 Refleaks 3.7:
https://buildbot.python.org/all/#/builders/431/builds/29

Warning -- Dangling processes: {<ForkServerProcess(QueueManager-2036, stopped[1])>}
Warning -- Dangling processes: {<ForkServerProcess(QueueManager-2036, stopped[1])>}

test_multiprocessing_spawn leaked [0, 2, 0] file descriptors, sum=2

Re-running test_multiprocessing_forkserver in verbose mode

test_multiprocessing_forkserver leaked [2, 0, 0] file descriptors, sum=2

[vstinner]

Bug seen on Pytho 3.7 branch, on Fedora Stable, RHEL7 and RHEL8, on different architectures.

aarch64 RHEL7 Refleaks 3.7:
https://buildbot.python.org/all/#builders/598/builds/22

PPC64LE RHEL7 Refleaks 3.7:
https://buildbot.python.org/all/#builders/414/builds/32

PPC64LE Fedora Stable Refleaks 3.7:
https://buildbot.python.org/all/#/builders/388/builds/31

[vstinner]

On aarch64 RHEL8 Refleaks 3.7, I managed to reproduce the issue with a single test:

./python -m test test_multiprocessing_fork -R 3:3 -m test.test_multiprocessing_fork.WithProcessesTestHeap.test_heap -v

[vstinner]

The following command fails randomly:

$ ./python -m test test_multiprocessing_fork -R 3:3 -m test.test_multiprocessing_fork.WithProcessesTestHeap.test_heap -v

Run 1:

test_multiprocessing_fork leaked [73, 52, 33] references, sum=158
test_multiprocessing_fork leaked [31, 23, 14] memory blocks, sum=68
test_multiprocessing_fork leaked [2, 0, 0] file descriptors, sum=2
test_multiprocessing_fork failed

Run 2:

1 test OK.

Run 6:

test_multiprocessing_fork leaked [0, 2, 0] file descriptors, sum=2
test_multiprocessing_fork failed

[vstinner]

New changeset 857d573 by Victor Stinner in branch '3.7':
bpo-39932: Fix multiprocessing test_heap() (GH-19690)
857d573