test_multiprocessing_spawn altered the execution environment in AMD64 FreeBSD Non-Debug 3.x

[pablogsal]

BPO	38842
Nosy	@vstinner, @pablogsal

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-07-13.19:52:30.936>
created_at = <Date 2019-11-18.23:39:53.296>
labels = ['tests', '3.9']
title = 'test_multiprocessing_spawn altered the execution environment in AMD64 FreeBSD Non-Debug 3.x'
updated_at = <Date 2021-07-13.19:52:30.936>
user = 'https://github.com/pablogsal'

bugs.python.org fields:

activity = <Date 2021-07-13.19:52:30.936>
actor = 'pablogsal'
assignee = 'none'
closed = True
closed_date = <Date 2021-07-13.19:52:30.936>
closer = 'pablogsal'
components = ['Tests']
creation = <Date 2019-11-18.23:39:53.296>
creator = 'pablogsal'
dependencies = []
files = []
hgrepos = []
issue_num = 38842
keywords = []
message_count = 7.0
messages = ['356929', '356930', '356931', '356932', '356949', '356950', '356951']
nosy_count = 2.0
nosy_names = ['vstinner', 'pablogsal']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue38842'
versions = ['Python 3.9']

[pablogsal]

Ran 352 tests in 243.972s
OK (skipped=34)
/usr/home/buildbot/python/3.x.koobs-freebsd-9e36.nondebug/build/Lib/multiprocessing/resource_tracker.py:203: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/usr/home/buildbot/python/3.x.koobs-freebsd-9e36.nondebug/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: '/psm_5b8ca94b': [Errno 2] No such file or directory: '/psm_5b8ca94b'
  warnings.warn('resource_tracker: %r: %s' % (name, e))

1 test altered the execution environment:
    test_multiprocessing_spawn

It seems that the shared memory leaked is causing the test to fail. 

[pablogsal]

https://buildbot.python.org/all/#builders/368/builds/98

[pablogsal]

Another failure that seems related:

https://buildbot.python.org/all/#/builders/279/builds/136

Ran 168 tests in 154.704s
OK (skipped=3)
Warning -- multiprocessing.process._dangling was modified by test_concurrent_futures
Before: set()
After: {<weakref at 0x7f48459bffb0; to 'SpawnProcess' at 0x7f48459baf00>}

[pablogsal]

The last test may have a different explanation, though

[vstinner]

The build was marked as failed because of this warning:

https://buildbot.python.org/all/#/builders/368/builds/98

...
test_rlock (test.test_multiprocessing_spawn.WithProcessesTestLock) ... ok
test_enable_logging (test.test_multiprocessing_spawn.WithProcessesTestLogging) ... ok
test_level (test.test_multiprocessing_spawn.WithProcessesTestLogging) ... ok
test_rapid_restart (test.test_multiprocessing_spawn.WithProcessesTestManagerRestart) ... ok
Warning -- Dangling processes: {<SpawnProcess name='QueueManager-212' pid=13560 parent=12641 stopped exitcode=-SIGTERM>}

This warning is tracked as bpo-38447.

[vstinner]

AMD64 RHEL7 3.8 was marked as failed because of:
https://buildbot.python.org/all/#/builders/279/builds/136

test_max_workers_too_large (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... skipped 'Windows-only process limit'
test_no_stale_references (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 0.57s ok
test_ressources_gced_in_workers (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 1.78s ok
test_shutdown_race_issue12456 (test.test_concurrent_futures.ProcessPoolSpawnProcessPoolExecutorTest) ... 0.42s
Warning -- reap_children() reaped child process 11860
ok

The test_concurrent_futures reap_children() issue is tracked as bpo-38546.

[vstinner]

Pablo: Feel free to close this issue as duplicate, or leave it open to track the original "resource_tracker" warning.