multiprocessing halts when child process crashes/quits

[JooEiras]

BPO	39752

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-02-25.17:24:16.048>
labels = ['extension-modules', 'type-bug', '3.8']
title = 'multiprocessing halts when child process crashes/quits'
updated_at = <Date 2020-02-25.17:24:16.048>
user = 'https://bugs.python.org/JooEiras'

bugs.python.org fields:

activity = <Date 2020-02-25.17:24:16.048>
actor = 'Jo\xc3\xa3o Eiras'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Extension Modules']
creation = <Date 2020-02-25.17:24:16.048>
creator = 'Jo\xc3\xa3o Eiras'
dependencies = []
files = []
hgrepos = []
issue_num = 39752
keywords = []
message_count = 1.0
messages = ['362651']
nosy_count = 1.0
nosy_names = ['Jo\xc3\xa3o Eiras']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue39752'
versions = ['Python 3.8']

[JooEiras]

Hi.

When one of the processes in a multiprocessing.pool picks up a task then then somehow crashes (and by crash I mean crashing the python process with something like a SEGV) or is killed, the pool in the main process will notice one of the workers died and will repopulate the pool, but it does not keep track which task was being handled by the process that died. As consequence, a caller waiting for a result will get stuck forever.

Example:
with multiprocessing.Pool(1) as pool:
result = pool.map_async(os._exit, [1]).get(timeout=2)

I found this because I was trying to use a lock with a spawned process on linux and that caused a crash and my program froze, but that is another issue.

[cwendling]

Looks like a duplicate of #66587, isn't it?