_result_handler dies on raised exceptions [multiprocessing]

[sindrig]

BPO	39190
Nosy	@sindrig
PRs	bpo-39190: Fix deadlock when callback raises #17795
Files	test_pool_error.py: Steps to reproduce

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-01-02.12:26:53.919>
labels = ['3.10', 'library', '3.9', 'type-crash', '3.11']
title = '_result_handler dies on raised exceptions [multiprocessing]'
updated_at = <Date 2021-09-19.22:01:28.549>
user = 'https://github.com/sindrig'

bugs.python.org fields:

activity = <Date 2021-09-19.22:01:28.549>
actor = 'iritkatriel'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-01-02.12:26:53.919>
creator = 'sindrig'
dependencies = []
files = ['48818']
hgrepos = []
issue_num = 39190
keywords = ['patch']
message_count = 1.0
messages = ['359194']
nosy_count = 1.0
nosy_names = ['sindrig']
pr_nums = ['17795']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'crash'
url = 'https://bugs.python.org/issue39190'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

Linked PRs

• gh-83371: handle exceptions from callbacks in process pools #131813
• gh-83371: route multiprocessing Pool callback failures to AsyncResult #131883

[sindrig]

Raising an Exception in a callback handler of apply_async and/or map_async will kill the _result_handler thread. This causes unexpected behavior as all subsequent callbacks won't be called and worse, pool.join will deadlock.

The documentation states that callbacks should return immediately but it does not warn the user against raising an exception.

Attached are steps to reproduce.