concurrent futures Executors accept tasks after interpreter shutdown

[mrknmc]

BPO	33097
Nosy	@pitrou, @tomMoral, @mrknmc
PRs	bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit #6144 [3.7] bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144) #6445

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-04-10.18:38:59.591>
created_at = <Date 2018-03-18.16:41:13.115>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'concurrent futures Executors accept tasks after interpreter shutdown'
updated_at = <Date 2018-04-10.21:36:03.684>
user = 'https://github.com/mrknmc'

bugs.python.org fields:

activity = <Date 2018-04-10.21:36:03.684>
actor = 'mrknmc'
assignee = 'none'
closed = True
closed_date = <Date 2018-04-10.18:38:59.591>
closer = 'pitrou'
components = ['Library (Lib)']
creation = <Date 2018-03-18.16:41:13.115>
creator = 'mrknmc'
dependencies = []
files = []
hgrepos = []
issue_num = 33097
keywords = ['patch']
message_count = 6.0
messages = ['314044', '314313', '315174', '315175', '315176', '315182']
nosy_count = 3.0
nosy_names = ['pitrou', 'tomMoral', 'mrknmc']
pr_nums = ['6144', '6445']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue33097'
versions = ['Python 3.7', 'Python 3.8']

[mrknmc]

Currently, one can submit a task to an executor (both ThreadPoolExecutor and ProcessPoolExecutor) during interpreter shutdown. One way to do this is to register function fun with atexit as below.

@atexit.register
def fun():
   pool.submit(print, "apple")

The future is accepted and goes into PENDING state. However, this can cause issues if the _python_exit function (located in concurrent/futures/thread.py and/or concurrent/futures/process.py) executes before function fun.

Function _python_exit will shutdown the running workers in the pool and hence there will be no workers running by the time fun is executed so the future will be left in PENDING state forever.

The solution submitted here is to instead raise a RuntimeException when a task is submitted during interpreter shutdown. This is the same behaviour as when the shutdown method of an executor is called explicitly.

[pitrou]

Thanks for spotting this. I will take a look soon, unless someone beats me to it.

[pitrou]

New changeset c4b695f by Antoine Pitrou (Mark Nemec) in branch 'master':
bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144)
c4b695f

[pitrou]

New changeset b262659 by Antoine Pitrou (Miss Islington (bot)) in branch '3.7':
bpo-33097: Fix submit accepting callable after executor shutdown by interpreter exit (GH-6144) (GH-6445)
b262659

[pitrou]

Thanks your contribution Mark!

[mrknmc]

Happy to contribute! Thanks for the review :)