tempfile.TemporaryDirectory deleted after sleep in threads

[colasleguernic]

BPO	44105
Files	test_tmpdir.py mwe.py: with sleep instead of subprocess.run

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-10-15.07:15:03.422>
created_at = <Date 2021-05-10.21:39:01.273>
labels = ['3.7', '3.8', 'type-bug', 'library']
title = 'tempfile.TemporaryDirectory deleted after sleep in threads'
updated_at = <Date 2021-10-15.07:15:03.421>
user = 'https://bugs.python.org/colasleguernic'

bugs.python.org fields:

activity = <Date 2021-10-15.07:15:03.421>
actor = 'serhiy.storchaka'
assignee = 'none'
closed = True
closed_date = <Date 2021-10-15.07:15:03.422>
closer = 'serhiy.storchaka'
components = ['Library (Lib)']
creation = <Date 2021-05-10.21:39:01.273>
creator = 'colas.le.guernic'
dependencies = []
files = ['50035', '50036']
hgrepos = []
issue_num = 44105
keywords = []
message_count = 3.0
messages = ['393436', '393445', '393446']
nosy_count = 1.0
nosy_names = ['colas.le.guernic']
pr_nums = []
priority = 'normal'
resolution = 'out of date'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue44105'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8']

[colasleguernic]

Yet another weird bug when one creates sub-processes from threads.
I found several issues mentioning ThreadPoolExecutor and subprocess.run but I believe this one is different.

When creating a temporary directory with tempfile.TemporaryDirectory (with or without context manager) it is sometimes deleted by a call to subprocess.run when both occurs inside a task sent to a ThreadPoolExecutor created without a context manager (that last part is very odd).

Running the attached file on python 3.8.5 or 3.8.10 shows the issue only occurs if the executor is not created through a context manager.

The bug does not seem to occur on python 3.9.5 or 3.10.0b1.

Using a ProcessPoolExecutor seems to fix the issue too.

[colasleguernic]

Actually, it has nothing to do with subprocess.run, replacing that line with sleep(random()) exhibits the same behavior.

[colasleguernic]

I guess this will be a won't fix as python3.8 is now in security-fix only mode.
I was misled by https://www.python.org/downloads/ that still indicates a bugfix maintenance status.