Resource leaks with threading.Thread

[POCEH]

BPO	40657
Nosy	@vstinner, @vadmium, @rahul-kumi
Files	thread.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-09-24.15:59:35.177>
created_at = <Date 2020-05-17.08:07:51.405>
labels = ['interpreter-core', 'invalid', '3.8', 'performance']
title = 'Resource leaks with threading.Thread'
updated_at = <Date 2021-09-24.15:59:35.177>
user = 'https://bugs.python.org/POCEH'

bugs.python.org fields:

activity = <Date 2021-09-24.15:59:35.177>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-09-24.15:59:35.177>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2020-05-17.08:07:51.405>
creator = 'POCEH'
dependencies = []
files = ['49162']
hgrepos = []
issue_num = 40657
keywords = []
message_count = 3.0
messages = ['369102', '369762', '402575']
nosy_count = 4.0
nosy_names = ['vstinner', 'martin.panter', 'rahul-kumi', 'POCEH']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'resource usage'
url = 'https://bugs.python.org/issue40657'
versions = ['Python 3.8']

[POCEH]

In my project I'm using intensively threading to transfer HTTP data - reading is in separate thread and the process run smoothly. However when I've try to optimize stopping of the process and attempt to use the daemon threads. The result was 'Can't start new thread' error under Linux. In order to investigate the issue I've create small test program and found more interesting problem: using non daemon threads leaks handles (under Windows/x64 Semaphore type), while under Linux looks like leaks some other resource, can't say exactly what. When using a daemon threads no leaks handles under Windows but under Linux, after long time running the problem arrives, the only one solution is to reboot of Linux/ARM64, restarting the Python process didn't help! Of course the problem could be race condition or some locking problem, didn't investigate further and in Python's core source.
Attached test programs reveals the problem very quickly, using Python 3.8.x interpreters. Also I've testing with lambdas and deletion and found that they do not influence the behavior.

[vadmium]

Perhaps this is the same as bpo-37788, introduced in 3.7.

[vstinner]

You must join thread using thread.join().

If I add .join() calls, threading.active_count() stays around 2, it doesn't grow.

It's not a bug in Python.