Deprecate thread debugging PYTHONTHREADDEBUG=1

[vstinner]

BPO	44584
Nosy	@vstinner, @miss-islington
PRs	bpo-44584: Deprecate PYTHONTHREADDEBUG env var #27065 [3.10] bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065) #27624

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2021-08-06.12:14:30.875>
created_at = <Date 2021-07-08.09:23:08.980>
labels = ['interpreter-core', '3.10', '3.11']
title = 'Deprecate thread debugging PYTHONTHREADDEBUG=1'
updated_at = <Date 2021-08-06.12:14:30.874>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-08-06.12:14:30.874>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2021-08-06.12:14:30.875>
closer = 'vstinner'
components = ['Interpreter Core']
creation = <Date 2021-07-08.09:23:08.980>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 44584
keywords = ['patch']
message_count = 5.0
messages = ['397130', '397131', '399071', '399074', '399077']
nosy_count = 2.0
nosy_names = ['vstinner', 'miss-islington']
pr_nums = ['27065', '27624']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue44584'
versions = ['Python 3.10', 'Python 3.11']

[vstinner]

Copy of my email to python-dev:
https://mail.python.org/archives/list/python-dev@python.org/thread/NMLGCDRUKLZSTK4UICJTKR54WRXU2ZGJ/

Hi,

Does anyone use threading debug PYTHONTHREADDEBUG=1 env var on a
Python debug build? If not, can I just remove it?

--

To fix a race condition at thread exit on Linux using the glibc, I
removed calls to pthread_exit() (PyThread_exit_thread()) in the
_thread module:

https://bugs.python.org/issue44434

A side effect of this change is the removal of the
"PyThread_exit_thread called" threading debug log when using
PYTHONTHREADDEBUG=1 environment variable.

I never used PYTHONTHREADDEBUG. I just tried it and it produces tons
of output in stdout about locks. It looks basically useless because it
produces way too many logs, and it pollutes stdout (ex: most Python
tests fail when it's enabled).

This debug mode requires to build Python in debug mode (./configure
--with-pydebug):

https://docs.python.org/dev/using/configure.html#python-debug-build

IMO there are enough external debugging tools to debug threading
issues. Python no longer has to come with its built-in logs.

I propose to deprecate the feature in Python 3.11 and remove it in 2
releases (Python 3.13).

Victor

[vstinner]

I never ever used PYTHONTHREADDEBUG=1. I just tried:
---

$ PYTHONTHREADDEBUG=1 ./python -m test -v test_sys
PyThread_init_thread called
PyThread_allocate_lock() -> 0x1f212c0
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f21300
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f21340
PyThread_allocate_lock called
PyThread_allocate_lock() -> 0x1f3ea70
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) called
PyThread_acquire_lock_timed(0x1f21340, -1, 0) -> 1
PyThread_release_lock(0x1f21340) called
(...)

======================================================================
FAIL: test_changing_sys_stderr_and_removing_reference (test.test_sys.SizeofTest)
----------------------------------------------------------------------

Traceback (most recent call last):
  File "/home/vstinner/python/main/Lib/test/test_sys.py", line 1525, in test_changing_sys_stderr_and_removing_reference
    self.assertEqual(out, b"")
     ^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: b'PyThread_init_thread called\nPyThread_all[59813 chars]ed\n' != b''
(...)
PyThread_release_lock(0x2113660) called
PyThread_free_lock(0x2113660) called
PyThread_free_lock(0x7fc6e8001300) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_free_lock(0x1f3ea70) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) called
PyThread_acquire_lock_timed(0x1f212c0, -1, 0) -> 1
PyThread_release_lock(0x1f212c0) called
PyThread_free_lock(0x1f212c0) called
PyThread_free_lock(0x1f21300) called
PyThread_free_lock(0x1f21340) called

---

This debug mode produces so many logs that it looks basically useless :-(

See also discussion at: #24241

[vstinner]

New changeset 4d77691 by Victor Stinner in branch 'main':
bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
4d77691

[miss-islington]

New changeset a11158e by Miss Islington (bot) in branch '3.10':
bpo-44584: Deprecate PYTHONTHREADDEBUG env var (GH-27065)
a11158e

[vstinner]

PYTHONTHREADDEBUG is now deprecated in Python 3.10, and planned for removal in 3.12.

If someone finds it useful, we can reconsider to keep the feature, but maybe redirect the output to stderr rather than stdout.