threading.local doesn't support cyclic garbage collecting

[pitrou]

BPO	3757
Nosy	@birkenfeld, @amauryfa, @pitrou
Files	threadlocal.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2010-08-09.22:51:49.109>
created_at = <Date 2008-09-02.12:14:04.621>
labels = ['extension-modules', 'type-bug', 'library']
title = "threading.local doesn't support cyclic garbage collecting"
updated_at = <Date 2010-08-09.22:51:49.077>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2010-08-09.22:51:49.077>
actor = 'pitrou'
assignee = 'none'
closed = True
closed_date = <Date 2010-08-09.22:51:49.109>
closer = 'pitrou'
components = ['Extension Modules', 'Library (Lib)']
creation = <Date 2008-09-02.12:14:04.621>
creator = 'pitrou'
dependencies = []
files = ['18364']
hgrepos = []
issue_num = 3757
keywords = ['patch']
message_count = 5.0
messages = ['72332', '112341', '112685', '112729', '113496']
nosy_count = 4.0
nosy_names = ['georg.brandl', 'amaury.forgeotdarc', 'pitrou', 'gps']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue3757'
versions = ['Python 3.1', 'Python 2.7', 'Python 3.2']

[pitrou]

tp_traverse and tp_clear on threading.local are defined, but the
Py_TPFLAGS_HAVE_GC flag is not set. As a result, cycles are not collected:

>>> import threading, weakref
>>> o = threading.local()
>>> class X(object): pass
... 
>>> x = X()
>>> x.o = o
>>> o.x = x
>>> wr = weakref.ref(x)
>>> del x, o
>>> import gc
>>> gc.collect()
0
>>> wr()
<__main__.X object at 0x9bb0dc4>

[birkenfeld]

Do you want to fix that?

[pitrou]

This is much more complicated than I thought. The threadstate dict holds a strong reference to each local dict, and therefore the GC will refuse to collect the local dict when there's a cycle.
(when there's no cycle, the local dict is cleared manually by the local object's tp_dealloc)

[pitrou]

This is a patch. It modifies the implementation to use intermediate dummy objects and various weakrefs. This allows to break reference cycles even when the thread state dict is still alive (because it isn't involved in the ref cycles anymore). This also has the benefit of fixing another wart, which is still present in the pure Python implementation, though (see "_threading_local keeps the local of the last stopped thread alive" in test_threading_local).

The pure Python implementation should probably be updated, or perhaps removed altogether (since the C version is always compiled anyway).

[pitrou]

Committed in r83918 (py3k), r83919 (3.1) and r83920 (2.7) after Benjamin ok'ed it on IRC.