-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
threading.enumerate(): reentrant call during a GC collection hangs #88588
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
The threading.enumerate() code is simple: # Active thread administration
_active_limbo_lock = _allocate_lock()
_active = {} # maps thread id to Thread object
_limbo = {}
def enumerate():
with _active_limbo_lock:
return list(_active.values()) + list(_limbo.values()) values(), list() and list+list operations can call trigger a GC collection depending on the GC thresholds, created Python objects, etc. During a GC collection, a Python object destructor (del) can again call threading.enumerate(). Problem: _active_limbo_lock is not re-entrant lock and so the second call hangs. In practice, this issue was seen in OpenStack which uses eventlet, when a destructor uses the logging module. The logging module uses threading.current_thread(), but this function is monkey-patched by the eventlet module. Extract of the eventlet function: def current_thread():
...
found = [th for th in __patched_enumerate() if th.ident == g_id]
... https://github.com/eventlet/eventlet/blob/master/eventlet/green/threading.py Attached PR makes _active_limbo_lock re-entrant to avoid this issue. I'm not sure if it's ok or not, I would appreciate to get a review and your feedback ;-) Attached file triggers the threading.enumerate() hang on re-entrant call. |
There are different ways to fix this issue:
(A) The problem is that functions other than threading.enumerate() (B) This is a simple and reliable option. But gc.disable() is process-wide: it affects all Python threads, and so it might have surprising side effects. Some code might rely on the current exact GC behavior. I would prefer to not disable the GC temporarily. |
Ok, the deadlock on reentrant call to threading.enumerate() is now fixed in 3.9, 3.10 and main (future 3.11) branches. I close the issue. Thanks for the reviews, as usual :-) |
Oh, I reopen the issue for PR 26741. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: