Memory leak with `threading.local` when tracing is active

[YuanchengJiang]

Bug report

Bug description:

import threading

events = []

def tracer(frame, event, arg):
    try:
        func_name = frame.f_code.co_name
    except Exception:
        func_name = 1.7976931348623157e308
    events.append((threading.current_thread().name, event, func_name))
    return tracer

def callback():
    for _ in range(3):
        try:
            next(callback.g)
        except StopIteration:
            break

callback.g = iter(range(3))

old_threading_trace = getattr(threading, "_trace_hook", Ellipsis)
try:
    threading.settrace(tracer)
    t = threading.Thread(target=callback, name="child-thread")
    t.start()
    t.join()
finally:
    threading.settrace(old_threading_trace)

=================================================================
==1247377==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x79ebcdce99c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x6021cf14b8fe in _PyObject_MallocWithType ../Include/internal/pycore_object_alloc.h:46
    #2 0x6021cf14b8fe in _PyType_AllocNoTrack ../Objects/typeobject.c:2505
    #3 0x6021cf14bb64 in PyType_GenericAlloc ../Objects/typeobject.c:2536
    #4 0x6021cf6a925f in create_localdummies ../Modules/_threadmodule.c:1617
    #5 0x6021cf6a925f in _ldict ../Modules/_threadmodule.c:1694
    #6 0x6021cf6a974b in local_setattro ../Modules/_threadmodule.c:1751
    #7 0x6021cf0c4421 in PyObject_SetAttr ../Objects/object.c:1476
    #8 0x6021cee5f3c8 in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:10750
    #9 0x6021cf324785 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:121
    #10 0x6021cf324785 in _PyEval_Vector ../Python/ceval.c:2005
    #11 0x6021cf516c34 in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #12 0x6021cf516c34 in call_trampoline ../Python/sysmodule.c:1080
    #13 0x6021cf516c34 in trace_trampoline ../Python/sysmodule.c:1116
    #14 0x6021cf419ff9 in _Py_call_instrumentation_line ../Python/instrumentation.c:1349
    #15 0x6021cee66880 in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:7024
    #16 0x6021cf324785 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:121
    #17 0x6021cf324785 in _PyEval_Vector ../Python/ceval.c:2005
    #18 0x6021cefa39bd in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #19 0x6021cefa39bd in PyObject_CallOneArg ../Objects/call.c:395
    #20 0x6021cf152065 in call_unbound_noarg ../Objects/typeobject.c:3041
    #21 0x6021cf152065 in slot_tp_finalize ../Objects/typeobject.c:10879
    #22 0x6021cf0c3e50 in PyObject_CallFinalizer ../Objects/object.c:585
    #23 0x6021cf0c3e50 in PyObject_CallFinalizerFromDealloc ../Objects/object.c:603
    #24 0x6021cf141200 in subtype_dealloc ../Objects/typeobject.c:2775
    #25 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #26 0x6021cf0721f0 in Py_DECREF ../Include/refcount.h:420
    #27 0x6021cf0721f0 in Py_XDECREF ../Include/refcount.h:513
    #28 0x6021cf0721f0 in dictkeys_decref ../Objects/dictobject.c:462
    #29 0x6021cf0721f0 in dictkeys_decref ../Objects/dictobject.c:446
    #30 0x6021cf0721f0 in dict_dealloc ../Objects/dictobject.c:3354
    #31 0x6021cf0721f0 in dict_dealloc ../Objects/dictobject.c:3328
    #32 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #33 0x6021cf08570b in Py_DECREF ../Include/refcount.h:420
    #34 0x6021cf08570b in _PyDict_Pop_KnownHash ../Objects/dictobject.c:3115
    #35 0x6021cf08570b in pop_lock_held ../Objects/dictobject.c:3149
    #36 0x6021cf08570b in PyDict_Pop ../Objects/dictobject.c:3157
    #37 0x6021cf6a2bdc in clear_locals ../Modules/_threadmodule.c:1860
    #38 0x6021cefa39bd in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #39 0x6021cefa39bd in PyObject_CallOneArg ../Objects/call.c:395
    #40 0x6021cf254d4f in handle_callback ../Objects/weakrefobject.c:989
    #41 0x6021cf254d4f in PyObject_ClearWeakRefs ../Objects/weakrefobject.c:1080
    #42 0x6021cf6a2ce4 in localdummy_dealloc ../Modules/_threadmodule.c:1421
    #43 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #44 0x6021cf4db134 in Py_DECREF ../Include/refcount.h:420
    #45 0x6021cf4db134 in PyThreadState_Clear ../Python/pystate.c:1673
    #46 0x6021cf6aa15a in thread_run ../Modules/_threadmodule.c:404
    #47 0x6021cf5260af in pythread_wrapper ../Python/thread_pthread.h:234
    #48 0x79ebcdc4aa41 in asan_thread_start ../../../../src/libsanitizer/asan/asan_interceptors.cpp:234
    #49 0x79ebcd98daa3  (/lib/x86_64-linux-gnu/libc.so.6+0x9caa3) (BuildId: 282c2c16e7b6600b0b22ea0c99010d2795752b5f)

Direct leak of 24 byte(s) in 1 object(s) allocated from:
    #0 0x79ebcdce99c7 in malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69
    #1 0x6021cf14b8fe in _PyObject_MallocWithType ../Include/internal/pycore_object_alloc.h:46
    #2 0x6021cf14b8fe in _PyType_AllocNoTrack ../Objects/typeobject.c:2505
    #3 0x6021cf14bb64 in PyType_GenericAlloc ../Objects/typeobject.c:2536
    #4 0x6021cf6a9210 in create_localdummies ../Modules/_threadmodule.c:1612
    #5 0x6021cf6a9210 in _ldict ../Modules/_threadmodule.c:1694
    #6 0x6021cf6a974b in local_setattro ../Modules/_threadmodule.c:1751
    #7 0x6021cf0c4421 in PyObject_SetAttr ../Objects/object.c:1476
    #8 0x6021cee5f3c8 in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:10750
    #9 0x6021cf324785 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:121
    #10 0x6021cf324785 in _PyEval_Vector ../Python/ceval.c:2005
    #11 0x6021cf516c34 in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #12 0x6021cf516c34 in call_trampoline ../Python/sysmodule.c:1080
    #13 0x6021cf516c34 in trace_trampoline ../Python/sysmodule.c:1116
    #14 0x6021cf419ff9 in _Py_call_instrumentation_line ../Python/instrumentation.c:1349
    #15 0x6021cee66880 in _PyEval_EvalFrameDefault ../Python/generated_cases.c.h:7024
    #16 0x6021cf324785 in _PyEval_EvalFrame ../Include/internal/pycore_ceval.h:121
    #17 0x6021cf324785 in _PyEval_Vector ../Python/ceval.c:2005
    #18 0x6021cefa39bd in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #19 0x6021cefa39bd in PyObject_CallOneArg ../Objects/call.c:395
    #20 0x6021cf152065 in call_unbound_noarg ../Objects/typeobject.c:3041
    #21 0x6021cf152065 in slot_tp_finalize ../Objects/typeobject.c:10879
    #22 0x6021cf0c3e50 in PyObject_CallFinalizer ../Objects/object.c:585
    #23 0x6021cf0c3e50 in PyObject_CallFinalizerFromDealloc ../Objects/object.c:603
    #24 0x6021cf141200 in subtype_dealloc ../Objects/typeobject.c:2775
    #25 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #26 0x6021cf0721f0 in Py_DECREF ../Include/refcount.h:420
    #27 0x6021cf0721f0 in Py_XDECREF ../Include/refcount.h:513
    #28 0x6021cf0721f0 in dictkeys_decref ../Objects/dictobject.c:462
    #29 0x6021cf0721f0 in dictkeys_decref ../Objects/dictobject.c:446
    #30 0x6021cf0721f0 in dict_dealloc ../Objects/dictobject.c:3354
    #31 0x6021cf0721f0 in dict_dealloc ../Objects/dictobject.c:3328
    #32 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #33 0x6021cf08570b in Py_DECREF ../Include/refcount.h:420
    #34 0x6021cf08570b in _PyDict_Pop_KnownHash ../Objects/dictobject.c:3115
    #35 0x6021cf08570b in pop_lock_held ../Objects/dictobject.c:3149
    #36 0x6021cf08570b in PyDict_Pop ../Objects/dictobject.c:3157
    #37 0x6021cf6a2bdc in clear_locals ../Modules/_threadmodule.c:1860
    #38 0x6021cefa39bd in _PyObject_VectorcallTstate ../Include/internal/pycore_call.h:169
    #39 0x6021cefa39bd in PyObject_CallOneArg ../Objects/call.c:395
    #40 0x6021cf254d4f in handle_callback ../Objects/weakrefobject.c:989
    #41 0x6021cf254d4f in PyObject_ClearWeakRefs ../Objects/weakrefobject.c:1080
    #42 0x6021cf6a2ce4 in localdummy_dealloc ../Modules/_threadmodule.c:1421
    #43 0x6021cf0c1ef1 in _Py_Dealloc ../Objects/object.c:3200
    #44 0x6021cf4db134 in Py_DECREF ../Include/refcount.h:420
    #45 0x6021cf4db134 in PyThreadState_Clear ../Python/pystate.c:1673
    #46 0x6021cf6aa15a in thread_run ../Modules/_threadmodule.c:404
    #47 0x6021cf5260af in pythread_wrapper ../Python/thread_pthread.h:234
    #48 0x79ebcdc4aa41 in asan_thread_start ../../../../src/libsanitizer/asan/asan_interceptors.cpp:234
    #49 0x79ebcd98daa3  (/lib/x86_64-linux-gnu/libc.so.6+0x9caa3) (BuildId: 282c2c16e7b6600b0b22ea0c99010d2795752b5f)

SUMMARY: AddressSanitizer: 48 byte(s) leaked in 2 allocation(s).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-140798: Fix memory leak with threading.local when finalizer resurrects threading_local_key #140836

[efimov-mikhail]

I've reworked a repro:

import threading

thread_local = threading.local()

class ClassWithDel:
    def __del__(self):
        pass

def tracer(frame, event, arg):
    thread_local.d = ClassWithDel()

threading.settrace(tracer)
t = threading.Thread()
t.start()

With this repro I have ASAN leak errors on 3.13, 3.14, 3.15.

I'll continue to investigate this.