Free-threaded importlib race recurses on lazy-submodule `__getattr__`

[SwayamInSync]

Bug report

Bug description:

Free-threaded CPython (cp3.14t), multiple threads first-touching a lazy submodule via the standard pattern

# pkg/__init__.py
def __getattr__(attr):
    if attr == "sub":
        import pkg.sub as sub
        return sub

recurse to RecursionError: maximum recursion depth exceeded.

Origin

First observed in numpy-quaddtype's free-threaded CI: numpy/numpy-quaddtype#88 (comment). The traceback recurses through numpy/__init__.py:745's lazy __getattr__ (line import numpy.rec as rec). The same shape reproduces with any stdlib package that uses the lazy-submodule __getattr__ pattern.

Cause

In Lib/importlib/_bootstrap.py:

• _load_unlocked (line 899) sets spec._initializing = False at line 930.
• _find_and_load_unlocked (line 1263) calls _load_unlocked, then at line 1313 does setattr(parent_module, child, module).

There is a window between line 930 and line 1313 where:

• sys.modules[name] is set,
• spec._initializing == False, but
• parent.__dict__[child] is missing.

The fast path in _find_and_load (line 1334) returns the module without taking the lock once it sees sys.modules[name] populated and _initializing == False. A second thread that enters this fast path inside the window then runs IMPORT_FROM 'sub' → getattr(parent, 'sub') → falls into the package's lazy __getattr__ → executes the same import pkg.sub as sub line → fast-paths again → same missing attribute → recurses to RecursionError.

Reproducer

# python3.14t repro.py
import os, sys, threading, tempfile, importlib._bootstrap as _b, traceback

pkg = tempfile.mkdtemp()
os.makedirs(os.path.join(pkg, "lazypkg", "sub"))
with open(os.path.join(pkg, "lazypkg", "__init__.py"), "w") as f:
    f.write(
        "def __getattr__(attr):\n"
        "    if attr == 'sub':\n"
        "        import lazypkg.sub as sub\n"
        "        return sub\n"
        "    raise AttributeError(attr)\n"
    )
with open(os.path.join(pkg, "lazypkg", "sub", "__init__.py"), "w") as f:
    f.write("X = 42\n")
sys.path.insert(0, pkg)

gate = threading.Event()
done = threading.Event()
_orig = _b._find_and_load_unlocked

def widen(name, import_):
    if name != "lazypkg.sub":
        return _orig(name, import_)
    parent = sys.modules["lazypkg"]
    spec = _b._find_spec(name, parent.__path__)
    parent.__spec__._uninitialized_submodules.append("sub")
    try:
        module = _b._load_unlocked(spec)
    finally:
        parent.__spec__._uninitialized_submodules.pop()
    gate.set()
    done.wait(timeout=5.0)
    setattr(parent, "sub", module)
    return module

_b._find_and_load_unlocked = widen
result = {}

def loader():
    import lazypkg
    lazypkg.sub

def observer():
    gate.wait()
    try:
        import lazypkg
        result["value"] = lazypkg.sub
    except BaseException as e:
        result["error"] = e
    done.set()

ta = threading.Thread(target=loader)
tb = threading.Thread(target=observer)
tb.start(); ta.start(); ta.join(); tb.join()

if "error" in result:
    traceback.print_exception(type(result["error"]), result["error"],
                              result["error"].__traceback__, limit=6)
    sys.exit(1)
print("OK"); sys.exit(0)

The shim widens the natural µs gap to seconds with one threading.Event. Everything else (the lazy __getattr__, IMPORT_FROM, attribute lookup) is plain upstream Python. Reproduces on both the free-threaded build and the GIL build.

Proposed fix

Keep spec._initializing == True until after the parent setattr:

• Remove spec._initializing = False from the success path of _load_unlocked (was at line 930).
• Clear it at the end of _find_and_load_unlocked, after setattr(parent_module, child, module) and _imp._set_lazy_attributes.
• For the two other callers of _load_unlocked (_load, the testing helper that doesn't attach to a parent; _builtin_from_name, for top-level builtins): they have no parent setattr, so wrap their call in try: ... finally: spec._initializing = False to preserve current behavior.

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Linked PRs

• gh-149728: Fix free-threaded race in importlib lazy-submodule fast path #149729

[StanFromIreland]

CC @JelleZijlstra

[ngoldbaum]

Also cc @kumaraditya303 because this affects NumPy