Skip to content

JIT hang in xml.etree.iterparse #149481

@savannahostrowski

Description

@savannahostrowski

It looks like there's a JIT hang that's been blocking my nightly benchmark runs across all machines for the past two nights (see here and here), as well as @diegorusso's runs on his runner (see here and here).

These runs always hang at the bm_xml_etree.iterparse benchmark on JIT builds.

A minimum reproducer:

import io
import xml.etree.ElementTree as ET

data = b"<r>" + b"<a/>" * 1000 + b"</r>"

for _ in range(200):
    for event, elem in ET.iterparse(io.BytesIO(data)):
        pass

After bisecting, it looks like this was introduced by #148745. I'm not sure about the exact mechanism, but two observations that may or may not be related:

  • The iterator returned by xml.etree.iterparse has tp_iternext == slot_tp_iternext.
  • The new specialization in Python/optimizer_bytecodes.c excludes PyGen_Type but doesn't have a corresponding exclusion for the slot_tp_iternext case:
if (type != NULL && type != &PyGen_Type && type->tp_iternext != NULL) {
    ...
    ADD_OP(_ITER_NEXT_INLINE, 0, (uintptr_t)type->tp_iternext);
}

Empirically, also excluding the case where tp_iternext == slot_tp_iternext makes the hang go away in my local builds, but I don't know whether that's the real fix or just an incidental one that happens to side-step the trigger.

Would appreciate a closer look from someone who knows the trace semantics here.

cc: @markshannon @NekoAsakura

Metadata

Metadata

Assignees

No one assigned

    Labels

    3.15new features, bugs and security fixesinterpreter-core(Objects, Python, Grammar, and Parser dirs)topic-JITtype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions