CALL_EX_PY and CALL_KW_BOUND_METHOD can bypass RecursionError after specialization

[marinelay]

Bug description

While verifying whether CPython's specialized opcodes preserve the behavior of their corresponding generic opcodes, I found an unexpected difference at the Python recursion limit.

The verification initially identified states in which CALL_EX_PY and CALL_KW_BOUND_METHOD did not refine their generic counterparts.
I then wrote a concrete Python reproducer and confirmed the difference on an actual CPython 3.15.0b4 build.

For the same Python call and arguments:

• the generic opcode raises RecursionError before entering the callee;
• the specialized opcode enters the callee and returns normally.

Reproducer

Save the following as repro.py:

import dis
import os
import sys

ARGS = ()
hits = 0
claimed = False


def ex_target():
    global hits
    hits += 1
    return 42


class Receiver:
    def target(self, *, value):
        global hits
        hits += 1
        return value


BOUND_TARGET = Receiver().target


def call_ex_probe(warm):
    global claimed

    while not warm:
        try:
            return call_ex_probe(False)
        except RecursionError:
            # Only the deepest active frame attempts the target call.
            # Outer frames propagate RecursionError instead of retrying
            # after one frame has unwound.
            if claimed:
                raise
            claimed = True
            warm = True

    # Warmup and recursion-boundary execution use this exact call site.
    return ex_target(*ARGS)


def call_kw_bound_method_probe(warm):
    global claimed

    while not warm:
        try:
            return call_kw_bound_method_probe(False)
        except RecursionError:
            if claimed:
                raise
            claimed = True
            warm = True

    return BOUND_TARGET(value=43)


case = os.environ["REPRO_CASE"]
mode = os.environ["REPRO_MODE"]

if case == "call-ex":
    probe = call_ex_probe
    interesting_opcodes = {"CALL_FUNCTION_EX", "CALL_EX_PY"}
else:
    probe = call_kw_bound_method_probe
    interesting_opcodes = {"CALL_KW", "CALL_KW_BOUND_METHOD"}

if mode == "specialized":
    for _ in range(100):
        probe(True)

opcodes = [
    instruction.opname
    for instruction in dis.get_instructions(probe, adaptive=True)
    if instruction.opname in interesting_opcodes
]

print("case:", case)
print("mode:", mode)
print("opcode:", opcodes)

hits_before = hits
sys.setrecursionlimit(200)

try:
    print("result:", probe(False))
except RecursionError:
    print("result: RecursionError")

print("target executions at boundary:", hits - hits_before)

Run each case in a fresh process:

$ REPRO_CASE=call-ex REPRO_MODE=control ./python repro.py
case: call-ex
mode: control
opcode: ['CALL_FUNCTION_EX']
result: RecursionError
target executions at boundary: 0

$ REPRO_CASE=call-ex REPRO_MODE=specialized ./python repro.py
case: call-ex
mode: specialized
opcode: ['CALL_EX_PY']
result: 42
target executions at boundary: 1

$ REPRO_CASE=call-kw-bound-method REPRO_MODE=control ./python repro.py
case: call-kw-bound-method
mode: control
opcode: ['CALL_KW']
result: RecursionError
target executions at boundary: 0

$ REPRO_CASE=call-kw-bound-method REPRO_MODE=specialized ./python repro.py
case: call-kw-bound-method
mode: specialized
opcode: ['CALL_KW_BOUND_METHOD']
result: 43
target executions at boundary: 1

The target functions and arguments are unchanged between control and specialized runs.
The only intentional difference is that the specialized run executes the relevant call site 100 times first, allowing CPython's normal adaptive specialization mechanism to replace the generic opcode.

Expected behavior

The generic and specialized opcodes should have the same observable behavior.

Given the current generic behavior, the specialized opcode should either deoptimize or raise RecursionError before executing the callee body.
Warming a call site should not change whether the target function is executed.

Suspected cause and source locations

AI assistance disclosure: The suspected-cause analysis and source-location summary in this section were prepared with assistance from OpenAI Codex using the GPT-5.6-sol model.
I independently ran and confirmed the runtime reproducer and checked the cited CPython source locations.

Generic inlined frame entry reaches start_frame, which calls
_Py_EnterRecursivePy():

• Python/bytecodes.c:6605-6609

_Py_EnterRecursivePy() decrements the recursion counter and invokes
_Py_CheckRecursiveCallPy() when the previous value was zero or less:

• Python/ceval_macros.h:406-408

The specialized paths instead end in _PUSH_FRAME. _PUSH_FRAME decrements
py_recursion_remaining, but does not perform the equivalent recursion check:

• Python/bytecodes.c:4661-4671

CPython defines _CHECK_RECURSION_REMAINING, which deoptimizes when the
remaining recursion budget is too low:

• Python/bytecodes.c:4644-4646

CALL_KW_PY includes this check before frame creation and _PUSH_FRAME:

• Python/bytecodes.c:5541-5549

However, CALL_KW_BOUND_METHOD does not include it:

• Python/bytecodes.c:5572-5581

Likewise, CALL_EX_PY proceeds from its callable guard directly to
_PY_FRAME_EX and _PUSH_FRAME, without _CHECK_RECURSION_REMAINING:

• Python/bytecodes.c:5793-5801

The generic CALL_FUNCTION_EX exact-Python-function path uses
DISPATCH_INLINED(new_frame), which subsequently reaches the checked
start_frame path:

• Python/bytecodes.c:5708-5730

The same two missing guards are still visible in CPython main at commit
36250a9b45cd898aa51c438cfb61fa1408266ffc:

• CALL_KW_BOUND_METHOD
• CALL_EX_PY

A possible fix may be to add _CHECK_RECURSION_REMAINING before the
corresponding _PY_FRAME_KW and _PY_FRAME_EX operations, but I have not
tested a patch and there may be stack/deoptimization ordering considerations.

Verification context

This was initially found while checking conditional contextual refinement between generic and specialized opcodes.

The formal counterexample occurs when py_recursion_remaining == 0:

generic:
    enters the checked start_frame path
    raises RecursionError

specialized:
    reaches unchecked _PUSH_FRAME
    executes the callee

The runtime reproducer above confirms the corresponding observable difference.

Environment

Python 3.15.0b4
CPython commit: 0a6fa6274a6c0ab38758302e85e1625920dbd2ad
Operating system: Ubuntu 24.04.4 LTS, Linux

I inspected the cited CPython main commit for the missing guards, but I have not yet executed the reproducer on a main-branch build.

CPython versions tested on:

3.15

Operating systems tested on:

Linux