JIT: Assertion failure for `WITHIN_STACK_BOUNDS()` in `optimize_uops`

[devdanzin]

Crash report

What happened?

It's possible to cause an abort in a patched JIT build with the code below. Removing some more code still aborts, but usually starts causing ignored exceptions or making the reproduction take longer (to the point of becoming probabilistic).

Here's the patch used, it might be possible to reduce it:

diff --git a/Include/internal/pycore_backoff.h b/Include/internal/pycore_backoff.h
index 7f60eb49508..fd80dedb27e 100644
--- a/Include/internal/pycore_backoff.h
+++ b/Include/internal/pycore_backoff.h
@@ -124,7 +124,7 @@ trigger_backoff_counter(void)
 // For example, 4095 does not work for the nqueens benchmark on pyperformance
 // as we always end up tracing the loop iteration's
 // exhaustion iteration. Which aborts our current tracer.
-#define JUMP_BACKWARD_INITIAL_VALUE 4000
+#define JUMP_BACKWARD_INITIAL_VALUE 63
 #define JUMP_BACKWARD_INITIAL_BACKOFF 6
 static inline _Py_BackoffCounter
 initial_jump_backoff_counter(void)
@@ -137,7 +137,7 @@ initial_jump_backoff_counter(void)
  * Must be larger than ADAPTIVE_COOLDOWN_VALUE,
  * otherwise when a side exit warms up we may construct
  * a new trace before the Tier 1 code has properly re-specialized. */
-#define SIDE_EXIT_INITIAL_VALUE 4000
+#define SIDE_EXIT_INITIAL_VALUE 63
 #define SIDE_EXIT_INITIAL_BACKOFF 6

 static inline _Py_BackoffCounter
diff --git a/Include/internal/pycore_optimizer.h b/Include/internal/pycore_optimizer.h
index e7177552cf6..0d76a5a3df0 100644
--- a/Include/internal/pycore_optimizer.h
+++ b/Include/internal/pycore_optimizer.h
@@ -86,7 +86,7 @@ PyAPI_FUNC(void) _Py_Executors_InvalidateCold(PyInterpreterState *interp);
 // Used as the threshold to trigger executor invalidation when
 // executor_creation_counter is greater than this value.
 // This value is arbitrary and was not optimized.
-#define JIT_CLEANUP_THRESHOLD 1000
+#define JIT_CLEANUP_THRESHOLD 10000

 int _Py_uop_analyze_and_optimize(
     PyFunctionObject *func,
@@ -118,7 +118,7 @@ static inline uint16_t uop_get_error_target(const _PyUOpInstruction *inst)
 }

 // Holds locals, stack, locals, stack ... co_consts (in that order)
-#define MAX_ABSTRACT_INTERP_SIZE 4096
+#define MAX_ABSTRACT_INTERP_SIZE 8192

 #define TY_ARENA_SIZE (UOP_MAX_TRACE_LENGTH * 5)

@@ -129,7 +129,7 @@ static inline uint16_t uop_get_error_target(const _PyUOpInstruction *inst)
 // progress (and inserting a new ENTER_EXECUTOR instruction). In practice, this
 // is the "maximum amount of polymorphism" that an isolated trace tree can
 // handle before rejoining the rest of the program.
-#define MAX_CHAIN_DEPTH 4
+#define MAX_CHAIN_DEPTH 16

 /* Symbols */
 /* See explanation in optimizer_symbols.c */
diff --git a/Python/optimizer.c b/Python/optimizer.c
index 9db894f0bf0..14cbf670dec 100644
--- a/Python/optimizer.c
+++ b/Python/optimizer.c
@@ -509,7 +509,7 @@ guard_ip_uop[MAX_UOP_ID + 1] = {


 #define CONFIDENCE_RANGE 1000
-#define CONFIDENCE_CUTOFF 333
+#define CONFIDENCE_CUTOFF 100

 #ifdef Py_DEBUG
 #define DPRINTF(level, ...) \

Here's the MRE for quick, reliable reproduction:

class WithGetItem:
    def __getitem__(self, item): return 5
with_getitem = WithGetItem()

class StatefulIndex:
    def __index__(self): return 0
stateful_index = StatefulIndex()

def f1():
    import sys
    import asyncio
    class MetaException1(type): pass
    class EvilException1(Exception, metaclass=MetaException1): pass
    class Meta: pass
    class EvilException2(Exception, metaclass=MetaException1): pass

    for i in range(1):
        x = i
        try:
            raise EvilException1()
        except EvilException1:
            pass

    def sync_corruptor(value):
        caller_frame = sys._getframe(1)
        caller_frame.f_locals['x'] = value

    async def evil_coro():
        x = 0
        sync_corruptor('corrupted_string')

    def recursive():
        for i in range(70):
            x = i
            try:
                raise EvilException1()
            except EvilException1:
                pass
        asyncio.run(evil_coro())
        res = with_getitem[stateful_index]
        recursive()

    try:
        recursive()
    except RecursionError:
        pass
    print("NOT REACHED")

for i in range(3):
    print(i)
    f1()

Here's the output and backtrace, it shows that the issue happens on the first f1 call, with the last print not being called:

0
python: Python/optimizer_cases.c.h:1117: int optimize_uops(PyFunctionObject *, _PyUOpInstruction *, int, int, _PyBloomFilter *): Assertion `WITHIN_STACK_BOUNDS()' failed.

Program received signal SIGABRT, Aborted.

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=6, no_tid=0) at ./nptl/pthread_kill.c:44
#1  __pthread_kill_internal (threadid=<optimized out>, signo=6) at ./nptl/pthread_kill.c:89
#2  __GI___pthread_kill (threadid=<optimized out>, signo=signo@entry=6) at ./nptl/pthread_kill.c:100
#3  0x00007ffff7c45e2e in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#4  0x00007ffff7c28888 in __GI_abort () at ./stdlib/abort.c:77
#5  0x00007ffff7c287f0 in __assert_fail_base (fmt=<optimized out>, assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at ./assert/assert.c:118
#6  0x00007ffff7c3c19f in __assert_fail (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, function=<optimized out>) at ./assert/assert.c:127
#7  0x0000555555934494 in optimize_uops (func=0x7ffff5aa2750, trace=0x7ffff6940000, trace_len=trace_len@entry=901, curr_stacklen=2, dependencies=0x555555d98cc0 <_PyRuntime+359664>)
    at Python/optimizer_cases.c.h:1117
#8  0x000055555592b0e2 in _Py_uop_analyze_and_optimize (func=0xcef3, buffer=0xcef3, length=6, length@entry=901, curr_stacklen=-137732115, dependencies=0x16)
    at Python/optimizer_analysis.c:534
#9  0x000055555592619a in uop_optimize (frame=0x7ffff6248e90, tstate=0x555555d988a8 <_PyRuntime+358616>, progress_needed=true, exec_ptr=<optimized out>) at Python/optimizer.c:1371
#10 _PyOptimizer_Optimize (frame=frame@entry=0x7ffff6248e90, tstate=tstate@entry=0x555555d988a8 <_PyRuntime+358616>) at Python/optimizer.c:164
#11 0x000055555586e4d5 in stop_tracing_and_jit (tstate=tstate@entry=0x555555d988a8 <_PyRuntime+358616>, frame=frame@entry=0x7ffff6248e90) at Python/ceval.c:1108
#12 0x0000555555865724 in _PyEval_EvalFrameDefault (tstate=tstate@entry=0x555555d988a8 <_PyRuntime+358616>, frame=<optimized out>, frame@entry=0x7ffff7fa7020, throwflag=throwflag@entry=0)
    at Python/generated_cases.c.h:11712
#13 0x000055555582d56b in _PyEval_EvalFrame (tstate=0x555555d988a8 <_PyRuntime+358616>, frame=0x7ffff7fa7020, throwflag=0) at ./Include/internal/pycore_ceval.h:121
#14 _PyEval_Vector (tstate=tstate@entry=0x555555d988a8 <_PyRuntime+358616>, func=func@entry=0x7ffff6bf6bd0, locals=locals@entry=0x7ffff6c05190, args=args@entry=0x0,
    argcount=argcount@entry=0, kwnames=kwnames@entry=0x0) at Python/ceval.c:2159

Output from running with PYTHON_LLTRACE=4:
1989_abort_lltrace.txt

Output from running with PYTHON_DEBUG=4:
1989_abort_opt_debug.txt

Found using lafleur.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.15.0a2+ (heads/main-dirty:dc9d2eea587, Nov 24 2025, 06:26:38) [Clang 21.1.2 (2ubuntu6)]

Linked PRs

• gh-141976: Protect against non-progressing specializations in tracing JIT #141989

[Fidget-Spinner]

In the future, could you please run PYTHON_LLTRACE together with PYTHON_OPT_DEBUG? It's hard to make sense of PYTHON_OPT_DEBUG if the execution of PYTHON_LLTRACE is not interleaved. Thank you!

[devdanzin]

Sorry, will do.

[Fidget-Spinner]

No need to apologise :). Thank you for all the great fuzzing work you do.

[Fidget-Spinner]

So this is a very interesting bug. It's in the trace recorder but manifests in the optimizer. What I think is happening is the following:

CALL
specializes immediately to
CALL_ALLOC_AND_ENTER_INIT
which immediately deopts due to DEOPT_IF(!_PyThreadState_HasStackSpace(tstate, code->co_framesize + _Py_InitCleanup.co_framesize)); or some other deopt.

So it's very possible for CALL_ALLOC_AND_ENTER_INIT to make no progress due to stack space checks which are dynamic!

@markshannon this is an example of a specialization that makes no progress but isn't buggy. In this case, the easiest fix instead of counters it to just inspect the adaptive counter i think. If it's non-zero, that indicates that a deopt happened.

[Fidget-Spinner]

Fix up at #141989

[markshannon]

I don't see how the behavior of the specializer matters. The generated trace should be correct regardless of what the specializer does.

[Fidget-Spinner]

I don't see how the behavior of the specializer matters. The generated trace should be correct regardless of what the specializer does.

No the problem is not the specializer. The problem is that the previous instruction is not actually what was actually executed. Please see the PR.

[markshannon]

IIUC, the instruction recorded was CALL_ALLOC_AND_ENTER_INIT, but the executed path was to deopt to CALL. This should be fine as the guard immediately after the call will fail at runtime, or be converted to an exit by the optimizer, which would be expecting execution in the __init__ function, not immediately after the call.

Are we missing a guard?

[Fidget-Spinner]

IIUC, the instruction recorded was CALL_ALLOC_AND_ENTER_INIT, but the executed path was to deopt to CALL.

Yes.

This should be fine as the guard immediately after the call will fail at runtime, or be converted to an exit by the optimizer

No the optimizer does not recognise that the sequence as invalid and tries to optimize across it, manifesting in the assertion failure.
We shouldn't fix this in the optimizer as there's nothing it's doing that is wrong there. The sequence of uops produced is just invalid due to the trace recorder.

[Fidget-Spinner]

Note that the assertion failure is in the optimizer, not in the interpreter. So it's the optimizer complaining about the trace.