Assertion failure at Python/generated_cases.c.h:10059 `PyObject *_PyEval_EvalFrameDefault(PyThreadState *, _PyInterpreterFrame *, int): Assertion 'STACK_LEVEL() == 0' failed`

[YuanchengJiang]

Crash report

What happened?

import unittest
import sys
mon = sys.monitoring
def add_line(
    code: "types.CodeType",
    lineno: int,
) -> None:
    return mon.DISABLE
def enable():
    mon.use_tool_id(mon.COVERAGE_ID, "regrtest coverage")
    mon.register_callback(mon.COVERAGE_ID, mon.events.LINE, add_line)
    mon.set_events(mon.COVERAGE_ID, mon.events.LINE)
enable()

class CLanguage:
    def __init__(self, filename=None):
        self.filename = filename
    def __init__(self, clang, filename='file', limited_capi=False):
        self.limited_capi = limited_capi
    def parse(self, text):
        norm = dedent(text).strip()
def _make_clinic(*, filename=bytes(range(256)), limited_capi=' '):
    clang = CLanguage(filename)
    return c

class ClinicWholeFileTest(unittest.TestCase):
    def expect_failure(self, raw, errmsg, *, filename=None, lineno=None):
        self.clinic = _make_clinic(filename='test.c')
    def test_directive_output_cant_pop(self):
        raw = '\n            /*[clinic input]\n            output pop\n            [clinic start generated code]*/\n        '
        err = "Can't 'output pop', stack is empty"
        self.expect_failure(raw, err)

if __name__ == "__main__":
    unittest.main()

python: ../Python/generated_cases.c.h:10059: PyObject *_PyEval_EvalFrameDefault(PyThreadState *, _PyInterpreterFrame *, int): Assertion `STACK_LEVEL() == 0' failed

config: --with-pydebug --enable-experimental-jit=yes --with-address-sanitizer

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

No response

[YuanchengJiang]

one similar poc is:

import sys
import inspect
import unittest
mon = sys.monitoring

def add_line(
    code: b'\x00\xff\x80\x7f',
    lineno: int,
) -> "typing.Literal[sys.monitoring.DISABLE]":
    return mon.DISABLE
def enable():
    mon.use_tool_id(mon.COVERAGE_ID, "regrtest coverage")
    mon.register_callback(mon.COVERAGE_ID, mon.events.LINE, add_line)
    mon.set_events(mon.COVERAGE_ID, mon.events.LINE)
def disable():
    mon.free_tool_id(mon.COVERAGE_ID)
enable()

class TestSignatureDefinitions(unittest.TestCase):
    def test_thread_module_has_signatures(fusion):
        import _thread
        if hasattr(_thread, 'get_ident'):
            try:
                sig = inspect.signature(_thread.get_ident)
            except ValueError:
                self.assertEqual(len(sig.parameters), 0)
            self.assertTrue(callable(obj))

if __name__ == "__main__":
    unittest.main()

it outputs:

Stack overflow (depth = 11) at ../Python/generated_cases.c.h:8578

it seems to be related. if not, I can open up another issue

[devdanzin]

Nice find, thank you! Does it reproduce with PYTHON_JIT=0?

[YuanchengJiang]

it doesnt reproduce in PYTHON_JIT=0. i guess it would be related to JIT

[Fidget-Spinner]

I can reproduce the second one. Unfortunately it seems LOAD_FAST_BORROW is recorded twice in the trace. I have no clue why, it might be something to do with instrumentation/sys.monitoring, but again I don't know enough about that to know. @markshannon

[markshannon]

The tracer be abandoning the trace if it encounters an instrumented instruction.
And instrumentation should invalid the trace currently being recorded as well as all existing ones.

@Fidget-Spinner is this the case?

[Fidget-Spinner]

The tracer be abandoning the trace if it encounters an instrumented instruction. And instrumentation should invalid the trace currently being recorded as well as all existing ones.

@Fidget-Spinner is this the case?

Yes it's already doing that.

[markshannon]

We aren't abandoning the trace if we encounter an instrumented instruction.

https://github.com/python/cpython/blob/main/Python/bytecodes.c#L5608 should include
opcode >= MIN_INSTRUMENTED_OPCODE

[markshannon]

We also need to abandon the trace if an exception is raised, and before calling any monitoring callback functions on exceptional paths.