gdb `py-bt` is no more compatible with Python 3.13+

[cielavenir]

Bug report

Bug description:

gdb ./python
r
Ctrl-C
py-bt

(gdb) py-bt
Traceback (most recent call first):
  (unable to read python frame information)

3.14.0b1 is affected and 3.14.0a7 is not affected.

Also I bisected git and found 1f5682f. And reverting it onto 3.14.0 fixes py-bt:

(gdb) py-bt
Traceback (most recent call first):
  File "/home/user/devel/cpython/Lib/_pyrepl/unix_console.py", line 439, in wait
    or bool(self.pollob.poll(timeout))
  File "/home/user/devel/cpython/Lib/_pyrepl/reader.py", line 703, in handle1
    self.console.wait(100)
  File "/home/user/devel/cpython/Lib/_pyrepl/reader.py", line 748, in readline
    self.handle1()
  File "/home/user/devel/cpython/Lib/_pyrepl/readline.py", line 395, in multiline_input
    return reader.readline()
  File "/home/user/devel/cpython/Lib/_pyrepl/simple_interact.py", line 143, in run_multiline_interactive_console
    statement = multiline_input(more_lines, ps1, ps2)
  File "/home/user/devel/cpython/Lib/_pyrepl/main.py", line 58, in interactive_console
    run_multiline_interactive_console(console)

This can be confirmed by building https://github.com/cielavenir/cpython/commits/3.14.0-pybt .

However I don't know why "reenabling autovectorization" fixes py-bt.

There is a similar report #127147 but this issue happens without --enable-optimizations.

Tested on Debian 13, GCC 14

CPython versions tested on:

3.14

Operating systems tested on:

Linux

[cielavenir]

(let me repost in this thread)

on 3.13, the affected commit is 5646f6f and you can try https://github.com/cielavenir/cpython/tree/3.13.9-pybt

official version 3.13.3 is affected but 3.13.2 is not affected.

[mpage]

I was able to reproduce this, but it's not clear to me what we should do about it. py-bt is failing because gcc is able to optimize the interpreter loop more effectively, which results in frame being optimized out:

cpython/Tools/gdb/libpython.py

Lines 1802 to 1805 in db098a4

	frame = self._gdbframe.read_var('frame')
	frame = PyFramePtr(frame)
	if not frame.is_optimized_out():
	return frame

I don't think the current implementation of py-bt will work reliably when optimizations are enabled (we compile with -O3 when running ./configure without any additional arguments).

A couple of options come to mind:

1. Use stronger language recommending that the debug build be used. It's only recommended right now. py-bt works fine for me on debian 13/gcc 14 when Python is configured with --with-pydebug.
2. Modify py-bt so that it uses a different approach to get the top-most frame. I think we might be able to read the current thread state from TLS and then grab the top-most frame from the thread state. That shouldn't be susceptible to compiler optimizations.

[cielavenir]

The thing is, we want RelWithDebInfo flavor of build...

we might be able to read the current thread state from TLS

I already tried this but it did not work

    def get_pyop(self):
        try:
            frame = self._gdbframe.read_var('frame')
            frame = PyFramePtr(frame)
            if not frame.is_optimized_out():
                return frame
            tstate = self._gdbframe.read_var('tstate')
            if tstate is None:
                return None
            frame = PyFramePtr(tstate.dereference()["current_frame"])
            if frame and not frame.is_optimized_out():
                return frame
            return None
        except ValueError:
            raise # return None

[mpage]

I already tried this but it did not work

What you have is reading tstate from the stack frame of the interpreter loop (and will potentially be affected in the same way as reading frame). I was suggesting reading the thread state from TLS, since that should be unaffected by compiler optimizations. Something like the following appears to work for me:

    def get_pyop(self):
        try:
            frame = self._gdbframe.read_var('frame')
            frame = PyFramePtr(frame)
            if not frame.is_optimized_out():
                return frame
            gilstate = self._gdbframe.read_var('_Py_tss_gilstate')
            if gilstate is None:
                return None
            frame = PyFramePtr(gilstate.dereference()["current_frame"])
            if frame:
                return frame
            return None
        except ValueError:
            return None

_Py_tss_gilstate is a TLS variable that stores the Python thread state associated with the current OS thread:

cpython/Python/pystate.c

Lines 75 to 77 in eb89286

	/* The "bound" thread state used by PyGILState_Ensure(),
	also known as a "gilstate." */
	_Py_thread_local PyThreadState *_Py_tss_gilstate = NULL;

[cielavenir]

@mpage Thank you but my test failed, _Py_tss_gilstate seems py3.15 only?

edit: _Py_tss_tstate did not work either