Adapt to exc_info changes in Py3.11

[scoder]

Py3.11 alpha has been changed to keep only a single exception value internally in sys.exc_info instead of a (type, value, traceback) tuple. There are a couple of places in Cython where we touch those fields and we should a) adapt and b) make sure we don't unnecessarily swap dead fields in Py3.11.

If this stays in CPython until the release, it's probably worth also backporting the necessary changes to 0.29.x.

First reported in #4499 (comment)

https://bugs.python.org/issue45711

[TeamSpen210]

There's some additional discussion here that indicates that (aside from two undocumented edge cases) the type and tb will always match the value, so Cython could actually just always discard the redundant pointers.

[scoder]

Yes, that's the idea.

[tacaswell]

The failures from this look like: (from installing cython itself)

    gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/tcaswell/.virtualenvs/bleeding/include -I/home/tcaswell/.pybuild/bleeding/include/python3.11 -c /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c -o build/temp.linux-x86_64-3.11/home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.o
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c: In function ‘__Pyx_PyErr_GetTopmostException’:
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23059:21: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23059 |     while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
          |                     ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23059:51: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23059 |     while ((exc_info->exc_type == NULL || exc_info->exc_type == Py_None) &&
          |                                                   ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c: In function ‘__Pyx__ExceptionSave’:
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23073:21: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23073 |     *type = exc_info->exc_type;
          |                     ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23075:19: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_traceback’
    23075 |     *tb = exc_info->exc_traceback;
          |                   ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c: In function ‘__Pyx__ExceptionReset’:
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23089:24: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23089 |     tmp_type = exc_info->exc_type;
          |                        ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23091:22: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_traceback’
    23091 |     tmp_tb = exc_info->exc_traceback;
          |                      ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23092:13: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23092 |     exc_info->exc_type = type;
          |             ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23094:13: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_traceback’
    23094 |     exc_info->exc_traceback = tb;
          |             ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c: In function ‘__Pyx__GetException’:
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23151:28: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23151 |         tmp_type = exc_info->exc_type;
          |                            ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23153:26: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_traceback’
    23153 |         tmp_tb = exc_info->exc_traceback;
          |                          ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23154:17: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_type’
    23154 |         exc_info->exc_type = local_type;
          |                 ^~
    /home/tcaswell/source/other_source/cython/Cython/Compiler/Scanning.c:23156:17: error: ‘_PyErr_StackItem’ {aka ‘struct _err_stackitem’} has no member named ‘exc_traceback’
    23156 |         exc_info->exc_traceback = local_tb;
          |                 ^~
    error: command '/lib/ccache/bin/gcc' failed with exit code 1

[YoSTEALTH]

Another victim to this bug: https://bugs.python.org/issue46428

Edit: was using Cython 3.0.0a10 btw

[da-woods]

With reference to the comment in the duplicate issue

I would rather have this fixed in the master branch first and wait with a release in 0.29.x until there is at least a beta release of CPython 3.11. I'd like to avoid churn in the legacy 0.29.x branch.

I suspect this may generate complaints - I think a good chunk of the performance tests needed for "have we made Python faster" depend on Cython 0.29.x, so there will be people chasing a compatible release

[hroncok]

The same concern applies to Fedora. We want to test our packages with Python 3.11 pre-releases, but can hardly do that without working Cython. We can backport patches from a draft pull request or whatnot, but backporting them directly from the master branch is usually quite tricky.

[scoder]

Let's get something clean into the master branch first. And then see what needs to be done to apply the same to 0.29.x, and what the impact is.