Support for Python 3.12 (assistance required)

[Lalufu]

This is an incomplete attemt to support the upcoming Python 3.12. Fedora 39 will ship with this, and has already built a beta release in the tree

With the changes contained in this MR so far, the code compiles against 3.12, but crashes almost immediately upon module load.

The current problem, as far as I can determine, is that uwsgi_python_init calls Py_Initialize(), and later init_uwsgi_embedded_module() tries to call PyImport_AppendInittab(). As of 3.12, this is no longer allowed (see https://github.com/python/cpython/blob/97a6a418167f1c8bbb014fab813e440b88cf2221/Python/import.c#L1419)

[Lalufu]

This is the backtrace when loading the module:

(gdb) bt
#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x00007ffff7490893 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2  0x00007ffff743e8ee in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x00007ffff74268ff in __GI_abort () at abort.c:79
#4  0x00007ffff6d9cde2 in fatal_error_exit (status=-1) at /usr/src/debug/python3.12-3.12.0~b4-1.fc39.x86_64/Python/pylifecycle.c:2707
#5  fatal_error (fd=<optimized out>, header=header@entry=1, prefix=prefix@entry=0x7ffff6fd9ff0 <__func__.10.lto_priv.1> "PyImport_AppendInittab", 
    msg=msg@entry=0x7ffff6f283c0 "PyImport_AppendInittab() may not be called after Py_Initialize()", status=status@entry=-1)
    at /usr/src/debug/python3.12-3.12.0~b4-1.fc39.x86_64/Python/pylifecycle.c:2818
#6  0x00007ffff6d9d158 in _Py_FatalErrorFunc (func=0x7ffff6fd9ff0 <__func__.10.lto_priv.1> "PyImport_AppendInittab", 
    msg=0x7ffff6f283c0 "PyImport_AppendInittab() may not be called after Py_Initialize()")
    at /usr/src/debug/python3.12-3.12.0~b4-1.fc39.x86_64/Python/pylifecycle.c:2904
#7  0x00007ffff6cfda6c in PyImport_AppendInittab (name=name@entry=0x7ffff7d1312f "uwsgi", initfunc=initfunc@entry=0x7ffff7cfd680 <init_uwsgi3>)
    at /usr/src/debug/python3.12-3.12.0~b4-1.fc39.x86_64/Python/import.c:1467
#8  0x00007ffff7d04f54 in init_uwsgi_embedded_module () at plugins/python/python_plugin.c:661
#9  init_uwsgi_embedded_module () at plugins/python/python_plugin.c:641
#10 0x00007ffff7d06dc2 in uwsgi_python_preinit_apps () at plugins/python/python_plugin.c:1098
#11 0x00005555555dcc77 in uwsgi_start (v_argv=<optimized out>) at core/uwsgi.c:3102
#12 0x00005555555def8c in uwsgi_setup (argc=<optimized out>, argv=0x7fffffffe238, envp=<optimized out>) at core/uwsgi.c:2545
#13 0x0000555555581e0d in main (argc=<optimized out>, argv=<optimized out>, envp=<optimized out>) at core/uwsgi.c:2047
(gdb) 

[ahesford]

Can you just move the PyImport_AppendInittab to right before the Py_Initialize() call in uwsgi_python_init?

[Lalufu]

I'm honestly not sure, the two calls are in different functions that are called at different stages of module initialization. I don't know enough about the internals of uwsgi to determine if this would be safe or not.

[ahesford]

I did a quick test and it seems to work, but a) I don't use uwsgi, and b) this project declares itself in "maintenance" mode.

[Lalufu]

Thanks for the hint. I've tested this with what projects I have that use python, and it doesn't seem to blow up in an obvious way so far.

[Lalufu]

That should work, too. I wasn't sure what style guide, if any, uwsgi follows, and tried to change as little as possible.

[xrmx]

Please also add 3.12 to the ci:

diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index 739bed40..009d4843 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -12,7 +12,7 @@ jobs:
     runs-on: ubuntu-20.04
     strategy:
       matrix:
-        python-version: ["2.7", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11"]
+        python-version: ["2.7", "3.6", "3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]
         test-suite: [unittest, python, deadlocks]
     steps:
     - name: Add deadnakes ppa
@@ -24,7 +24,7 @@ jobs:
           libpcre3-dev libjansson-dev libcap2-dev \
           curl check
     - name: Install distutils
-      if: contains(fromJson('["3.6","3.7","3.8","3.9","3.10","3.11"]'), matrix.python-version)
+      if: contains(fromJson('["3.6","3.7","3.8","3.9","3.10","3.11","3.12"]'), matrix.python-version)
       run: |
         sudo apt install --no-install-recommends -qqyf python${{ matrix.python-version }}-distutils \
     - uses: actions/checkout@v2

[Lalufu]

From just looking at the diff, I'm not sure this is going to fly. Python 3.12 has finally deprecated and removed `distutils`, so I'm not sure there is an appropriate package to be installed? Not familiar with ubuntu enough and their packaging.

[xrmx]

Thanks!

[frosty24]

Won't we need to add 3.12 in trove classifiers as well like following PR:
#2485