Cannot compile for Python 3.10 and 3.11

[kr41]

Steps to reproduce:

$ python3.10 -m venv py310
$ py310/bin/pip install validr

Output:

Installing collected packages: validr
  Running setup.py install for validr ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for validr did not run successfully.
  │ exit code: 1
  ╰─> [1381 lines of output]
      VALIDR_SETUP_MODE=c
      running install
      /home/krat/Projects/py310/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.10
      creating build/lib.linux-x86_64-3.10/validr
      copying src/validr/exception.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/schema.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/validator.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/__init__.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/model.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/_validator_py.py -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/_exception_py.py -> build/lib.linux-x86_64-3.10/validr
      creating build/lib.linux-x86_64-3.10/validr/_vendor
      copying src/validr/_vendor/email_validator.py -> build/lib.linux-x86_64-3.10/validr/_vendor
      copying src/validr/_vendor/__init__.py -> build/lib.linux-x86_64-3.10/validr/_vendor
      copying src/validr/_vendor/durationpy.py -> build/lib.linux-x86_64-3.10/validr/_vendor
      copying src/validr/_vendor/fqdn.py -> build/lib.linux-x86_64-3.10/validr/_vendor
      running egg_info
      writing src/validr.egg-info/PKG-INFO
      writing dependency_links to src/validr.egg-info/dependency_links.txt
      writing requirements to src/validr.egg-info/requires.txt
      writing top-level names to src/validr.egg-info/top_level.txt
      reading manifest file 'src/validr.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file 'src/validr.egg-info/SOURCES.txt'
      copying src/validr/_exception_c.c -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/_exception_c.pyx -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/_validator_c.c -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/_validator_c.pyx -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/model.pyi -> build/lib.linux-x86_64-3.10/validr
      copying src/validr/schema.pyi -> build/lib.linux-x86_64-3.10/validr
      running build_ext
      building 'validr._exception_c' extension
      creating build/temp.linux-x86_64-3.10
      creating build/temp.linux-x86_64-3.10/src
      creating build/temp.linux-x86_64-3.10/src/validr
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/home/krat/Projects/py310/include -I/usr/include/python3.10 -c src/validr/_exception_c.c -o build/temp.linux-x86_64-3.10/src/validr/_exception_c.o
      src/validr/_exception_c.c: In function ‘__Pyx_call_return_trace_func’:
      src/validr/_exception_c.c:1075:15: error: ‘PyThreadState’ {aka ‘struct _ts’} has no member named ‘use_tracing’; did you mean ‘tracing’?
       1075 |       tstate->use_tracing = 0;
            |               ^~~~~~~~~~~
            |               tracing


      *** a lot of similar lines go here ***


      /usr/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

For Python 3.11 the output differs, but it's still a compilation error.

[guyskk]

It seems Cython not well support Python 3.10 yet, need wait Cython 3.0 release.

A workaround is use pure python mode:

VALIDR_SETUP_MODE=py pip install validr

[kr41]

I checked, Cython works well on Python 3.10

$ python3.10 -m venv env
$ source env/bin/activate
(env) $ pip install cython wheel
(env) $ VALIDR_SETUP_MODE=pyx python setup.py bdist_wheel
(env) $ ls dist/
validr-1.2.1-cp310-cp310-linux_x86_64.whl

The problem is in distribution itself. You uploaded on PyPI source distribution (tar.gz file) with pre-generated C code. And by default setup.py compiles that C files into binaries. It seems Cython generates different code for Python 3.10 and older versions, so that it works on older versions but it doesn't on 3.10+.

Consider to upload binary wheels for each platform/version instead. You can build them by CIBuildWheels tool. Here is how I do it on my project ValidX. And yes, I'm not a user of your library, I just run benchmarks against it. Good work, by the way :)

[guyskk]

Thank you for the information! You are correct.

I try build in macOS but failed, maybe Cython not support it well now.

docker run --platform linux/amd64 -ti python:3.10.7 /bin/bash
$ pip install cython
$ VALIDR_SETUP_MODE=pyx pip install validr
...
      In file included from /usr/local/include/python3.10/unicodeobject.h:1046,
                       from /usr/local/include/python3.10/Python.h:83,
                       from src/validr/_exception_c.c:16:
      /usr/local/include/python3.10/cpython/unicodeobject.h:446:26: note: declared here
        446 | static inline Py_ssize_t _PyUnicode_get_wstr_length(PyObject *op) {
            |                          ^~~~~~~~~~~~~~~~~~~~~~~~~~
      error: command '/usr/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> validr

[kr41]

If I build it from sources (without C files), it works. If I try to install it from PyPI (with C files), it doesn't. It seems Cython doesn't generate new code, when C files already exist.

[guyskk]

Fixed in #61, just released version 1.2.2.