Add support for Python 3.12

[cbrnr]

Maybe this already works, let's see.

[cbrnr]

It seems like setup-python doesn't have the final 3.12 release yet, so let's wait until this happens.

actions/setup-python#736

[cbrnr]

Python 3.12 should now be available. @skjerns could you restart the jobs please (I don't have permissions to do that myself)?

[skjerns]

perfect, thanks :)

[cbrnr]

Any chance for a new release so that people can use pyedflib with Python 3.12?

[skjerns]

sure :) https://github.com/holgern/pyedflib/releases/tag/v0.1.35

[skjerns]

Ah damn, it doesn't seemt to build for 3.12

https://github.com/holgern/pyedflib/actions/runs/6403993363/job/17383647330#step:5:4202

      Downloading numpy-1.26.0.tar.gz (15.6 MB)
         ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 15.6/15.6 MB 90.4 MB/s eta 0:00:00
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Installing backend dependencies: started
      Installing backend dependencies: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'error'
      error: subprocess-exited-with-error
  
      × Preparing metadata (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> [58 lines of output]
          + /opt/python/cp312-cp312/bin/python /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/vendored-meson/meson/meson.py setup /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-python-native-file.ini
          The Meson build system
          Version: 1.2.99
          Source dir: /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc
          Build dir: /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build
          Build type: native build
          Project name: NumPy
          Project version: 1.26.0
          C compiler for the host machine: cc (gcc 10.2.1 "cc (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)")
          C linker for the host machine: cc ld.bfd 2.35-5
          C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (GCC) 10.2.1 20210130 (Red Hat 10.2.1-11)")
          C++ linker for the host machine: c++ ld.bfd 2.35-5
          Cython compiler for the host machine: cython (cython 3.0.2)
          Host machine cpu family: x86
          Host machine cpu: i686
          Program python found: YES (/opt/python/cp312-cp312/bin/python)
          Found pkg-config: /usr/bin/pkg-config (0.27.1)
          Run-time dependency python found: YES 3.12
          Has header "Python.h" with dependency python-3.12: YES
          Compiler for C supports arguments -fno-strict-aliasing: YES
          Test features "SSE SSE2" : Supported
          Test features "SSE3" : Supported
          Test features "SSSE3" : Supported
          Test features "SSE41" : Supported
          Test features "POPCNT" : Supported
          Test features "SSE42" : Supported
          Test features "AVX" : Supported
          Test features "F16C" : Supported
          Test features "FMA3" : Supported
          Test features "AVX2" : Supported
          Test features "AVX512F" : Supported
          Test features "AVX512CD" : Supported
          Test features "AVX512_KNL" : Supported
          Test features "AVX512_KNM" : Supported
          Test features "AVX512_SKX" : Supported
          Test features "AVX512_CLX" : Supported
          Test features "AVX512_CNL" : Supported
          Test features "AVX512_ICL" : Supported
          Test features "AVX512_SPR" : Unsupported due to Arguments "-msse, -msse2, -msse3, -mssse3, -msse4.1, -mpopcnt, -msse4.2, -mavx, -mf16c, -mfma, -mavx2, -mno-mmx, -mavx512f, -mavx512cd, -mavx512vl, -mavx512bw, -mavx512dq, -mavx512vnni, -mavx512ifma, -mavx512vbmi, -mavx512vbmi2, -mavx512bitalg, -mavx512vpopcntdq, -mavx512fp16" are not supported
          Configuring npy_cpu_dispatch_config.h using configuration
          Message:
          CPU Optimization Options
            baseline:
              Requested : min
              Enabled   : SSE SSE2
            dispatch:
              Requested : max -xop -fma4
              Enabled   : SSE3 SSSE3 SSE41 POPCNT SSE42 AVX F16C FMA3 AVX2 AVX512F AVX512CD AVX512_KNL AVX512_KNM AVX512_SKX AVX512_CLX AVX512_CNL AVX512_ICL
  
          Library m found: YES
          Found CMake: /usr/local/bin/cmake (3.27.6)
          WARNING: CMake Toolchain: Failed to determine CMake compilers state
          Run-time dependency openblas found: NO (tried pkgconfig and cmake)
          Run-time dependency openblas found: NO (tried pkgconfig and cmake)
  
          ../../numpy/meson.build:207:4: ERROR: Problem encountered: No BLAS library detected! Install one, or use the `allow-noblas` build option (note, this may be up to 100x slower for some linear algebra operations).
  
          A full log can be found at /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-logs/meson-log.txt
          [end of output]
  
      note: This error originates from a subprocess, and is likely not a problem with pip.
    error: metadata-generation-failed
  
    × Encountered error while generating package metadata.
    ╰─> See above for output.
  
    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.
    error: subprocess-exited-with-error
    
    × pip subprocess to install build dependencies did not run successfully.
    │ exit code: 1
    ╰─> See above for output.
    
    note: This error originates from a subprocess, and is likely not a problem with pip.
    full command: /opt/python/cp312-cp312/bin/python /opt/_internal/cpython-3.12.0/lib/python3.12/site-packages/pip/__pip-runner__.py install --ignore-installed --no-user --prefix /tmp/pip-build-env-09389xue/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools oldest-supported-numpy 'cython<3.0'
    cwd: [inherit]
    Installing build dependencies: finished with status 'error'
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  

[cbrnr]

Why is it trying to build NumPy from source?

[skjerns]

I think for building the Cython extension with cimports numpy.
I'm sure there is/was a reason for that, but I don't know whether it's still necessary and could be done elsewise. It was setup like that before I joined the project and I never looked into it.

[cbrnr]

For some reason, it does not find a BLAS library:

https://github.com/holgern/pyedflib/actions/runs/6403993363/job/17383647330#step:5:4169

Why do the other build < 3.12 work then? Don't they need one (e.g. openblas)? Maybe explicitly installing openblas could solve the issue?

[skjerns]

Good question. Maybe not available on the Python 3.12 runner?

[cbrnr]

It should be the same ubuntu-latest image for all Python versions I think. Unless cibuildwheel is using their own container images for each Python version. It's worth a shot trying to install openblas just for Python 3.12 I think.

[cbrnr]

The container image for x86 is the same for all Python versions (manylinux2014_i686). Everything works on manylinux2014_x86_64, including Python 3.12. I can't read the detailed log (it says A full log can be found at /tmp/pip-install-j2f24fv3/numpy_1809761ae9dd45a5a79d83e1a855eadc/.mesonpy-9ii57wvt/build/meson-logs/meson-log.txt), because I don't know to access that location from the results website.

I see two options that require only little work:

1. Consider dropping the i686 architecture completely. Almost all major packages only build for x86_64.
2. Skip Python 3.12 for i686.

Figuring out why the build for 3.12 on i686 is failing will require more digging around. Personally, I'd go with option 1 (people requiring the package for i686 can always build from source).

[skjerns]

This seems to be related: numpy/numpy#24703

Even though I agree that x86 can probably be dropped, I'd rather leave it, who knows on which devices this is running. I'll try to explicitly add openblas to python>3.11, else Skipping i686 for >3.11 would be the best solution I'd say.

[skjerns]

finally it worked!

new release it out

[cbrnr]

Nice! Why though? 😀

[skjerns]

Nice! Why though? 😀

What do you mean?

[cbrnr]

I mean, why is it working now? Did you change anything?

[skjerns]

Ah, sorry, I meant: finally excluding x68 for py3.12 worked. Installing openblas did not solve it unfortunately.

I had to wiggle the settings for the wheels.yaml quite a bit until I found out a setting that had the intended effect.

this worked:

          CIBW_SKIP: "pp* cp312*i686"
          CIBW_PRERELEASE_PYTHONS: False