Add support for Python 3.13 free-threaded build

[ngoldbaum]

I'm currently working on adding support for the Python 3.13 free-threaded build to projects in the scientific python ecosystem. We are tracking this work at https://github.com/Quansight-Labs/free-threaded-compatibility. Right now we're focusing on projects relatively low in the stack. scikit-learn isn't the lowest in the stack but it has a lot of tests that perform multithreaded workflows so running the scikit-learn tests is a productive way to elucidate threading bugs in scikit-learn and its dependencies.

Currently, scikit-learn builds fine and almost all the tests pass with the GIL disabled 🎉

The following tests have failures:

FAILED sklearn/ensemble/tests/test_voting.py::test_sample_weight[42] - AssertionError:
FAILED sklearn/model_selection/tests/test_search.py::test_random_search_cv_results_multimetric - AssertionError:
FAILED sklearn/semi_supervised/tests/test_self_training.py::test_classification[threshold-base_estimator1] - AssertionError:
FAILED sklearn/semi_supervised/tests/test_self_training.py::test_classification[k_best-base_estimator1] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_matrix-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[csr_array-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_matrix-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-linear-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-linear-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-poly-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-poly-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train0-y_train0-X_test0] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-rbf-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-sigmoid-X_train2-y_train2-X_test2] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_svc[lil_array-sigmoid-X_train3-y_train3-X_test3] - AssertionError:
FAILED sklearn/svm/tests/test_sparse.py::test_consistent_proba - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_libsvm_iris - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_svc_clone_with_callable_kernel - AssertionError:
FAILED sklearn/svm/tests/test_svm.py::test_consistent_proba - AssertionError:
FAILED sklearn/tests/test_build.py::test_openmp_parallelism_enabled - AssertionError:
FAILED sklearn/tests/test_pipeline.py::test_pipeline_memory - AssertionError:

Note that some of these failures are flaky and you may not get the same list if you run the scikit-learn tests. I also saw one deadlock in test_calibration.py::test_parallel_execution, so I'd also suggest running pytest with e.g. -o fauthandler_timeout=60 set so you will get python tracebacks for all threads if any test hangs and runs longer than 60 seconds. Your mileage may vary on how fast your computer is, so 60 seconds may not be sufficient if any tests take that long to run on your system.

My plan is to go through these failures and figure out what's going wrong. Some of these may not be specific to the free-threaded build.

To run the tests with the GIL disabled, you'll need to install 3.13b1 or the latest version of the CPython 3.13 branch, build cython, numpy, and scipy from the latest version of their respective git repositories. You'll need to e.g. pip install pybind11 and pythran after installing numpy and before building scipy. Then build scikit-learn as an editable install and run:

PYTHON_GIL=0 python -m pytest sklearn

[ogrisel]

Thanks for your investigation.

My plan is to go through these failures and figure out what's going wrong. Some of these may not be specific to the free-threaded build.

Indeed we do not have a Python 3.13 build yet so maybe some of those failures are related to that.

To run the tests with the GIL disabled, you'll need to install 3.13b1 or the latest version of the CPython 3.13 branch, build cython, numpy, and scipy from the latest version of their respective git repositories.

That would probably be too long to build all of those dependencies from source on the CI. Maybe upstream packages such as numpy and scipy could publish nightly builds for Python 3.13? As far as I know this is not the case yet:

• https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/numpy/
• https://pypi.anaconda.org/scientific-python-nightly-wheels/simple/scipy/

However, according to https://peps.python.org/pep-0703/#python-build-modes, there are two CPython ABIs for the foreseeable future and I am not sure how those two ABIs can be reflected in wheel filenames.

https://peps.python.org/pep-0703/#distribution mentions a dedicated conda channel but I am not sure about the status of this channel.

[ogrisel]

In any case, feel free to open PRs to address some of those failures you discovered even if we don't have a nogil CPython 3.13.X CI yet. We can merge them even if they can only be properly tested manually for now.

Note that we do have a 3.9 nogil CI running nightly on Azure Pipelines and it has been mostly green for a year or more. But I assume that 3.13 nogil has evolved a bit from 3.9 nogil hence the new failures.

[ngoldbaum]

Indeed we do not have a Python 3.13 build yet so maybe some of those failures are related to that.

Great, I might look at setting that up along with fixing the 3.13-specific issues.

Testing the free-threaded build will need to wait a little bit. I'm hoping that the setup-python action will gain support for installing it via a configuration flag (actions/setup-python#771). If that ends up not being quickly fixed now that the first beta is out, I'll look at setting up an Ubuntu-based CI job using the deadsnakes repo (once that gets updated with a 3.13 free-threaded build). See @hugovk's blog post.

However, according to https://peps.python.org/pep-0703/#python-build-modes, there are two CPython ABIs for the foreseeable future and I am not sure how those two ABIs can be reflected in wheel filenames.

There is a new ABI tag (3.13t), so you'd need to upload two python 3.13 wheels, one with the normal GIL-enabled ABI and one with the free-threaded ABI.

Also, one big issue is that pip doesn't know about the ABI tag yet (pypa/pip#12634). However that should be fixed in pip 24.1.

https://peps.python.org/pep-0703/#distribution mentions a dedicated conda channel but I am not sure about the status of this channel.

I don't think this has been set up yet. The last I heard the Anaconda Inc team was planning to start work on free-threaded support around the release of the first beta (so, nowish).

In any case, feel free to open PRs to address some of those failures you discovered even if we don't have a nogil CPython 3.13.X CI yet. We can merge them even if they can only be properly tested manually for now.

Great, thanks for this. Would creating a nogil or free-threaded label to tag issues and PRs with be OK?

[ngoldbaum]

Note that we do have a 3.9 nogil CI running nightly on Azure Pipelines and it has been mostly green for a year or more. But I assume that 3.13 nogil has evolved a bit from 3.9 nogil hence the new failures.

And it probably makes sense to retire this CI job since the 3.13 free-threaded build should replace it and we've been upstreaming support using the Py_GIL_DISABLED macro, which isn't used in the 3.9 nogil fork.

[lesteve]

I am Interested to help on this, don't hesitate to ping me on this kind of topics! Together with @ogrisel I was involved with setting up the scikit-learn CI for the nogil Python 3.9 and opening upstream bugs when we encountered them.

I have created the no-GIL label (following the same naming as scipy and numpy for consistency although I have always used nogil personally). Note that label search will match "nogil" and "free-threaded" since both terms appear in the label description.

Issues and PRs will need manual labeling by a scikit-learn maintainer for now, but I guess this is good enough for now, let me know if you have better suggestions!

[hugovk]

Naming things: when accepting PEP 703, the Steering Council said they wanted us to use a positive term like "free-threaded" rather than negative terms like "no-GIL":

We want to avoid negatives in terms and flags and such, so we won’t get into double-negative terrain (like we do when we talk about ‘non no-GIL’). We’d like a positive, clear term to talk about the no-GIL build, and we’re suggesting ‘free-threaded’.

And we use "topic-free-threading" for CPython:

https://github.com/python/cpython/labels/topic-free-threading

[lesteve]

Fair enough, I renamed the label to "free-threading", personally I don't care too much as long as I can find the label easily in the labels search for example like in the screenshot below (finding free-threaded label when searching nogil):

You may want to advocate for a similar change in numpy and scipy 😉

Edit: contrary to the screenshot, I ended up using free-threading (and not free-threaded) to match CPython.

[lesteve]

That would probably be too long to build all of those dependencies from source on the CI. Maybe upstream packages such as numpy and scipy could publish nightly builds for Python 3.13?

This seems definitely desirable. Alternatively in the shorter term, if there was a Docker image (even in an experimental state) with Python free-threaded build + latest numpy release and scipy release, this may make it more likely that curious but busy people (for example me) give it a go.

[ogrisel]

with Python free-threaded build + latest numpy release and scipy release

It could even be a nightly/weekly updated docker image with the dev versions of numpy and scipy. If a free docker hub account is enough we could use the auto-build feature of docker hub or similar:

https://docs.docker.com/docker-hub/builds/

Not sure about build time limit/usage quotas though.

EDIT: still I would prefer installing from a pip wheel house, e.g. https://pypi.anaconda.org/scientific-python-nightly-free-threaded/ to workaround any pip ABI tag conflicts that would prevent us to directly reuse https://pypi.anaconda.org/scientific-python-nightly-wheels/ .

[ngoldbaum]

to workaround any pip ABI tag conflicts that would prevent us to directly reuse https://pypi.anaconda.org/scientific-python-nightly-wheels/

There's a new ABI tag, so the new nightly wheels can live alongside the old ones on https://pypi.anaconda.org/scientific-python-nightly-wheels/. We just need to wait for a new pip release that can recognize and install the new ABI tag and for the the actual wheels to be produced. I'd like to look at that for NumPy at least later this week.

[ngoldbaum]

Ah but we probably will need to wait on cibuildwheel support first: pypa/cibuildwheel#1657

So maybe a docker image would be best until beta 2?

[rgommers]

if a free docker hub account is enough we could use the auto-build feature of docker hub or similar:

Just a note that we got rid of Docker images and weekly pushes to Docker Hub in NumPy and SciPy after they limited their free plan (and the machinery was a bit of a pain to begin with). Setting that up again isn't worth it imho. CPython 3.13 beta 2 is only two weeks away (May 28th), so I'd wait for that and if you want something in the meantime then building a one-off Docker image by hand to use for a few weeks until nightly wheels arrive should be okay I'd think. Investing time in auto-building Docker images is probably not the most productive way of spending time.

[betatim]

👍 for being patient for two weeks instead of building something "bespoke" that we should then un-build again.

[ogrisel]

We just need to wait for a new pip release that can recognize and install the new ABI tag and for the the actual wheels to be produced.

We would also need to make sure that older pip releases are not confused by the presence of wheels with the new ABI tag and do not fetch those wheels by mistake when trying to run nightly builds on the non-free-threaded version of CPython 3.13.

[ogrisel]

I built CPython main locally with the --disable-gil with:

  GDBM_CFLAGS="-I$(brew --prefix gdbm)/include" \                                                             
  GDBM_LIBS="-L$(brew --prefix gdbm)/lib -lgdbm" \
  ./configure --config-cache \
              --with-pydebug \
              --with-system-libmpdec \
              --with-openssl="$(brew --prefix openssl@3.0)" \
              --disable-gil && make -s -j10

• some manual CFLAGS/LDFLAGS for zlib installed with brew.

Then I could successfully run ./python.exe -m ensurepip to install pip needed by venv.

Then I created a venv with ./python.exe -m ~/.venvs/free-threading without running make install

And then I tried to build numpy with pip install --no-build-isolation -e . but I got the following error that I don't know how to fix.

      + /Users/ogrisel/.venvs/free-threading/bin/python /Users/ogrisel/code/numpy/vendored-meson/meson/meson.py setup /Users/ogrisel/code/numpy /Users/ogrisel/code/numpy/build/cp314td -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/Users/ogrisel/code/numpy/build/cp314td/meson-python-native-file.ini
      The Meson build system
      Version: 1.2.99
      Source dir: /Users/ogrisel/code/numpy
      Build dir: /Users/ogrisel/code/numpy/build/cp314td
      Build type: native build
      Project name: NumPy
      Project version: 2.1.0.dev0+git20240519.2970735
      C compiler for the host machine: cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.3.9.4)")
      C linker for the host machine: cc ld64 1053.12
      C++ compiler for the host machine: c++ (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.3.9.4)")
      C++ linker for the host machine: c++ ld64 1053.12
      Cython compiler for the host machine: cython (cython 3.0.10)
      Host machine cpu family: aarch64
      Host machine cpu: aarch64
      Program python found: YES (/Users/ogrisel/.venvs/free-threading/bin/python)
      Found pkg-config: /opt/homebrew/bin/pkg-config (0.29.2)
      Run-time dependency python found: NO (tried pkgconfig, pkgconfig and sysconfig)
      
      ../../meson.build:41:12: ERROR: Python dependency not found

any suggestion @ngoldbaum?

[ogrisel]

I think @lesteve had a similar problem on Linux so it's probably not macOS specific?

[rgommers]

The Meson build system
Version: 1.2.99

Not sure if it matters, but that's not latest numpy main (or an outdated git submodule). In the free-threaded CI job in the numpy repo (log), it's at 1.4.99.

[lesteve]

I tried again, not sure what I did differently this time to be honest, but it went a bit further. This time, I get a build error in numpy error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’? ...

More details

  FAILED: numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/meson-generated_numpy_random__mt19937.pyx.c.o
  ccache cc -Inumpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p -Inumpy/random -I../numpy/random -I../numpy/random/src -Inumpy/_core -I../numpy/_core -Inumpy/_core/include -I../numpy/_core/include -I../numpy/_core/src/common -Inumpy -I/opt/python-free-threading/include/python3.13td -I/home/lesteve/dev/numpy/.mesonpy-2arbmkr6/meson_cpu -fvisibility=hidden -fdiagnostics-color=always -DNDEBUG -Wall -Winvalid-pch -std=c11 -O3 -fno-strict-aliasing -msse -msse2 -msse3 -DNPY_HAVE_SSE2 -DNPY_HAVE_SSE -DNPY_HAVE_SSE3 -fPIC -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -DNPY_NO_DEPRECATED_API=0 -MD -MQ numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/meson-generated_numpy_random__mt19937.pyx.c.o -MF numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/meson-generated_numpy_random__mt19937.pyx.c.o.d -o numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/meson-generated_numpy_random__mt19937.pyx.c.o -c numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:2380:80: error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’?
   2380 | static CYTHON_INLINE PyObject *__Pyx_PyVectorcall_FastCallDict(PyObject *func, __pyx_vectorcallfunc vc, PyObject *const *args, size_t nargs, PyObject *kw);
        |                                                                                ^~~~~~~~~~~~~~~~~~~~
        |                                                                                vectorcallfunc
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11238:69: error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’?
  11238 | static PyObject *__Pyx_PyVectorcall_FastCallDict_kw(PyObject *func, __pyx_vectorcallfunc vc, PyObject *const *args, size_t nargs, PyObject *kw)
        |                                                                     ^~~~~~~~~~~~~~~~~~~~
        |                                                                     vectorcallfunc
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11283:80: error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’?
  11283 | static CYTHON_INLINE PyObject *__Pyx_PyVectorcall_FastCallDict(PyObject *func, __pyx_vectorcallfunc vc, PyObject *const *args, size_t nargs, PyObject *kw)
        |                                                                                ^~~~~~~~~~~~~~~~~~~~
        |                                                                                vectorcallfunc
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c: In function ‘__Pyx_CyFunction_CallAsMethod’:
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11972:6: error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’?
  11972 |      __pyx_vectorcallfunc vc = __Pyx_CyFunction_func_vectorcall(cyfunc);
        |      ^~~~~~~~~~~~~~~~~~~~
        |      vectorcallfunc
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:2470:45: error: initialization of ‘int’ from ‘vectorcallfunc’ {aka ‘struct _object * (*)(struct _object *, struct _object * const*, long unsigned int,  struct _object *)’} makes integer from pointer without a cast [-Wint-conversion]
   2470 | #define __Pyx_CyFunction_func_vectorcall(f) (((PyCFunctionObject*)f)->vectorcall)
        |                                             ^
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11972:32: note: in expansion of macro ‘__Pyx_CyFunction_func_vectorcall’
  11972 |      __pyx_vectorcallfunc vc = __Pyx_CyFunction_func_vectorcall(cyfunc);
        |                                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11975:16: error: implicit declaration of function ‘__Pyx_PyVectorcall_FastCallDict’; did you mean ‘__Pyx_PyObject_FastCallDict’? [-Wimplicit-function-declaration]
  11975 |         return __Pyx_PyVectorcall_FastCallDict(func, vc, &PyTuple_GET_ITEM(args, 0), (size_t)PyTuple_GET_SIZE(args), kw);
        |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        |                __Pyx_PyObject_FastCallDict
  numpy/random/_mt19937.cpython-313td-x86_64-linux-gnu.so.p/numpy/random/_mt19937.pyx.c:11975:16: error: returning ‘int’ from a function with return type ‘PyObject *’ {aka ‘struct _object *’} makes pointer from integer without a cast [-Wint-conversion]
  11975 |         return __Pyx_PyVectorcall_FastCallDict(func, vc, &PyTuple_GET_ITEM(args, 0), (size_t)PyTuple_GET_SIZE(args), kw);
        |                ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  [502/610] Compiling C++ object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/src_umath_stringdtype_ufuncs.cpp.o
  In file included from ../numpy/_core/src/umath/stringdtype_ufuncs.cpp:21:
  ../numpy/_core/src/umath/string_buffer.h:265:23: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
    265 |     inline Buffer<enc>()
        |                       ^
  ../numpy/_core/src/umath/string_buffer.h:265:23: note: remove the ‘< >’
  ../numpy/_core/src/umath/string_buffer.h:270:23: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
    270 |     inline Buffer<enc>(char *buf_, npy_int64 elsize_)
        |                       ^
  ../numpy/_core/src/umath/string_buffer.h:270:23: note: remove the ‘< >’
  [503/610] Compiling C object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/meson-generated_lowlevel_strided_loops.c.o
  [504/610] Compiling C object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/src_umath_ufunc_object.c.o
  [505/610] Compiling C++ object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/src_npysort_selection.cpp.o
  [506/610] Compiling C++ object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/src_npysort_quicksort.cpp.o
  [507/610] Compiling C++ object numpy/_core/_multiarray_umath.cpython-313td-x86_64-linux-gnu.so.p/src_umath_string_ufuncs.cpp.o
  In file included from ../numpy/_core/src/umath/string_ufuncs.cpp:21:
  ../numpy/_core/src/umath/string_buffer.h:265:23: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
    265 |     inline Buffer<enc>()
        |                       ^
  ../numpy/_core/src/umath/string_buffer.h:265:23: note: remove the ‘< >’
  ../numpy/_core/src/umath/string_buffer.h:270:23: warning: template-id not allowed for constructor in C++20 [-Wtemplate-id-cdtor]
    270 |     inline Buffer<enc>(char *buf_, npy_int64 elsize_)
        |                       ^
  ../numpy/_core/src/umath/string_buffer.h:270:23: note: remove the ‘< >’


</details>

[ngoldbaum]

FWIW, I also see that using a virtualenv like @ogrisel is trying to do.

Using pip 24.0 from /Users/goldbaum/.venvs/free-threading/lib/python3.13/site-packages/pip (python 3.13)
/Users/goldbaum/.venvs/free-threading/lib/python3.13/site-packages/pip/_internal/metadata/importlib/_envs.py:111: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pip._vendor.pkg_resources import find_distributions
Processing /Users/goldbaum/Documents/numpy
  Running command Preparing metadata (pyproject.toml)
  + /Users/goldbaum/.venvs/free-threading/bin/python /Users/goldbaum/Documents/numpy/vendored-meson/meson/meson.py setup /Users/goldbaum/Documents/numpy /Users/goldbaum/Documents/numpy/.mesonpy-guksbdeb -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/Users/goldbaum/Documents/numpy/.mesonpy-guksbdeb/meson-python-native-file.ini
  The Meson build system
  Version: 1.4.99
  Source dir: /Users/goldbaum/Documents/numpy
  Build dir: /Users/goldbaum/Documents/numpy/.mesonpy-guksbdeb
  Build type: native build
  Project name: NumPy
  Project version: 2.1.0.dev0+git20240522.83b4a23
  C compiler for the host machine: ccache cc (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.1.0.2.5)")
  C linker for the host machine: cc ld64 1022.1
  C++ compiler for the host machine: ccache c++ (clang 15.0.0 "Apple clang version 15.0.0 (clang-1500.1.0.2.5)")
  C++ linker for the host machine: c++ ld64 1022.1
  Cython compiler for the host machine: cython (cython 3.0.10)
  Host machine cpu family: aarch64
  Host machine cpu: aarch64
  Program python found: YES (/Users/goldbaum/.venvs/free-threading/bin/python)
  Found pkg-config: YES (/opt/homebrew/bin/pkg-config) 0.29.2
  Run-time dependency python found: NO (tried pkgconfig, pkgconfig and sysconfig)

  ../meson.build:41:12: ERROR: Python dependency not found

I think maybe because python isn't visible in your $PATH? FWIW I've been using pyenv to manage python builds and dealing with $PATH for local builds. In numpy's meson configuration we detect python using find_package.

[ngoldbaum]

I tried again, not sure what I did differently this time to be honest, but it went a bit further. This time, I get a build error in numpy error: unknown type name ‘__pyx_vectorcallfunc’; did you mean ‘vectorcallfunc’? ...

This is fixed in cython main, you'll need to compile cython from the git repo before building numpy.

[ngoldbaum]

Run-time dependency python found: NO (tried pkgconfig, pkgconfig and sysconfig)

Ahh, sorry! This is because meson can't find python.h or libpython, so you need to set e.g. C_INCLUDE_PATH and LD_LIBRARY_PATH, I think.