BLD Check build dependencies in meson.build

[lesteve]

I recently realised that we don't check build dependencies version when building with Meson. I was getting some weird Cython errors until I realised that I had 3.0.8 in my env.

I added the necessary logic in sklearn/meson.build to do something similar as what we do in setup.py.

This is a bit unfortunate that the logic is a bit duplicated between meson.build and setup.py but I think it is acceptable. I reused the sklearn/_min_dependencies.py script to avoid hard-coding minimum versions in yet another place.

I originally thought we could run the same Python script file from meson.build and setup.py, except that there are some complications:

• if we want to reuse packaging version comparison we need to to import sklearn.external._packaging.parse and for this you need the builtins.__SETUP__SKLEARN__ hack to allow partial import of sklearn before sklearn is actually built
• the script needs to be in the root folder in order to be able to import sklearn, or you need to do sys.path manipulations

Full disclosure: version comparison in meson.build only takes into account X.Y.Z versions i.e. not full PEP440. I think this is OK, since for minimum dependencies it is very rare that you want a dev version that is close to a min dependency.

[github-actions]

✔️ Linting Passed

All linting checks passed. Your pull request is in excellent shape! ☀️

_{Generated for commit: f0f1685. Link to the linter CI: here}

[jeremiedbb]

I don't think this one is necessary since we have the requirement in pyproject.toml. meson is already aware of the lower bound

[jeremiedbb]

I tested and meson raises an error in that case

[lesteve]

I could remove it, but I like the fact that you have an early error if you have a too old Python.

I tested it locally and for now you need to install numpy and scipy to be told "actually you know what your Python is too old, sorry".

It seems it is meson-python that raises the error according to the error:

  ('\x1b[31m',)meson-python: error: Package requires Python version >=3.9, running on 3.8.18

As noted in #28721 (comment) it would be nice if you can get the error without going through pip, e.g. spin build (if we use it one day) will call Meson directly.

[jeremiedbb]

I was indeed able to build with cython 3.0.8 without any complain. Since the min dependencies are listed in pyproject.toml, is there no automated way to make meson aware of them ?

[lesteve]

I haven't found anything, pyproject.toml is Python-specific and Meson is a generic build tool. You can run a command from meson.build and check its exit code, see https://mesonbuild.com/External-commands.html for more details.

I think there are a few options:

1. do what is currently done in this PR, _min_dependencies.py for getting minimum versions + Meson-native version comparison. I am not saying this is great, but to me this seems a reasonable trade-off. I spent a bit of time trying to find something that I found significantly better and I was not able to, but better suggestions welcome!
2. write an external Python script but we need packaging to compare versions. packaging is vendored inside pip, maybe we could assume that in most cases you have pip when you are trying to build scikit-learn ...
3. use our vendored packaging but we need __SKLEARN_SETUP__ hack + sys.path manipulations to import from sklearn before sklearn is built
4. maybe better suggestions?

Note: we could give a more complicated pip install command with --check-build-dependencies but that would only work when building with Meson through pip. If one day we use spin, spin build will call meson directly (i.e. not going through pip) and as such will not error in there are some build dependencies missing.

One thing I had some hope for would be to use run_command + right pip incantation to check build-dependencies in a light-weight manner (assuming pip is installed which is the most common case I think). I thought maybe

pip install . --dry-run --no-build-isolation --check-build-dependencies`

may work, but no it does not, causing an error because scikit-learn is already being built

LookupError: file:///home/lesteve/dev/scikit-learn is already being built: file:///home/lesteve/dev/scikit-learn

Side-comment: actually even pip does not check build-dependencies are satisfied unless you use --check-build-dependencies. See pypa/pip#11116 if you are curious why.

[lesteve]

Another thing I am not too sure about is the cross-compiling case which is why I only check numpy and scipy when building natively (i.e. not cross-compiling). It seems like when cross-compiling the build machine can not necessarily call the host machine Python interpreter, maybe more details can be found in scipy/scipy#16783, scipy/scipy#14812 and https://mesonbuild.com/Cross-compilation.html.

There is a comment that mentions this in sklearn/meson.build (taken from Scipy):

# For cross-compilation it is often not possible to run the Python interpreter
# in order to retrieve numpy's include directory. It can be specified in the
# cross file instead:
#   [properties]
#   numpy-include-dir = /abspath/to/host-pythons/site-packages/numpy/core/include
#
# This uses the path as is, and avoids running the interpreter.