QST: Install from source without blas with pip

[snowman2]

When trying out latest release candidate:

From: pyproj4/pyproj#1330 (comment)

Is there an environment variable you can set to not require blas when installing numpy?

/meson.build:207:4: ERROR: Problem encountered: No BLAS library detected! Install one, or use the `allow-noblas` build option (note, this may be up to 100x slower for some linear algebra operations).

[mattip]

Is this a duplicate of #24704? I think the problem is that the project has a test dependency on NumPy and ends up building from source instead of using pre-built wheels.

[rgommers]

Besides the "wheels are coming", the answer to the question when you build from source is (as hinted at in the error message) that this works:

$ # the -C syntax requires pip 23.1; for older pip versions replace it with --config-settings=
$ pip install numpy -C-Dallow-noblas=true
$ # or with pypa/build from a local repo
$ python -m build -C-Dallow-noblas=true

There is no environment variable to do the same thing. We may relax the requirement for having BLAS installed, but I think it'd be good to see how many people find the current state a problem with the final 1.26.0 release which will include 32-bit Windows wheels.

[rgommers]

While the question is answered, let's leave this issue open until at least a week after the 1.26.0 release date, so it's easier to discover for others.

[snowman2]

Thanks for the responses 👍

Is this a duplicate of #24704?

These failures are different because they are for pypy builds and for the i686 architecture. If the wheels are coming, this shouldn't be an issue.

[snowman2]

For anyone who runs into this - based on:

• https://meson-python.readthedocs.io/en/latest/how-to-guides/meson-args.html

• numpy/.circleci/config.yml

  Line 61 in ba2935d

  pip install . --config-settings=setup-args="-Dallow-noblas=true"

This is the syntax for pip:

python -m pip install numpy --config-settings=setup-args="-Dallow-noblas=true" 

[aptalca]

We may relax the requirement for having BLAS installed, but I think it'd be good to see how many people find the current state a problem with the final 1.26.0 release which will include 32-bit Windows wheels.

This will be an issue for aarch64 users on musl based systems as numpy does not provide binary wheels for that. We (linuxserver.io) are facing that issue on our docker images that are alpine based with aarch64 images

[rgommers]

@aptalca thanks for that comment. To make sure I'm understanding it right: are you saying that you're seeing failed builds and that's a problem, or also that it's fine to avoid the failed builds by default by building fallback linear algebra routines that may be ~50x slower?

[aptalca]

Sorry, I should have clarified.

We are seeing failed builds due to the newly enforced openblas requirement. Previously, numpy would build without openblas just fine. Now we have to either add openblas or add the build argument to allow building without openblas.

In most cases (for us) the speed difference is not noticeable as the apps we build don't rely on complex calculations.

[Itsmittyhere]

I am trying to install:
~/klippy-env/bin/pip install -v numpy

I get this error:
Library m found: YES
Found CMake: /usr/bin/cmake (3.18.4)
WARNING: CMake Toolchain: Failed to determine CMake compilers state
Run-time dependency openblas found: NO (tried pkgconfig and cmake)
Run-time dependency openblas found: NO (tried pkgconfig and cmake)

../../numpy/meson.build:207:4: ERROR: Problem encountered: No BLAS library detected! Install one, or use the allow-noblas build option (note, this may be up to 100x slower for some linear algebra operations).

A full log can be found at /tmp/pip-install-gmgwihuj/numpy_9524064cd0ae4863b894d750518a40a5/.mesonpy-utl0nn8j/build/meson-logs/meson-log.txt
error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /home/pi/klippy-env/bin/python /home/pi/klippy-env/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmp2fcjefqc
cwd: /tmp/pip-install-gmgwihuj/numpy_9524064cd0ae4863b894d750518a40a5
Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

[notice] A new release of pip is available: 23.2 -> 23.2.1
[notice] To update, run: python -m pip install --upgrade pip
pi@voron:~ $

I tried to do this python update, but apparently it doesn't update either. I have no idea what to do.
Thanks in advance.

[mattip]

@Itsmittyhere what version of python are you using on what platform? Do you intend to build from source? You need to install the development version of a BLAS library.

[fuglede]

FWIW: I ran into this one while using cibuildwheel to build a package that relies on NumPy. Based on your discussion, I have to imagine there's some way of configuring setup.cfg to specify "allow-noblas" somehow, but since I have no particular need for supporting PyPy/i686, I could get around it by adding

[tool.cibuildwheel]
skip = ["pp*", "*i686"]

to pyproject.toml.

[rgommers]

Thanks for the comment @fuglede. Using numpy as a build dependency on platforms for which we don't publish wheels is a tricky one, and perhaps the best reason so far to go back to allow-noblas=true.

There's no config file or environment variable to set that can address this I believe. The way to do it would be to call pip install numpy -C-Dallow-noblas=true in the "before build" stage of the cibuildwheel invocation.

[ReenigneArcher]

@rgommers replacing cmake with pkg-config resolved it. Thank you!

[mattip]

It would be nice to debug why the CMake files supplied with openblas are not sufficient. Is there a command-line interface to the cmake module discovery mechanism?

[rfrenchseti]

Just adding to the above comments, I just ran into the same problem trying to update a package I maintain (that depends on numpy) to work with Python 3.12. I tried everything I could think of, including adding the cibuildwheel pre-install of numpy with the allow-noblas=true option and I just couldn't get it to work. Disabling the build of *i686 eventually solved the problem, but is obviously not ideal.

[ReenigneArcher]

@rfrenchseti try adding these packages before installing numpy.

        pkg-config \
        libopenblas-dev \

Names may be different depending on which distro.

[kif]

Just for info, we face the same issue when doing CI on an architecture where no wheels are provided (PPC64le).
Numpy should just build by default, even with the largest warning about bad performances, but it should not fail building because so many projects have numpy as a dependency.

[rgommers]

Thanks for the input everyone. I just opened gh-25063 to revert the default to allow-noblas=true. That should land in 1.26.2 and 2.0.

[TimotheusBachinger]

For anyone who runs into this - based on:

• https://meson-python.readthedocs.io/en/latest/how-to-guides/meson-args.html

• numpy/.circleci/config.yml

  Line 61 in ba2935d

  pip install . --config-settings=setup-args="-Dallow-noblas=true"

This is the syntax for pip:

python -m pip install numpy --config-settings=setup-args="-Dallow-noblas=true" 

For everybody running into that (like myself): This works only from numpy version 1.26.1!