DOC: How to install Numpy on Apple silicon

[vlebert]

Issue with current documentation:

I have a Macbook PRO M1 and used to follow this procedure to install Numpy optimized with Apple Silicon hardware :

https://gist.github.com/MarkDana/a9481b8134cf38a556cf23e1e815dafb

$ conda install cython pybind11
$ pip install --no-binary :all: --no-use-pep517 numpy

But

• This technique is not working anymore

ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of mesonpy in pyproject.toml

• I read here and there that MacOS 14 (Sonoma) should allow better native installation through on Apple silicon

Source :

• https://mail.python.org/archives/list/numpy-discussion@python.org/message/W74HI226VJSOU7B6ZFKHLMVUFD4E7ITD/

For macOS 14 we will most likely, if things go well, ship wheels linked against the new ILP64 Accelerate build. Due to packaging limitations (the packaging library ignores minor versions of macOS), we can't ship wheels for >=13.3, only >=14.

As MacOS 14 is now live, I ran the common benchmark scripts and here are the results

1. With previous Numpy installation (pip install --no-binary :all: --no-use-pep517 numpy)
2. Conda installation in a new fresh env
3. Pip installlation in a new fresh env

Numpy install	benchmark script	Results
1. previous install	dario.py	23
2. conda install	dario.py	35
3. pip install	dario.py	38
1. previous install	mysvd.py	1.05
2. conda install	mysvd.py	11.44
3. pip install	mysvd.py	2.71

So, what is the proper way to install Numpy on Apple Silicon ? And what will be in the future ? How to get as good results as previous trick (pip install --no-binary :all: --no-use-pep517 numpy)

It is really difficult to find updated information on this matter.

For the records :

mysvd.py :

import time
import numpy as np
np.random.seed(42)
a = np.random.uniform(size=(300, 300))
runtimes = 10

timecosts = []
for _ in range(runtimes):
    s_time = time.time()
    for i in range(100):
        a += 1
        np.linalg.svd(a)
    timecosts.append(time.time() - s_time)

print(f'mean of {runtimes} runs: {np.mean(timecosts):.5f}s')

Idea or request for content:

No response

[ngoldbaum]

You should be able to build numpy locally if you drop the --no-use-pep517 (I think? I don't have an apple silicon mac).

We recently switched from numpy.distutils to meson as numpy's build backend so the --no-use-pep517 part of the invocation you're using is out of date.

[ngoldbaum]

I'm not sure why the conda packages and wheels are so much slower than local builds, though

[vlebert]

pip install --no-binary :all:does work and give me same results as 1. in my previous table. So this remains the best way to install Numpy

[ngoldbaum]

Closing since the installation issue is solved. Feel free to open another issue about the performance difference.

[rgommers]

performance will be due to a build from source on macOS 14 picking up Accelerate instead of using OpenBLAS. svd is known to be a lot faster with the new Accelerate on arm64.

For the next release we'll have wheels that use Accelerate.

[alanwilter]

As per the title of this issue, is the documentation updated? If so, where?
I'm using a MBP 16 M1 Max, with Sonoma 14.1.1 and numpy 1.26.1. How do I know if my numpy is using Accelerate?

[alanwilter]

Snooping in my Mac and I found this:

.../lib/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-darwin.so:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
        @loader_path/../.dylibs/libopenblas64_.0.dylib (compatibility version 0.0.0, current version 0.0.0)

So I it seems that the wheel I'm using is not compiled with Accelerate, shame.

[alanwilter]

Then I did:

pip install -U --no-binary :all: numpy

and now I can see:

.../lib/python3.9/site-packages/numpy/linalg/lapack_lite.cpython-39-darwin.so:
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1336.0.0)
        /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)

But I didn't notice any difference in my particular application.

[rgommers]

There will be wheels built against Accelerate for NumPy 2.0; it's a work in progress right now.

[alanwilter]

Thanks for the update.