Numba on the Apple M1

[xhochy]

Starting this as a thread to track the support of this processor. It seems like most of the things work but I guess other people will come around and also look for "what's missing".

I got it to run using packages from conda-forge and LLVM 10. My personal code runs fine and most of the unit tests pass but not all.

This depends on the following PRs to be merged on some form:

• Support cross-compiling llvmlite: Support building with CMake on posix systems llvmlite#677
• Build LLVM 10 for osx-arm64 on conda-forge: 10.x: Build for osx-arm64 conda-forge/llvmdev-feedstock#111
• Build llvmlite for osx-arm64 on conda-forge: ARM OSX Migrator conda-forge/llvmlite-feedstock#50
• Build numba for osx-arm64 on conda-forge: PR to be made once the above ones are merged.

I will post issues as I come along in separate comments below.

[xhochy]

In general, the hashing algorithm of CPython leads to the UserWarning every time you use Numba:

/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/cpython/hashing.py:484: UserWarning: FNV hashing is not implemented in Numba. See PEP 456 https://www.python.org/dev/peps/pep-0456/ for rationale over not using FNV. Numba will continue to work, but hashes for built in types will be computed using siphash24. This will permit e.g. dictionaries to continue to behave as expected, however anything relying on the value of the hash opposed to hash as a derived property is likely to not work as expected.

[xhochy]

We have one test that fails due to a different hash:

ERROR: test_basic_unicode (numba.tests.test_hashing.TestUnicodeHashing)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_hashing.py", line 80, in check_hash_values
    self.assertEqual(nb_hash, hash(val))
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/unittest/case.py", line 912, in assertEqual
    assertion_func(first, second, msg=msg)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/unittest/case.py", line 905, in _baseAssertEqual
    raise self.failureException(msg)
AssertionError: 5733495426409129890 != 1570954067822328427

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_hashing.py", line 267, in test_basic_unicode
    self.check_hash_values([kind1_string[:i]])
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_hashing.py", line 85, in check_hash_values
    print(abs(val), hashing._PyHASH_MODULUS - 1)
TypeError: bad operand type for abs(): 'str'

Stdout:
val, nb_hash, hash(val)
ab 5733495426409129890 1570954067822328427
abs(val), hashing._PyHASH_MODULUS - 1

[xhochy]

All other 5 failures are due to wrongly compiled libraries. This is definitely something to look for in the setup.py and its dependencies as this was a cross-compiled build of numba:

======================================================================
ERROR: test_manage_memory (numba.tests.test_nrt.TestNrtExternalCFFI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_nrt.py", line 609, in test_manage_memory
    ffi, mod = self.compile_cffi_module(name, source, cdef)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_nrt.py", line 568, in compile_cffi_module
    mod = import_dynamic(name)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/support.py", line 641, in import_dynamic
    __import__(modname)
ImportError: dlopen(/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/cffi_test_TestNrtExternalCFFI_test_manage_memory-0jfvgy30/TestNrtExternalCFFI_test_manage_memory.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/cffi_test_TestNrtExternalCFFI_test_manage_memory-0jfvgy30/TestNrtExternalCFFI_test_manage_memory.cpython-38-darwin.so: mach-o, but wrong architecture
	/private/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/cffi_test_TestNrtExternalCFFI_test_manage_memory-0jfvgy30/TestNrtExternalCFFI_test_manage_memory.cpython-38-darwin.so: mach-o, but wrong architecture

Stderr:
/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/cffi/cparser.py:163: UserWarning: Global variable 'status' in cdef(): for consistency with C it should have a storage class specifier (usually 'extern')
  warnings.warn("Global variable '%s' in cdef(): for consistency "

======================================================================
ERROR: test_compile_nrt (numba.tests.test_pycc.TestCC)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_pycc.py", line 283, in test_compile_nrt
    with self.check_cc_compiled(self._test_module.cc_nrt) as lib:
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_pycc.py", line 179, in check_cc_compiled
    with self.check_c_ext(self.tmpdir, cc.name) as lib:
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_pycc.py", line 84, in check_c_ext
    lib = import_dynamic(name)
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/support.py", line 641, in import_dynamic
    __import__(modname)
ImportError: dlopen(/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_pycc-nft583ic/pycc_test_nrt.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_pycc-nft583ic/pycc_test_nrt.cpython-38-darwin.so: mach-o, but wrong architecture
	/private/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_pycc-nft583ic/pycc_test_nrt.cpython-38-darwin.so: mach-o, but wrong architecture

======================================================================
ERROR: test_from_buffer_pyarray (numba.tests.test_cffi.TestCFFI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_cffi.py", line 30, in setUp
"""
Hash implementations for Numba types
"""

import math
import numpy as np
import sys
import ctypes
import warnings
from collections import namedtuple

import llvmlite.binding as ll
import llvmlite.llvmpy.core as lc
from llvmlite import ir

from numba.core.extending import (
    overload, overload_method, intrinsic, register_jitable)
from numba.core import errors
from numba.core import types, utils
from numba.core.unsafe.bytes import grab_byte, grab_uint64_t

_py38_or_later = utils.PYVERSION >= (3, 8)

# This is Py_hash_t, which is a Py_ssize_t, which has sizeof(size_t):
# https://github.com/python/cpython/blob/d1dd6be613381b996b9071443ef081de8e5f3aff/Include/pyport.h#L91-L96    # noqa: E501
_hash_width = sys.hash_info.width
_Py_hash_t = getattr(types, 'int%s' % _hash_width)
_Py_uhash_t = getattr(types, 'uint%s' % _hash_width)

# Constants from CPython source, obtained by various means:
# https://github.com/python/cpython/blob/d1dd6be613381b996b9071443ef081de8e5f3aff/Include/pyhash.h    # noqa: E501
_PyHASH_INF = sys.hash_info.inf
_PyHASH_NAN = sys.hash_info.nan
_PyHASH_MODULUS = _Py_uhash_t(sys.hash_info.modulus)
_PyHASH_BITS = 31 if types.intp.bitwidth == 32 else 61  # mersenne primes
_PyHASH_MULTIPLIER = 0xf4243  # 1000003UL
_PyHASH_IMAG = _PyHASH_MULTIPLIER
_PyLong_SHIFT = sys.int_info.bits_per_digit
_Py_HASH_CUTOFF = sys.hash_info.cutoff
_Py_hashfunc_name = sys.hash_info.algorithm


# hash(obj) is implemented by calling obj.__hash__()


@overload(hash)
def hash_overload(obj):
    def impl(obj):
        return obj.__hash__()
"~/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/cpython/hashing.py" 719L, 26149C
    mod.init_ool()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 142, in init_ool
    ffi_ool, mod = load_ool_module()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 111, in load_ool_module
    mod = import_dynamic('cffi_usecases_ool')
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/support.py", line 641, in import_dynamic
    __import__(modname)
ImportError: dlopen(/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-5bjdebon/cffi_usecases_ool.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-5bjdebon/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture
	/private/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-5bjdebon/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture

======================================================================
ERROR: test_two_funcs (numba.tests.test_cffi.TestCFFI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_cffi.py", line 30, in setUp
    mod.init_ool()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 142, in init_ool
    ffi_ool, mod = load_ool_module()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 111, in load_ool_module
    mod = import_dynamic('cffi_usecases_ool')
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/support.py", line 641, in import_dynamic
    __import__(modname)
ImportError: dlopen(/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-kfns3l73/cffi_usecases_ool.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-kfns3l73/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture
	/private/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-kfns3l73/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture

======================================================================
ERROR: test_user_defined_symbols (numba.tests.test_cffi.TestCFFI)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/test_cffi.py", line 30, in setUp
    mod.init_ool()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 142, in init_ool
    ffi_ool, mod = load_ool_module()
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/cffi_usecases.py", line 111, in load_ool_module
    mod = import_dynamic('cffi_usecases_ool')
  File "/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/tests/support.py", line 641, in import_dynamic
    __import__(modname)
ImportError: dlopen(/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-rwced46f/cffi_usecases_ool.cpython-38-darwin.so, 2): no suitable image found.  Did find:
	/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-rwced46f/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture
	/private/var/folders/km/wl0tx9197gz3__0_h7y9gfyr0000gn/T/numba-tests.501/test_cffi-rwced46f/cffi_usecases_ool.cpython-38-darwin.so: mach-o, but wrong architecture

I did run the tests using python -m numba.runtests -b -j --random='0.20' --exclude-tags='long_running' -m 1 -- numba.tests

[stuartarchibald]

@xhochy thanks for starting to look at this. The Numba build farm should get some suitable hardware soon and we'll be able to help!

In the above, the hash tests are likely failing because of this:

/Users/uwe/mambaforge/envs/numba-test/lib/python3.8/site-packages/numba/cpython/hashing.py:484: UserWarning: FNV hashing is not implemented in Numba. See PEP 456 https://www.python.org/dev/peps/pep-0456/ for rationale over not using FNV. Numba will continue to work, but hashes for built in types will be computed using siphash24. This will permit e.g. dictionaries to continue to behave as expected, however anything relying on the value of the hash opposed to hash as a derived property is likely to not work as expected.

Where did you get this Python from? It's been compiled without siphash24 which I think requires actually forcing that to happen: https://github.com/python/cpython/blob/master/configure.ac#L2953-L2972

The tests failing where a DSO fails to load looks like an incorrect toolchain/invalid tooling problem, I think these tests will be using ${CC} to compile a library and something is evidently wrong as it's likely targeting x86 (mach-o, but wrong architecture).

[xhochy]

Where did you get this Python from? It's been compiled without siphash24 which I think requires actually forcing that to happen: https://github.com/python/cpython/blob/master/configure.ac#L2953-L2972

This is the Python build from conda-forge as defined in https://github.com/conda-forge/python-feedstock/blob/89902c593957440acefd3acd65dfb242bd2b8ace/recipe/build_base.sh. There seems to be some ignored issues when building in the log: conda-forge/python-feedstock#430 , we're going to investigate.

[xhochy]

With python 3.8.6 h12cc5a1_2_cpython the hashing issues are now resolved 👍

[stuartarchibald]

With python 3.8.6 h12cc5a1_2_cpython the hashing issues are now resolved +1

@xhochy great, thanks for confirming.

[xhochy]

With conda-forge/python-feedstock#434 and conda-forge/python-feedstock#433 all the above issues are now fixed. The test suite passes smoothly. Closing as this is working as expected, conda-forge packages for the 0.52 release on osx-arm64 are also available now.

[seibert]

Nice! Thanks for all the work on this.

[sourestdeeds]

Im still getting an error when using the Miniconda arm osx install. The same one as described here.. The link must be for an older version on the git. Can I update somehow or will I have to find another?

[xhochy]

Im still getting an error when using the Miniconda arm osx install. The same one as described here.. The link must be for an older version on the git. Can I update somehow or will I have to find another?

Which error exactly of the many above?

[sourestdeeds]

All of them, I think the miniforge distro for Mac arm is using an older build. The fail seems to be a result of llvm. I'll post the traceback tomorrow if its useful