Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

M1 Mac install fails due to numba #189

Closed
janosh opened this issue Jan 28, 2022 · 4 comments
Closed

M1 Mac install fails due to numba #189

janosh opened this issue Jan 28, 2022 · 4 comments

Comments

@janosh
Copy link
Contributor

janosh commented Jan 28, 2022

pip install pyxtal fails on M1 Macs due to numba not yet publishing an arm64 wheel to PyPI.

Looks like they're working on that in numba/numba#7779.

In the meantime though, perhaps it would make sense to turn numba from a regular dependency to an extras_require since it's only used by a single function in pyxtal:

PyXtal/pyxtal/XRD.py

Lines 613 to 619 in 9a17834

@nb.njit(nb.f8[:](nb.f8[:], nb.f8, nb.f8, nb.f8, nb.f8, nb.i8), cache = True)
def mod_pseudo_voigt(x, fwhm, A, eta_h, eta_l, N):
"""
A modified split-type pseudo-Voigt function for profiling peaks
- Izumi, F., & Ikeda, T. (2000).
"""

pymatgen for instance does that with many of its optional dependencies through the from monty.dev import requires decorator which informs users what install command to run once they try to use the functionality that actually requires it. See e.g. here:

https://github.com/materialsproject/pymatgen/blob/07794afb624f7613b3dbcebdf0034cab4c9ee7ea/pymatgen/electronic_structure/plotter.py#L3893-L3899

pip install logs 
  building 'numba._dynfunc' extension
  Warning: Can't read registry to find the necessary compiler setting
  Make sure that Python modules winreg, win32api or win32con are installed.
  C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk

  creating build/temp.macosx-12-arm64-3.10
  creating build/temp.macosx-12-arm64-3.10/numba
  compile options: '-I/opt/homebrew/include -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/sqlite/include -I/Users/janosh/.venv/py310/include -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c'
  clang: numba/_dynfuncmod.c
  In file included from numba/_dynfuncmod.c:1:
  numba/_dynfunc.c:140:5: warning: excess elements in struct initializer [-Wexcess-initializers]
      0,                         /* tp_print */
      ^
  numba/_dynfunc.c:245:5: warning: excess elements in struct initializer [-Wexcess-initializers]
      0,                         /* tp_print */
      ^
  numba/_dynfunc.c:457:5: warning: excess elements in struct initializer [-Wexcess-initializers]
      0,                                        /* tp_print */
      ^
  3 warnings generated.
  clang -bundle -undefined dynamic_lookup -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk build/temp.macosx-12-arm64-3.10/numba/_dynfuncmod.o -L/opt/homebrew/lib -L/opt/homebrew/opt/openssl@1.1/lib -L/opt/homebrew/opt/sqlite/lib -o build/lib.macosx-12-arm64-3.10/numba/_dynfunc.cpython-310-darwin.so
  building 'numba._dispatcher' extension
  C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk

  creating build/temp.macosx-12-arm64-3.10/numba/core
  creating build/temp.macosx-12-arm64-3.10/numba/core/typeconv
  compile options: '-I/Users/janosh/.venv/py310/lib/python3.10/site-packages/numpy/core/include -I/opt/homebrew/include -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/sqlite/include -I/Users/janosh/.venv/py310/include -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c'
  clang: numba/_dispatcher.cclang: numba/_dispatcherimpl.cpp
  clang: numba/_hashtable.c

  clang: numba/_typeof.c
  clang: numba/core/typeconv/typeconv.cpp
  numba/_dispatcher.c:26:13: error: no member named 'use_tracing' in 'struct _ts'
      tstate->use_tracing = 0;
      ~~~~~~  ^
  numba/_dispatcher.c:28:13: error: no member named 'use_tracing' in 'struct _ts'
      tstate->use_tracing = ((tstate->c_tracefunc != NULL)
      ~~~~~~  ^
  numba/_dispatcher.c:290:17: error: no member named 'use_tracing' in 'struct _ts'
      if (tstate->use_tracing && tstate->c_profilefunc)
          ~~~~~~  ^
  numba/_dispatcher.c:499:13: error: no member named 'use_tracing' in 'struct _ts'
      if (ts->use_tracing && ts->c_profilefunc) {
          ~~  ^
  numba/_dispatcher.c:657:5: warning: excess elements in struct initializer [-Wexcess-initializers]
      0,                                           /* tp_print */
      ^
  1 warning and 4 errors generated.
  error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -I/Users/janosh/.venv/py310/lib/python3.10/site-packages/numpy/core/include -I/opt/homebrew/include -I/opt/homebrew/opt/openssl@1.1/include -I/opt/homebrew/opt/sqlite/include -I/Users/janosh/.venv/py310/include -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c numba/_dispatcher.c -o build/temp.macosx-12-arm64-3.10/numba/_dispatcher.o" failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for numba
  Running setup.py clean for numba
  Building wheel for llvmlite (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/janosh/.venv/py310/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"'; __file__='"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-wheel-8igniqqp
       cwd: /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/
  Complete output (26 lines):
  running bdist_wheel
  /Users/janosh/.venv/py310/bin/python3.10 /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py
  LLVM version... Traceback (most recent call last):
    File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 105, in main_posix
      out = subprocess.check_output([llvm_config, '--version'])
    File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 420, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 501, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 966, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 1842, in _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 191, in <module>
      main()
    File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 185, in main
      main_posix('osx', '.dylib')
    File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 107, in main_posix
      raise RuntimeError("%s failed executing, please point LLVM_CONFIG "
  RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config
  error: command '/Users/janosh/.venv/py310/bin/python3.10' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for llvmlite
  Running setup.py clean for llvmlite
Failed to build numba llvmlite
Installing collected packages: llvmlite, py3Dmol, numba, ase, pyxtal
    Running setup.py install for llvmlite ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/janosh/.venv/py310/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"'; __file__='"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-record-2l1i6fxh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/janosh/.venv/py310/include/site/python3.10/llvmlite
         cwd: /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/
    Complete output (31 lines):
    running install
    /Users/janosh/.venv/py310/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
      warnings.warn(
    running build
    got version from file /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/llvmlite/_version.py {'version': '0.35.0', 'full': 'ea23b026930cc00824c907172383f54c9d438e6b'}
    running build_ext
    /Users/janosh/.venv/py310/bin/python3.10 /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py
    LLVM version... Traceback (most recent call last):
      File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 105, in main_posix
        out = subprocess.check_output([llvm_config, '--version'])
      File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 420, in check_output
        return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
      File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 501, in run
        with Popen(*popenargs, **kwargs) as process:
      File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 966, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/opt/homebrew/Cellar/python@3.10/3.10.1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 1842, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 191, in <module>
        main()
      File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 185, in main
        main_posix('osx', '.dylib')
      File "/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/ffi/build.py", line 107, in main_posix
        raise RuntimeError("%s failed executing, please point LLVM_CONFIG "
    RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config
    error: command '/Users/janosh/.venv/py310/bin/python3.10' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/janosh/.venv/py310/bin/python3.10 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"'; __file__='"'"'/private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-install-_mcqbso0/llvmlite_9de1532961024ca0aa8941b4f2daa05c/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/5m/sb0wx83j0m36xk_zswkfnffh0000gn/T/pip-record-2l1i6fxh/install-record.txt --single-version-externally-managed --compile --install-headers /Users/janosh/.venv/py310/include/site/python3.10/llvmlite Check the logs for full command output.
@qzhu2017
Copy link
Owner

It looks like that you are using python3.10. Numba guys are working on the fix of python3.10 and new numpy version. I think it would be easier to just keep it for a while. They should be able to catch up soon. I have no problem with python3.7-3.9

Regarding the option of putting Numba as a extra library, I tried it but I failed. Something like the following won't work

@requires(nb is not None, "numba not imported! Please install numba.")
@nb.njit(nb.f8[:](nb.f8[:], nb.f8, nb.f8, nb.f8, nb.f8, nb.i8), cache = True)
def mod_pseudo_voigt(x, fwhm, A, eta_h, eta_l, N):

I don't know how to deal with this. So I will simply disable the pip install via python3.10. However, you can do python setup.py install with py3.10 if you want.

@janosh
Copy link
Contributor Author

janosh commented Jan 28, 2022

I guess we can convert this issue into a tracker for when the py3.10 wheel becomes available then.

@janosh
Copy link
Contributor Author

janosh commented Jan 28, 2022

Regarding the option of putting Numba as a extra library, I tried it but I failed. Something like the following won't work

Did you try this?

@requires(nb is not None, "numba could not be imported! Please install numba.")
def _mod_pseudo_voigt(x, fwhm, A, eta_h, eta_l, N):
    ...


if nb is not None:
    mod_pseudo_voigt = nb.njit(
        nb.f8[:](nb.f8[:], nb.f8, nb.f8, nb.f8, nb.f8, nb.i8), cache = True
    )(_mod_pseudo_voigt)

@jan-janssen
Copy link

@qzhu2017 Any update on this issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jan-janssen @qzhu2017 @janosh