Missing package: llvmlite

[joyeJ]

Package name: llvmlite
Issue type: Missing package
Link to PyPI page: https://pypi.org/project/llvmlite
Link to piwheels page: https://www.piwheels.org/project/llvmlite
Version: e.g. 1.2.3 or all
Python version: e.g. 3.5 or all

[bennuttall]

Note: please complete the issue template properly in future. I have filled it out for you.

Here's the output for the build attempt on Raspbian Stretch: https://paste.debian.net/1091345/

[bennuttall]

Update here: numba/llvmlite#314

[bennuttall]

Closing due to no activity.

[nmstoker]

Hello! 🙂 I believe I've got a solution to allow llvmlite to be installed - I've only tested it on an RPI 4 with Feb 2020 release of Buster Lite so far but I was able to do it as follows:

• By installing LLVM with apt:

sudo apt install llvm

(noting that it installs version 7, which is listed as compatible with llvmlite here: Compatibility)

• Then find out where llvm-config was installed with:

which llvm-config

• And pasting that result into this LLVM_CONFIG=/path/to/llvm-config pip3 install llvmlite for me results in:

LLVM_CONFIG=/usr/bin/llvm-config pip3 install llvmlite

which chugs away for a while until it successfully installed (output details below)

Not sure if it makes a difference but I ran this within a virtual environment.

@bennuttall Does that help? If there's more you'd like me to try I might be able to (I've got some earlier RPis around so could give it a whirl on another one in the next week or so, just to confirm it works more widely). And let me know if I should open a new issue for this (or can this one just be re-opened?)

Output from pip:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting llvmlite
Using cached https://files.pythonhosted.org/packages/dc/8f/5c6d0e8be1585247c4fd33b84dc66d1fa6449b77d27f42a51d45242307c9/llvmlite-0.32.0.tar.gz
Building wheels for collected packages: llvmlite
Running setup.py bdist_wheel for llvmlite ... error
Complete output from command /home/pi/Projects/TTS/venv/bin/python3 -u -c "import setuptools, tokenize;file='/tmp/pip-install-nj_04ks0/llvmlite/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /tmp/pip-wheel-cibhohaq --python-tag cp37:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help

error: invalid command 'bdist_wheel'

---

Failed building wheel for llvmlite
Running setup.py clean for llvmlite
Failed to build llvmlite
Installing collected packages: llvmlite
Running setup.py install for llvmlite ... done
Successfully installed llvmlite-0.32.0

[nmstoker]

Maybe this isn't working. Seem to be running into a problem with importing numba, so perhaps llvmlite isn't working despite the installation message. I'd better do a bit more research first... sorry!

EDIT: the problem with numba was resolved by changing the version of numpy I had installed (1.15.4->1.16.1), so seemingly unrelated to my proposed fix for llvmlite
Therefore perhaps the proposal can be tried to produce a suitable wheel?

[nmstoker]

Hi @bennuttall - I expect you're busy, but when you get a moment, can this be re-opened?

I believe my approach means installing llvmlite can now work - as I haven't helped resolve one of these piwheel issues before, please bear with me, but if there's anything additional I should try or explain regarding the above, do let me know.

[bennuttall]

here's the build output for 0.32.0 on buster:

020-04-17T14:55:01,167 Using version 0.32.0 (newest of versions: 0.32.0)
2020-04-17T14:55:01,172 Collecting llvmlite==0.32.0
2020-04-17T14:55:01,175   Created temporary directory: /tmp/pip-unpack-mzy8t_jn
2020-04-17T14:55:01,178   Getting credentials from keyring for files.pythonhosted.org
2020-04-17T14:55:01,322   Downloading llvmlite-0.32.0.tar.gz (103 kB)
2020-04-17T14:55:01,786   Added llvmlite==0.32.0 from https://files.pythonhosted.org/packages/dc/8f/5c6d0e8be1585247c4fd33b84dc66d1fa6449b77d27f42a51d45242307c9/llvmlite-0.32.0.tar.gz#sha256=d65c645c32fae7b50852e80597eaef8369f826b75017d5c8d07c800e37eaed89 to build tracker '/tmp/pip-req-tracker-xno4si9p'
2020-04-17T14:55:01,788     Running setup.py (path:/tmp/pip-wheel-8zs2sbun/llvmlite/setup.py) egg_info for package llvmlite
2020-04-17T14:55:01,790     Running command python setup.py egg_info
2020-04-17T14:55:04,044     running egg_info
2020-04-17T14:55:04,048     creating /tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info
2020-04-17T14:55:04,051     writing /tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/PKG-INFO
2020-04-17T14:55:04,056     writing dependency_links to /tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/dependency_links.txt
2020-04-17T14:55:04,061     writing requirements to /tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/requires.txt
2020-04-17T14:55:04,063     writing top-level names to /tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/top_level.txt
2020-04-17T14:55:04,067     writing manifest file '/tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/SOURCES.txt'
2020-04-17T14:55:04,373     reading manifest file '/tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/SOURCES.txt'
2020-04-17T14:55:04,383     writing manifest file '/tmp/pip-wheel-8zs2sbun/llvmlite/pip-egg-info/llvmlite.egg-info/SOURCES.txt'
2020-04-17T14:55:04,509   Source in /tmp/pip-wheel-8zs2sbun/llvmlite has version 0.32.0, which satisfies requirement llvmlite==0.32.0 from https://files.pythonhosted.org/packages/dc/8f/5c6d0e8be1585247c4fd33b84dc66d1fa6449b77d27f42a51d45242307c9/llvmlite-0.32.0.tar.gz#sha256=d65c645c32fae7b50852e80597eaef8369f826b75017d5c8d07c800e37eaed89
2020-04-17T14:55:04,511   Removed llvmlite==0.32.0 from https://files.pythonhosted.org/packages/dc/8f/5c6d0e8be1585247c4fd33b84dc66d1fa6449b77d27f42a51d45242307c9/llvmlite-0.32.0.tar.gz#sha256=d65c645c32fae7b50852e80597eaef8369f826b75017d5c8d07c800e37eaed89 from build tracker '/tmp/pip-req-tracker-xno4si9p'
2020-04-17T14:55:04,515 Building wheels for collected packages: llvmlite
2020-04-17T14:55:04,525   Created temporary directory: /tmp/pip-wheel-7mojo7tc
2020-04-17T14:55:04,526   Building wheel for llvmlite (setup.py): started
2020-04-17T14:55:04,527   Destination directory: /tmp/pip-wheel-7mojo7tc
2020-04-17T14:55:04,527   Running command /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-8zs2sbun/llvmlite/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-8zs2sbun/llvmlite/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-7mojo7tc
2020-04-17T14:55:06,600   running bdist_wheel
2020-04-17T14:55:06,625   /usr/bin/python3 /tmp/pip-wheel-8zs2sbun/llvmlite/ffi/build.py
2020-04-17T14:55:06,966   LLVM version... Traceback (most recent call last):
2020-04-17T14:55:06,967     File "/tmp/pip-wheel-8zs2sbun/llvmlite/ffi/build.py", line 106, in main_posix
2020-04-17T14:55:06,968       out = subprocess.check_output([llvm_config, '--version'])
2020-04-17T14:55:06,969     File "/usr/lib/python3.7/subprocess.py", line 395, in check_output
2020-04-17T14:55:06,969       **kwargs).stdout
2020-04-17T14:55:06,970     File "/usr/lib/python3.7/subprocess.py", line 472, in run
2020-04-17T14:55:06,971       with Popen(*popenargs, **kwargs) as process:
2020-04-17T14:55:06,972     File "/usr/lib/python3.7/subprocess.py", line 775, in __init__
2020-04-17T14:55:06,972       restore_signals, start_new_session)
2020-04-17T14:55:06,973     File "/usr/lib/python3.7/subprocess.py", line 1522, in _execute_child
2020-04-17T14:55:06,974       raise child_exception_type(errno_num, err_msg, err_filename)
2020-04-17T14:55:06,974   FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config': 'llvm-config'

2020-04-17T14:55:06,976   During handling of the above exception, another exception occurred:

2020-04-17T14:55:06,978   Traceback (most recent call last):
2020-04-17T14:55:06,978     File "/tmp/pip-wheel-8zs2sbun/llvmlite/ffi/build.py", line 168, in <module>
2020-04-17T14:55:06,979       main()
2020-04-17T14:55:06,979     File "/tmp/pip-wheel-8zs2sbun/llvmlite/ffi/build.py", line 158, in main
2020-04-17T14:55:06,980       main_posix('linux', '.so')
2020-04-17T14:55:06,980     File "/tmp/pip-wheel-8zs2sbun/llvmlite/ffi/build.py", line 109, in main_posix
2020-04-17T14:55:06,980       "to the path for llvm-config" % (llvm_config,))
2020-04-17T14:55:06,981   RuntimeError: llvm-config failed executing, please point LLVM_CONFIG to the path for llvm-config
2020-04-17T14:55:07,008   error: command '/usr/bin/python3' failed with exit status 1
2020-04-17T14:55:07,114   Building wheel for llvmlite (setup.py): finished with status 'error'
2020-04-17T14:55:07,115   ERROR: Failed building wheel for llvmlite
2020-04-17T14:55:07,116   Running setup.py clean for llvmlite
2020-04-17T14:55:07,116   Running command /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-8zs2sbun/llvmlite/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-8zs2sbun/llvmlite/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
2020-04-17T14:55:09,185   running clean
2020-04-17T14:55:09,187   'build/lib.linux-armv7l-3.7' does not exist -- can't clean it
2020-04-17T14:55:09,188   'build/bdist.linux-armv7l' does not exist -- can't clean it
2020-04-17T14:55:09,188   'build/scripts-3.7' does not exist -- can't clean it
2020-04-17T14:55:09,194   removing '/tmp/pip-wheel-8zs2sbun/llvmlite/llvmlite/__pycache__' (and everything under it)
2020-04-17T14:55:09,200   removing '/tmp/pip-wheel-8zs2sbun/llvmlite/__pycache__' (and everything under it)
2020-04-17T14:55:09,427 Failed to build llvmlite
2020-04-17T14:55:09,428 Cleaning up...
2020-04-17T14:55:09,430   Removing source in /tmp/pip-wheel-8zs2sbun/llvmlite
2020-04-17T14:55:09,509 Removed build tracker: '/tmp/pip-req-tracker-xno4si9p'
2020-04-17T14:55:09,513 ERROR: Failed to build one or more wheels
2020-04-17T14:55:09,514 Exception information:
2020-04-17T14:55:09,514 Traceback (most recent call last):
2020-04-17T14:55:09,514   File "/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/base_command.py", line 186, in _main
2020-04-17T14:55:09,514     status = self.run(options, args)
2020-04-17T14:55:09,514   File "/usr/local/lib/python3.7/dist-packages/pip/_internal/commands/wheel.py", line 189, in run
2020-04-17T14:55:09,514     "Failed to build one or more wheels"
2020-04-17T14:55:09,514 pip._internal.exceptions.CommandError: Failed to build one or more wheels

[nmstoker]

Thanks. There's a line about half way down that indicates it is having a problem finding llvm-config:

2020-04-17T14:55:06,974 FileNotFoundError: [Errno 2] No such file or directory: 'llvm-config': 'llvm-config'

So if, as detailed above, LLVM is installed first and then the environment variable is set as: LLVM_CONFIG=/usr/bin/llvm-config
then it looks like it would probably succeed.

[nmstoker]

Hi @bennuttall is there anything in particular I should do to help this along?

Is there some way I could get in touch to discuss it more? Happy to put in some effort to resolve this, just want to know I'm directly it properly 🙂

[bennuttall]

Hmm we don't currently have a mechanism for setting environment variables. The whole system is automated so obviously it's not a case of running a single command.

[nmstoker]

Ah, I hadn't appreciated that, but I can see why that would be a problem! Does that kind of mechanism seem completely out of scope or might it have benefit in enough cases to warrant looking into it further?

It sounds like I need to read up a bit more on how the piwheels process works behind the scenes!

[bennuttall]

Ah, I hadn't appreciated that, but I can see why that would be a problem! Does that kind of mechanism seem completely out of scope or might it have benefit in enough cases to warrant looking into it further?

Maybe. It certainly complicates things. Will need to run it by @waveform80.

It sounds like I need to read up a bit more on how the piwheels process works behind the scenes!

Take a look at the FAQs here: https://www.piwheels.org/faq.html and the docs: https://piwheels.readthedocs.io/en/latest/

[brifordwylie]

I understand this issue is closed (which is fine/good) but just in case someone finds this issue and wants to make recent versions of llvmliite work, for things liike fastparquet, here's the two liner...

$ sudo apt install llvm-9
$ LLVM_CONFIG=/usr/lib/llvm-9/bin/llvm-config pip3 install llvmlite

[djm1176]

For anyone who is trying to get llvmlite installed, try the the following:

$ sudo apt install llvm-9
$ LLVM_CONFIG=/usr/bin/llvm-9/bin/llvm-config-9 pip install llvmlite

Note the 2nd line has a 9 at the end of the directory