-
Notifications
You must be signed in to change notification settings - Fork 847
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ARM support #13
Comments
...but manylinux tag does not support ARM? |
Thanks for making that great Python installer. When will there be a version for raspberry pi? Best regards Gerhard |
Pre-built wheel distributions which target ARM based Linux devices can't be done with current toolchains. The PEP (https://www.python.org/dev/peps/pep-0513/) which defines the manylinux tag recognizes only certain architectures (x86, i686). Please see also this issue: pypa/manylinux#84 However, Python's packaging system is evolving all the time and this might be possible at some point in the future. |
I hope that time will come fast. I have seen some packages that could be installed with pip Examp. https://github.com/samjabrahams/tensorflow-on-raspberry-pi Do they create their own toolchain? |
Yes, they have created their own special Docker image with all the correct dependencies and it runs on actual RPi hardware. Tensorflow itself has some tools to generate the package after the build. What I would need to make this happen:
This is not a task that can be done over a weekend, it will require most probably several weeks, maybe months, of work. |
Raspberry pi have a build system where they try to build all the pypi packages -- piwheels. piwheels website: https://www.piwheels.hostedpi.com/ They are receptive to PRs to include Debian dependencies here: https://github.com/bennuttall/piwheels/ I guess all your dependencies aren't in the Debian they use though? Also, compiling on real hardware can take a long time for big C++ libraries. |
In the latest raspian it automatically downloads the piwheels if they exist. |
I have no idea. I do not maintain those packages and I don't know how they were created. |
We haven't built Python 2 wheels - we might do in future. But there's not much future left. |
@bennuttall only use case I have for python2 wheels is for dependencies for Inkscape extensions, as Inkscape still uses python2. |
I try to build this project on QEMU based docker image, it quite slow and often make docker build timeout. it's really painful... |
when Can I "pip install opencv_contrib_python" in Raspberry Pi? |
PyPi does not support ARM wheels yet. I have no idea if it will get support for ARM wheels and if it does when it happens. |
It does! But I wouldn't recommend publishing Arm wheels, it's a bit messy. See piwheels/piwheels#66 Raspberry Pi users can use piwheels (configured by default in Raspbian) to get pre-built packages which work on all Raspberry Pi models. We just haven't built for Python 2 so far. |
I should add, I recently added the *-headless builds from this repo too (thanks to @skvark's fine work!), so you can "pip install opencv_contrib_python_headless" too. I watch this repo, so when there's a new release I try to add it to piwheels as soon as I'm able, but there'll usually be some delay. |
@bennuttall I am sorry that : |
@cooldeaaap1 |
@cooldeaaap1 when you say |
@waveform80
|
@skvark, thanks for the great work. I am confused when you say pypi doesn't support arm, it does. Unless I am misunderstanding you, if you look at something I published sslam-0.0.3-cp27-cp27mu-linux_armv7l.whl it is a wheel file for armv7. I did nothing special, here is my setup.py, just a simple c extension. I did build the package on a real raspberry pi, not travis, because that is only x86. Also, my stuff is trivial compared to opencv. Again, I might be misunderstanding your comment. 😄 |
@cooldeaaap1 could be a bug in 3.2 (there's something vaguely similar mentioned in opencv/opencv_contrib#1175) but if 3.3 works then I'd just stick with that (or later). @walchko See last FAQ at https://www.piwheels.org/faq.html - it's slightly out of date in that pypi.org is now the "next gen" PyPI that's talked about but the rest of it still stands (there's tons of ARM architectures and stuff compiled on Raspbian on a Pi isn't "true" ARMv7 because it still has to be compatible with the ARMv6 userland). |
@waveform80 I am sorry that : I had better to use "3.2.0.7" ,because I want to use a prject https://github.com/AirtestProject/Airtest |
Yeah, you can upload arbitrary ARM builds to PyPI nowadays. My comment was a bit misleading. What I mean is that there is no universal support for different ARM based boards. Yes, I could start building these packages for ARM on a Raspberry Pi but that does not mean that the packages are compatible with other boards. Therefore, it does not make any sense for this repository to start supporting one product line (RasPi) as there are many different ARM based boards. For example almost all mobile phones which are running Android are ARM based (!). It would be cool to run However, thanks to @bennuttall's and @waveform80's great work, you can use pre-built wheels on Raspberry Pi's. For more information, see https://www.piwheels.org/. |
@skvark I'm in the same situation described before. After installing I have tried to reinstall |
I don't maintain Raspberry Pi builds. Please open an issue to https://github.com/bennuttall/piwheels. |
According to travis-ci/travis-ci#2790, only cross-compiling to Android is available out of the box. https://github.com/Pi4J/pi4j/blob/master/.travis.yml is an example showing it's possible to cross-compile for Pi with a downloaded " As per Running Travis CI tests on ARM architecture | tomaz.me, it's also possible to run tests with Finally, https://bennuttall.com/piwheels-building-a-faster-python-package-repository-for-raspberry-pi-users/ and https://www.piwheels.hostedpi.com/ say that the piwheels project compile everything from PyPI (though, if the latter is correct, only for py3.4 and py3.5 -- looks strange, maybe the info is outdated). So all we need to do is provide an |
We attempt to build everything from PyPI (and keep everything that succeeds :). As OpenCV is a little more tricky (as you noted there's no sdist that works "out of the box") we handle that separately; specifically I watch this repo and when @skvark makes a new release I build them on my Pis and import them into piwheels when they're done (usually a couple of days later!). Tensorflow is also handled separately (specifically, the maintainers send Ben the pre-built wheels and he imports them). We only handle py34 and py35 at the moment because we (currently) only support Raspbian Jessie (py34) and Raspbian Stretch (py35). At some point, when I've got a spare couple of weekends, we'll add py27 to that mix but it's not a trivial addition (the build queue logic is already fairly complex and multiple major versions makes things even more complex). We probably won't be supporting anything more recent until Raspbian Buster makes an appearance (there was a vague attempt at supporting py36 via Ubuntu images on the builders, but that just caused more trouble than it was worth because some py36 users had built their own on Raspbian, some were using Ubuntu, and thus each had different lib versions installed so we're sticking strictly to Raspbian for now). |
Anyway, as noted above - if you've any issues with the piwheels builds, do feel free to open issues at https://github.com/bennuttall/piwheels |
To resolve the "cannot open shared object file" error, see https://blog.piwheels.org/how-to-work-out-the-missing-dependencies-for-a-python-package/ |
Travis now supports natively building for the ARM architecture: https://blog.travis-ci.com/2019-10-07-multi-cpu-architecture-support It's currently in alpha so be aware of possible problems. |
In particular, no |
I wonder, why not to use cross compilation? |
@skvark There is an option to get Arm servers via the WorksOnArm project. Please see the https://github.com/WorksOnArm/cluster/issues/209 for details. Tagging @vielmetti for visibility. |
There has also been some progress in getting wheels built for some Python projects for arm64 - notably this report from numpy at https://twitter.com/pypackages/status/1274444742621888513 . There's a couple of options through the Works on Arm effort mentioned above:
the preference for dedicated systems is for projects that have some compute density needed - ideally you'd want to build more than one package, or be working on a system that is a critical piece not yet ported but that lots of other things depend on, etc. Much of the time, Drone or Travis is a more effective way of getting CI done than dedicated systems. |
Hello, I guess this entire thread is yet focused on embedded devices such as raspberry pi however is there any further plan to support ARM64 architecture and corresponding wheels for new Apple silicon since there is no wheel compiled for macOS / arm64? Thanks! |
Please use the search function, there is a dedicated issue for arm64. |
ARM builds would be needed to support for example Raspberry Pi.
The text was updated successfully, but these errors were encountered: