-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Python] Failed to install on aarch64 #25796
Comments
Kouhei Sutou / @kou: |
Matthew Meen: CMAKE_SYSTEM_PROCESSOR "aarch64" I've attached the output of cmake --system-information too. |
Kouhei Sutou / @kou: Could you also show full log when you run And could you try 1.0.0? |
Matthew Meen: (test_env) ubuntu@ip-10-143-19-162:/usr/local/test_env$ sudo pip install pyarrow==1.0.0 This is the full output for pip install pyarrow, which finds the latest as 0.17.1: pyarrow_017.txt I'll try cloning and building 1.0.0 directly shortly. |
Matthew Meen: |
Kouhei Sutou / @kou: |
Krisztian Szucs / @kszucs: |
Kouhei Sutou / @kou: |
Krisztian Szucs / @kszucs: |
Eamonn Nugent: CMake Error at cmake_modules/SetupCxxFlags.cmake:368 (message):
Unsupported arch flag: -march=.
Call Stack (most recent call first):
CMakeLists.txt:100 (include)
Is there a good workaround for this? Or should I wait for the next release |
Kouhei Sutou / @kou: I'll create a pull request to fix this. Workaround: |
Eamonn Nugent:
-- Looking for python3.8
-- Found Python lib /usr/local/lib/libpython3.8.so
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29")
-- Could NOT find Arrow (missing: Arrow_DIR)
-- Checking for module 'arrow'
-- No package 'arrow' found
CMake Error at /usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
Could NOT find Arrow (missing: ARROW_INCLUDE_DIR ARROW_LIB_DIR
ARROW_FULL_SO_VERSION ARROW_SO_VERSION)
Call Stack (most recent call first):
/usr/share/cmake-3.13/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
cmake_modules/FindArrow.cmake:412 (find_package_handle_standard_args)
cmake_modules/FindArrowPython.cmake:46 (find_package)
CMakeLists.txt:210 (find_package)
-- Configuring incomplete, errors occurred!
See also "/tmp/pip-install-av0q_7o5/pyarrow/build/temp.linux-aarch64-3.8/CMakeFiles/CMakeOutput.log".
error: command 'cmake' failed with exit status 1
----------------------------------------
ERROR: Failed building wheel for pyarrow
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow which use PEP 517 and cannot be installed directly
The command '/bin/sh -c PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow' returned a non-zero code: 1
Failing Dockerfile on an ARMv8 system: FROM python:3.8-buster
RUN apt update
RUN apt -y install gcc g++ cmake
RUN PYARROW_CMAKE_OPTIONS="-DARROW_ARMV8_ARCH=armv8-a" pip install pyarrow
If there's anything I can do to help debug, please, feel free to let me know.
|
Kouhei Sutou / @kou: You can do the latter by |
Matthew Meen:
@kou - thanks for the quick turnaround on this, hopefully it makes it into a release build soon. |
Kouhei Sutou / @kou: |
Kouhei Sutou / @kou: |
My team is attempting to migrate some workloads from x86-64 to ARM64, a blocker for this is PyArrow failing to install.
pip install pyarrow
fails to build the wheel as -march isn't correctly resolved:It's possible to get the build to work after editing
cmake_modules/SetupCxxFlags.cmake
to force ARROW_ARMV8_ARCH_FLAG to end up as an architecture such as 'armv8-a' - although some more elaborate logic is really needed to pick up the correct extensions.I can see that there have been a number of items discussed in the past both on Jira and in GitHub issues ranging from simple fixes to the cmake script to more elaborate fixes cross-product for arch detection - but I wasn't able to discern how the project wishes to proceed.
With AWS pushing their ARM-based instances heavily at this point I would advocate for picking a direction before an influx of new issues.
Environment: AWS m6g (ARM64 'Graviton2' CPU):
Features : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm lrcpc dcpop asimddp ssbs
CPU implementer : 0x41
CPU architecture: 8
CPU variant : 0x3
CPU part : 0xd0c
CPU revision : 1
OS: Linux version 5.3.0-1032-aws (buildd@bos02-arm64-053) (gcc version 7.5.0 (Ubuntu/Linaro 7.5.0-3ubuntu1
18.04)) #3418.04.2-Ubuntu SMP Fri Jul 24 10:03:03 UTC 2020Reporter: Matthew Meen
Assignee: Kouhei Sutou / @kou
Original Issue Attachments:
PRs and other links:
Note: This issue was originally created as ARROW-9744. Please see the migration documentation for further details.
The text was updated successfully, but these errors were encountered: