[Python] ERROR: Failed building wheel for pyarrow on FreeBSD

[ghost]

Describe the bug, including details regarding any error messages, version, and platform.

OS: FreeBSD 13.2
Python versions 3.9.17 / 3.10 / 3.11
Environments: venv
pip install pyarrow

Cmake Error at CMakeLists.txt:377 (target_link_libraries)
Target "arrow_python" links to:
Arrow::arrow_shared
but the target was not found.Possible reasons include:
* There is a typo in the targets name
* A find_package call is missing for an IMPORTED target.
* An ALIAS target is missing.
Cmake Generate step failed. Build files cannot be regenerated correctly.
error: command '/usr/local/bin/cmake' failed with exit code 1
ERROR: Failed building wheel for pyarrow
Failed to build pyarrow
ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects

Component(s)

Python

[vwbusguy]

Also seeing this in an Ubuntu 22.04.2 container environment built with current Jupyter upstream images. This was working 9 hours ago and started breaking in our CI about 2 hours ago against the same image spec, so it seems to be a change in something that happened today.

[kou]

You need to install Apache Arrow C++ before you install pyarrow.

[vwbusguy]

That wasn't true before today, but I tried that too and it complains about the CMake Arrow files already existing a bunch of times and then fails.

[kou]

I can't help you without any specific information such as Dockerfile and all full logs.

[vwbusguy]

Sure - compare Containerfile in main and develop branches. What is in main was working as of this morning. I added libarrow-dev to Containerfile in the develop branch: https://github.com/UCSB-PSTAT/pstat-174-container-image

[kou]

I can't see logs...

[kou]

Anyway, I think that @vwbusguy's problem isn't same as the original issue. Please open a new issue instead of reusing this.

[vwbusguy]

Literally getting the same error:

ERROR: Could not build wheels for pyarrow, which is required to install pyproject.toml-based projects

Are you not able to replicate the issue by building from our repo? If you want, I can file another issue that's not specific to FreeBSD, but I would be surprised if they aren't related.

      -- Could NOT find ArrowPython (missing: ArrowPython_DIR)
      -- Checking for module 'arrow-python'
      --   No package 'arrow-python' found
      CMake Error at /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
        Could NOT find ArrowPython (missing: ARROW_PYTHON_INCLUDE_DIR
        ARROW_PYTHON_LIB_DIR) (found version "12.0.1")
      Call Stack (most recent call first):
        /usr/share/cmake-3.22/Modules/FindPackageHandleStandardArgs.cmake:594 (_FPHSA_FAILURE_MESSAGE)
        cmake_modules/FindArrowPython.cmake:76 (find_package_handle_standard_args)
        CMakeLists.txt:231 (find_package)
      
      
      -- Configuring incomplete, errors occurred!
      See also "/tmp/pip-install-vw2qa89c/pyarrow_09ec236588224cdb9ec84329f87f1712/build/temp.linux-x86_64-cpython-311/CMakeFiles/CMakeOutput.log".
      error: command '/usr/bin/cmake' failed with exit code 1
      [end of output]

EDIT - OK, I'm done working for the day, but I'll bow out on providing more info on our end on this issue if you think it isn't relevant.

[ghost]

You need to install Apache Arrow C++ before you install pyarrow.

Did that, now it states: No module named 'cmake' exist.
did pip install cmake so running latest version but same error.
P.s. i build Apache Arrow C++ from source with -DARROW_PYTHON=ON .
if i do outside venv same problem as i stated at the beginning. obviously i was using --user option with pip.

[kou]

How did you install Apache Arrow C++?
You need to specify install prefix you specified when you run cmake ... -DCMAKE_INSTALL_PREFIX=XXX ... for Apache Arrow C++:

CMAKE_PREFIX_PATH=/path/to/prefix pip install pyarrow

See also: https://arrow.apache.org/docs/dev/developers/python.html

[assignUser]

Apache Arrow C++ from source with -DARROW_PYTHON=ON

That option was removed from the cpp lib in 10.0 when the python related code was moved into the python package. Is it possible that you have installed a version of the cpp lib that is != 12.0 (current pyarrow version)?

[kou]

(-DARROW_PYTHON=ON was deprecated but it's still available for keeping backward compatibility.)

[vwbusguy]

that you have installed a version of the cpp lib that is != 12.0 (current pyarrow version)

If arrow was installed through FreeBSD 13's ports, then it would have given the 8.0 version. In my case in the other issue, this error was from having a newer arrow (12.0) installed while trying to compile pyarrow 8.0. I assume the same error would work in both directions, if that were the case.

[ghost]

I gave up on it. even if i dont have arrow 8.0 from freebsd i still have same issues.
I might try later but now im just ok with it as if i need something i just use windows 11.

Under Arch it works fine but FreeBSD is not a Linux and there are some issues with some python packages trough pip so for now as long as i have W11 im ok have no time nor effort to engage into the battle with pip and all pythooon stuff :)