[C++][Python] CMake cannot find Arrow/Arrow Python when building PyArrow

[asfimport]

When building on master yesterday the PyArrow built worked fine. Today there is an issue with CMake unable to find packages. See:

 

-- Finished CMake build and install for PyArrow C++
creating /Users/alenkafrim/repos/arrow/python/build/temp.macosx-12-arm64-3.9
-- Running cmake for PyArrow
cmake -DPYTHON_EXECUTABLE=/Users/alenkafrim/repos/pyarrow-dev-9/bin/python -DPython3_EXECUTABLE=/Users/alenkafrim/repos/pyarrow-dev-9/bin/python -DPYARROW_CPP_HOME=/Users/alenkafrim/repos/arrow/python/build/dist "" -DPYARROW_BUILD_CUDA=off -DPYARROW_BUILD_SUBSTRAIT=off -DPYARROW_BUILD_FLIGHT=on -DPYARROW_BUILD_GANDIVA=off -DPYARROW_BUILD_DATASET=on -DPYARROW_BUILD_ORC=off -DPYARROW_BUILD_PARQUET=on -DPYARROW_BUILD_PARQUET_ENCRYPTION=off -DPYARROW_BUILD_PLASMA=off -DPYARROW_BUILD_GCS=off -DPYARROW_BUILD_S3=on -DPYARROW_BUILD_HDFS=off -DPYARROW_USE_TENSORFLOW=off -DPYARROW_BUNDLE_ARROW_CPP=off -DPYARROW_BUNDLE_BOOST=off -DPYARROW_GENERATE_COVERAGE=off -DPYARROW_BOOST_USE_SHARED=on -DPYARROW_PARQUET_USE_SHARED=on -DCMAKE_BUILD_TYPE=release /Users/alenkafrim/repos/arrow/python
CMake Warning:
  Ignoring empty string ("") provided on the command line.




-- The C compiler identification is AppleClang 13.1.6.13160021
-- The CXX compiler identification is AppleClang 13.1.6.13160021
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- System processor: arm64
-- Performing Test CXX_SUPPORTS_ARMV8_ARCH
-- Performing Test CXX_SUPPORTS_ARMV8_ARCH - Success
-- Arrow build warning level: PRODUCTION
-- Configured for RELEASE build (set with cmake -DCMAKE_BUILD_TYPE={release,debug,...})
-- Build Type: RELEASE
-- Generator: Unix Makefiles
-- Build output directory: /Users/alenkafrim/repos/arrow/python/build/temp.macosx-12-arm64-3.9/release
-- Found Python3: /Users/alenkafrim/repos/pyarrow-dev-9/bin/python (found version "3.9.13") found components: Interpreter Development.Module NumPy 
-- Found Python3Alt: /Users/alenkafrim/repos/pyarrow-dev-9/bin/python  
CMake Error at /opt/homebrew/Cellar/cmake/3.24.1/share/cmake/Modules/CMakeFindDependencyMacro.cmake:47 (find_package):
  By not providing "FindArrow.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Arrow", but
  CMake did not find one.


  Could not find a package configuration file provided by "Arrow" with any of
  the following names:


    ArrowConfig.cmake
    arrow-config.cmake


  Add the installation prefix of "Arrow" to CMAKE_PREFIX_PATH or set
  "Arrow_DIR" to a directory containing one of the above files.  If "Arrow"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  build/dist/lib/cmake/ArrowPython/ArrowPythonConfig.cmake:54 (find_dependency)
  CMakeLists.txt:240 (find_package)

I did a clean built on the latest master. Am I missing some variables that need to be set after #13892 ?

I am calling cmake with these flags:

 

cmake -DCMAKE_INSTALL_PREFIX=$ARROW_HOME \
              -DCMAKE_INSTALL_LIBDIR=lib \
              -DCMAKE_BUILD_TYPE=debug \
              -DARROW_WITH_BZ2=ON \
              -DARROW_WITH_ZLIB=ON \
              -DARROW_WITH_ZSTD=ON \
              -DARROW_WITH_LZ4=ON \
              -DARROW_WITH_SNAPPY=ON \
              -DARROW_WITH_BROTLI=ON \
              -DARROW_PLASMA=OFF \
              -DARROW_PARQUET=ON \
              -DPARQUET_REQUIRE_ENCRYPTION=OFF \
              -DARROW_PYTHON=ON \
              -DARROW_FLIGHT=ON \
              -DARROW_JEMALLOC=OFF \
              -DARROW_S3=ON \
              -DARROW_GCS=OFF \
              -DARROW_BUILD_TESTS=ON \
              -DARROW_DEPENDENCY_SOURCE=AUTO \
              -DARROW_INSTALL_NAME_RPATH=OFF \
              -DARROW_EXTRA_ERROR_CONTEXT=ON \
              -GNinja \
              ..
        popd 

and building python with 

python setup.py build_ext --inplace 

 

 

cc @kou  

Reporter: Alenka Frim / @AlenkaF

Related issues:

• [C++][Docs] Update build document to follow new CMake package (is fixed by)
• [C++][Python][Docs] Provide instructions for "fixing" build environment issues (relates to)

_{Note: This issue was originally created as ARROW-17577. Please see the migration documentation for further details.}

[asfimport]

Kouhei Sutou / @kou:
Sorry for your inconvenient...
I think that you have PKG_CONFIG_PATH=$ARROW_HOME/lib/pkgconfig environment variable. You need to set CMAKE_PREFIX_PATH=$ARROW_HOME environment variable instead to find Arrow and other CMake packages.

I'll update related documents by ARROW-17575.

[asfimport]

Alenka Frim / @AlenkaF:
Thank you Kou, this fix solves the issue for me!