[CI][C++][Python] Verification jobs fail on ubuntu for Python due to missing libzstd

[raulcd]

Describe the bug, including details regarding any error messages, version, and platform.

Verification jobs for Python on Ubuntu 20.04 and 22.04 due to zstd not found:

verify-rc-source-python-linux-ubuntu-20.04-amd64
verify-rc-source-python-linux-ubuntu-22.04-amd64

CMake Warning at /tmp/arrow-HEAD.xCuHo/install/lib/cmake/Gandiva/FindzstdAlt.cmake:29 (find_package):
  By not providing "Findzstd.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "zstd", but
  CMake did not find one.

  Could not find a package configuration file provided by "zstd" with any of
  the following names:

    zstdConfig.cmake
    zstd-config.cmake

  Add the installation prefix of "zstd" to CMAKE_PREFIX_PATH or set
  "zstd_DIR" to a directory containing one of the above files.  If "zstd"
  provides a separate development package or SDK, be sure it has been
  installed.
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /tmp/arrow-HEAD.xCuHo/install/lib/cmake/Gandiva/GandivaConfig.cmake:64 (find_dependency)
  CMakeLists.txt:679 (find_package)


-- Checking for module 'libzstd'
--   No package 'libzstd' found
CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find zstdAlt (missing: ZSTD_LIB ZSTD_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /tmp/arrow-HEAD.xCuHo/install/lib/cmake/Gandiva/FindzstdAlt.cmake:124 (find_package_handle_standard_args)
  /usr/share/cmake-3.16/Modules/CMakeFindDependencyMacro.cmake:47 (find_package)
  /tmp/arrow-HEAD.xCuHo/install/lib/cmake/Gandiva/GandivaConfig.cmake:64 (find_dependency)
  CMakeLists.txt:679 (find_package)


-- Configuring incomplete, errors occurred!
See also "/arrow/python/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeOutput.log".
See also "/arrow/python/build/temp.linux-x86_64-cpython-38/CMakeFiles/CMakeError.log".
error: command '/usr/bin/cmake' failed with exit code 1
Failed to verify release candidate. See /tmp/arrow-HEAD.xCuHo for details.

It started failing since this PR was merged which seems could be related:

#35023

Those are the commits since the first failure:
6120345...6432a23

This should be fixed for 12.0.0 as this commit was the last one introduced before the feature freeze.

Component(s)

C++, Continuous Integration

[raulcd]

I have no idea what is happening but could be something to do with GandivaConfig.cmake.in temporary modifying CMAKE_MODULE_PATH?

if(DEFINED CMAKE_MODULE_PATH)
  set(GANDIVA_CMAKE_MODULE_PATH_OLD ${CMAKE_MODULE_PATH})
else()
  unset(GANDIVA_CMAKE_MODULE_PATH_OLD)
endif()
set(CMAKE_MODULE_PATH "${CMAKE_CURRENT_LIST_DIR}")

[raulcd]

@kou any idea?

[kou]

Oh, sorry...
I'll take a look at this.