Improve installation hygiene of built and header-only dependencies

[vyasr]

Description

Historically we have taken a very piecemeal approach to controlling when we build some dependency statically or dynamically, and we've patched out header-only dependencies from our installation on a case-by-case basis. This PR standardizes that process with a few common controls. Specifically, we assume that there are almost no practical use cases where we want a CPM-downloaded package to be built as a shared library; if we want to dynamically link, it will be because we found the shared library on the system (e.g. a conda environment). Further, we assume that installation of private header-only deps is almost never desirable, so we should always EXCLUDE_FROM_ALL all private dependencies.

Since rmm and rapids-logger are public dependencies, handling them is a bit more challenging. If they are downloaded as part of the build, they must also be installed along with the necessary CMake, headers, and libraries. In the case where we are building a shared libcudf.so but statically linking in the dependencies, however, we instead subsume those dependencies with whole-archive linking and revendor the headers directly.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[bdice]

All seems fine to me. Devcontainers and pip/conda will be verified by CI, but it'd be good to make sure that Spark builds work with these changes (sometimes that CI job is failing for other reasons).

[jameslamb]

Left a suggestion, but otherwise approving.

[coderabbitai]

📝 Walkthrough

Summary by CodeRabbit

• Chores
  • Refactored CMake build system for improved control over static and shared dependencies.
  • Added flexible build options for managing dependency installation and export behavior.
  • Enhanced dependency configuration for test and Java/JNI builds.
  • Optimized dependency linking mechanism for shared library builds.

Walkthrough

This PR refactors cudf's CMake build system to centralize static/shared dependency control. It introduces a 3-state CUDF_BUILD_STATIC_DEPS cache variable (ON/OFF/FORCE) that drives derived configuration of dependency installation and CPM packaging behavior, parameterizes thirdparty dependency functions to accept shared/static and EXCLUDE_FROM_ALL controls, reworks the cudf target to absorb static dependencies via whole-archive while promoting their transitive public dependencies, and updates build-specific configurations (tests, Java) to opt into the new strategy.

Changes

Centralized Dependency Control & Parameterization

Layer / File(s)	Summary
Cache Variable Foundation cpp/CMakeLists.txt	Introduces CUDF_BUILD_STATIC_DEPS (3-state: ON/OFF/FORCE) as the primary control, derives CUDA_STATIC_RUNTIME, CUDF_INSTALL_LIBRARY_DEPS, CUDF_EXCLUDE_DEPS_FROM_ALL, and CUDF_DEPS_BUILD_SHARED as centralized flags.
CPM Behavior & Conditional Export cpp/CMakeLists.txt, cpp/cmake/thirdparty/get_arrow.cmake, cpp/cmake/thirdparty/get_nvtx.cmake	Enables CPM_DOWNLOAD_ALL when CUDF_BUILD_STATIC_DEPS is FORCE; updates rapids_logger and nvtx3 to conditionally export packages and exclude from install based on CUDF_INSTALL_LIBRARY_DEPS.
RMM Parameterization cpp/cmake/thirdparty/get_rmm.cmake	Converts find_and_configure_rmm to accept BUILD_SHARED and EXCLUDE_FROM_ALL, adjusts CPM arguments and target type detection, propagates rmm_SOURCE_DIR/rmm_BINARY_DIR to parent scope.
NVComp Installation & Target Selection cpp/cmake/thirdparty/get_nvcomp.cmake	Adds INSTALL_LIBRARY option; conditionally installs artifacts; exposes both nvcomp::nvcomp and nvcomp::nvcomp_static targets; selects appropriate target based on build mode.
Standard Dependency Threading cpp/cmake/thirdparty/get_{arrow,croaring,dlpack,flatbuffers,kvikio,zstd,nanoarrow}.cmake, cpp/cmake/thirdparty/get_{cucollections,thread_pool}.cmake	Consistently parameterize dependency functions to accept EXCLUDE_FROM_ALL and/or BUILD_SHARED; remove local _exclude_from_all helpers; thread through CUDF_EXCLUDE_DEPS_FROM_ALL and CUDF_DEPS_BUILD_SHARED; nanoarrow selects shared/static target variant.
CUDF Target Linkage & Whole-Archive Absorption cpp/CMakeLists.txt	Reworks cudf target to absorb static rmm/rapids_logger via whole-archive, promotes absorbed deps' public transitive dependencies into cudf's PUBLIC interface with filtering, merges export metadata into cudf-exports, strips nvtx3 from rmm when absorbed.
Header Installation Fallback cpp/CMakeLists.txt	Adds header-only installation for rmm, rapids_logger, and nvtx3 when CUDF_INSTALL_LIBRARY_DEPS is OFF, supporting standalone DSO deployments.
Build-Specific Overrides cpp/tests/CMakeLists.txt, java/src/main/native/CMakeLists.txt	Test build disables library installation and marks dependencies EXCLUDE_FROM_ALL; Java build enables library installation for JNI static linking.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Improve installation hygiene of built and header-only dependencies' directly and clearly describes the main objective of the PR: standardizing and improving how dependencies are installed, which is the primary focus of all the changes across multiple CMake configuration files.
Description check	✅ Passed	The description comprehensively explains the rationale, approach, and implementation details of standardizing dependency controls, covering the key changes for static builds, private header-only dependencies, and public dependencies (rmm/rapids-logger) with whole-archive absorption.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cpp/CMakeLists.txt`:
- Around line 1101-1112: The code is mutating the actual rmm::rmm target
(_rmm_real / _rmm_libs) to remove nvtx3::nvtx3-cpp; instead, stop altering the
real RMM target and perform the nvtx3 filter only on the interface list you plan
to promote into cuDF (e.g., build a separate _iface_libs variable from
get_target_property(... INTERFACE_LINK_LIBRARIES) and list(REMOVE_ITEM
_iface_libs nvtx3::nvtx3-cpp) before using set_property or installing/setting
cuDF's INTERFACE_LINK_LIBRARIES), leaving rmm::rmm and its ALIASED_TARGET
untouched so other in-tree targets still see nvtx headers.
- Around line 320-323: CUDF_BUILD_STATIC_DEPS currently sets CPM_DOWNLOAD_ALL ON
only when equal to "FORCE", but because CPM_DOWNLOAD_ALL is a CMake cache
variable it must be explicitly reset when not in FORCE mode; update the CMake
logic around CUDF_BUILD_STATIC_DEPS to add an else() branch that sets
CPM_DOWNLOAD_ALL to OFF (mirroring how CUDA_STATIC_RUNTIME is handled) so
previously configured FORCE builds don’t leave CPM_DOWNLOAD_ALL cached ON after
switching to ON or OFF.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 963eee15-17d8-48e3-b7e5-f35aeb338748

📥 Commits

Reviewing files that changed from the base of the PR and between 64a3109 and 7890c4a.

📒 Files selected for processing (15)

• cpp/CMakeLists.txt
• cpp/cmake/thirdparty/get_arrow.cmake
• cpp/cmake/thirdparty/get_croaring.cmake
• cpp/cmake/thirdparty/get_cucollections.cmake
• cpp/cmake/thirdparty/get_dlpack.cmake
• cpp/cmake/thirdparty/get_flatbuffers.cmake
• cpp/cmake/thirdparty/get_kvikio.cmake
• cpp/cmake/thirdparty/get_nanoarrow.cmake
• cpp/cmake/thirdparty/get_nvcomp.cmake
• cpp/cmake/thirdparty/get_nvtx.cmake
• cpp/cmake/thirdparty/get_rmm.cmake
• cpp/cmake/thirdparty/get_thread_pool.cmake
• cpp/cmake/thirdparty/get_zstd.cmake
• cpp/tests/CMakeLists.txt
• java/src/main/native/CMakeLists.txt

[vyasr]

/merge

Stale, addressed.