Skip to content

Simplify and document building offline/with vendored dependencies #19635

@blenderfreaky

Description

@blenderfreaky

When building in an offline environment, for example in a Nix sandbox or an airgapped environment, the build will try and fail to find several dependencies. Without internet access, FetchContent will always fail unless FETCHCONTENT_SOURCE_DIR_<name> is set.

@sarnex as discussed in #19347, here is a list of the dependencies I ran into and how they're currently handled:

opencl/CMakeLists.txt

ocl-icd: always uses FetchContent. There's a TODO comment for that already

ocl-headers: you can set OpenCL_HEADERS, but when I try that, it fails. (Cannot specify compile definitions for target "Headers" which is not built by this project.)

sycl/cmake/modules/FetchEmhash.cmake / xptifw/src/CMakeLists.txt

emhash: Should be able to find system install, though I haven't tested that as it isn't pre-packaged for nix

llvm/lib/SYCLLowerIR/CMakeLists.txt

vc-intrinsics: find_package is defined, but when not using that, there's seems to be two ways and I'm unsure which is preferred: adding the repo to llvm/projects/LLVM_EXTERNAL_PROJECTS, or setting LLVMGenXIntrinsics_SOURCE_DIR.

llvm/lib/SYCLNativeCPUUtils/CMakeLists.txt

oneapi-ck: you have to set NATIVECPU_OCK_USE_FETCHCONTENT=Off and OCK_SOURCE_DIR both. Sidenote, this also gives Calling FetchContent_Populate(oneapi-ck) is deprecated

llvm-spirv/CMakeLists.txt

spirv-headers: you have to set LLVM_EXTERNAL_SPIRV_HEADERS_SOURCE_DIR.

unified-runtime/source/common/CMakeLists.txt

unified-memory-framework: will use find_package but only if UR_USE_EXTERNAL_UMF=On is set

hdr_histogram: only used if UR_ENABLE_LATENCY_HISTOGRAM=On is set. Will always use FetchContent

unified-runtime/test/CMakeLists.txt

googletest: will always use FetchContent

This also applies to UMF (which uses FetchContent for googlebenchmark as well)

cmake/FetchLevelZero.cmake

level-zero: needs(?) to be specified via UR_LEVEL_ZERO_LOADER_LIBRARY, UR_LEVEL_ZERO_INCLUDE_DIR
compute-runtime: find_package works and is the default.

UMF

jemalloc only uses find_package on Windows, and always uses FetchContent otherwise.


I think it would be nice to have a documentation entry on these dependencies (maybe a brief section on vendored/offline builds).
Also, there's a good bit of inconsistency between these. Some packages default to find_package, some to FetchContent, some toggle based off a boolean flag (UR_USE_EXTERNAL_UMF).

Tangentially, what's the most stable way to get the git tags used (for use in an update script)? Is grepping for set(<DEP>_TAG ([a-z0-9]+)) or similar stable-ish?

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions