[Offload] Prevent by default passing CMAKE_CXX_FLAGS to GPU build

[jhuber6]

Summary:
People will use this to pass incompatible flags. The default behavior is
to inherit the top level options for all the runtimes. Some of these
aren't compatible. The cache file should put users in a most-useful
state, and this also serves as documentation. Just override it so users
need to explicit if they want a non-standard flag on their GPU build.

[llvmbot]

@llvm/pr-subscribers-offload

Author: Joseph Huber (jhuber6)

Changes

Summary:
People will use this to pass incompatible flags. The default behavior is
to inherit the top level options for all the runtimes. Some of these
aren't compatible. The cache file should put users in a most-useful
state, and this also serves as documentation. Just override it so users
need to explicit if they want a non-standard flag on their GPU build.

---

Full diff: https://github.com/llvm/llvm-project/pull/160881.diff

1 Files Affected:

• (modified) offload/cmake/caches/Offload.cmake (+6)

diff --git a/offload/cmake/caches/Offload.cmake b/offload/cmake/caches/Offload.cmake
index 3747a1d3eb299..09262f8ce6d1e 100644
--- a/offload/cmake/caches/Offload.cmake
+++ b/offload/cmake/caches/Offload.cmake
@@ -7,3 +7,9 @@ set(RUNTIMES_nvptx64-nvidia-cuda_CACHE_FILES "${CMAKE_SOURCE_DIR}/../libcxx/cmak
 set(RUNTIMES_amdgcn-amd-amdhsa_CACHE_FILES "${CMAKE_SOURCE_DIR}/../libcxx/cmake/caches/AMDGPU.cmake" CACHE STRING "")
 set(RUNTIMES_nvptx64-nvidia-cuda_LLVM_ENABLE_RUNTIMES "compiler-rt;libc;openmp;libcxx;libcxxabi" CACHE STRING "")
 set(RUNTIMES_amdgcn-amd-amdhsa_LLVM_ENABLE_RUNTIMES "compiler-rt;libc;openmp;libcxx;libcxxabi" CACHE STRING "")
+
+# Prevent people mistakenly sending CPU flags to the GPU build.
+set(RUNTIMES_nvptx64-nvidia-cuda_CMAKE_CXX_FLAGS "" CACHE STRING "")
+set(RUNTIMES_amdgcn-amd-amdhsa_CMAKE_CXX_FLAGS "" CACHE STRING "")
+set(RUNTIMES_nvptx64-nvidia-cuda_CMAKE_C_FLAGS "" CACHE STRING "")
+set(RUNTIMES_amdgcn-amd-amdhsa_CMAKE_C_FLAGS "" CACHE STRING "")

[shiltian]

Based on my experience, many people do use this flag globally to pass compile flags, even for GPU targets, not to build LLVM though. I understand is not great and do support this change, but I think it might be better to document this.

[jhuber6]

Based on my experience, many people do use this flag globally to pass compile flags, even for GPU targets, not to build LLVM though. I understand is not great and do support this change, but I think it might be better to document this.

My thinking is that the vast majority of people are not intending this flag to show up on the GPU portion, and if they want it to they can override this. I really need to write some more comprehensive runtimes documentation, what we have now is certainly lacking.

[jplehr]

I agree that this should be documented properly. And I personally would say that the update to the documentation should be part of this PR.

[jhuber6]

I agree that this should be documented properly. And I personally would say that the update to the documentation should be part of this PR.

From https://openmp.llvm.org/SupportAndFAQ.html#q-how-to-build-an-openmp-gpu-offload-capable-compiler

Support for the device library comes from a separate build of the OpenMP library that targets the GPU architecture. Building it requires enabling the runtime targets, or setting the target manually when doing a standalone build. This is done with the LLVM_RUNTIME_TARGETS option and then enabling the OpenMP runtime for the GPU target via RUNTIMES_LLVM_ENABLE_RUNTIMES. It’s possible to set different flags for each device library by using RUNTIMES_CMAKE_CXX_FLAGS. Refer to the cache file for the specific invocation.

[jhuber6]

The goal of this is to set a reasonable default if someone uses the cache file, and to document the flags used if someone's thinking of modifying them.