Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

offload: Fails to build libomp with missing and no known rule to make it #90707

Closed
sylvestre opened this issue May 1, 2024 · 9 comments
Closed

offload: Fails to build libomp with missing and no known rule to make it #90707

sylvestre opened this issue May 1, 2024 · 9 comments
Labels
build-problem cmake Build system in general and CMake in particular offload openmp

Comments

@sylvestre
Copy link
Collaborator

sylvestre commented May 1, 2024 •
edited by VoltrexKeyva
Loading

with

cmake -S ../llvm/ -B build-llvm \
-G Ninja \
-DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \
-DLLVM_VERSION_SUFFIX= \
-DCMAKE_SUPPRESS_REGENERATION=ON \
-DCMAKE_BUILD_TYPE=Release \
-DPACKAGE_VENDOR=Debian \
-DENABLE_LINKER_BUILD_ID=ON \
-DLLVM_TARGETS_TO_BUILD=Native \
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \
-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \
-DLLVM_ENABLE_PIC=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_BUILD_DOCS=OFF \
-DLLVM_INCLUDE_GO_TESTS=OFF \
-DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \
-DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \
-DLLVM_ENABLE_CURL=ON \
-DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \
-DCLANG_PLUGIN_SUPPORT=OFF \
-DCLANG_BUILD_EXAMPLES=OFF \
-DCLANG_DEFAULT_LINKER=ld \
-DCLANG_DEFAULT_RTLIB=libgcc \
-DCOMPILER_RT_BUILD_XRAY=OFF \
-DCOMPILER_RT_INCLUDE_TESTS=OFF \
-DCOMPILER_RT_USE_LIBCXX=OFF \
-DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \
-DLIBUNWIND_INSTALL_LIBRARY=OFF \
-DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \
-DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \
-DLIBOMP_ENABLE_RTTI=OFF \
-DLIBOMP_OMPT_SUPPORT=OFF \
-DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \
-DLLVM_LIBC_GPU_BUILD=ON \
 -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON  \
-DCLANG_ENABLE_BOOTSTRAP=ON

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:

-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?
@sylvestre sylvestre added cmake Build system in general and CMake in particular openmp build-problem offload labels May 1, 2024
@llvmbot
Copy link
Collaborator

llvmbot commented May 1, 2024

@llvm/issue-subscribers-openmp

Author: Sylvestre Ledru (sylvestre)

with ``` cmake -S ../llvm/ -B build-llvm \ -G Ninja \ -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \ -DLLVM_VERSION_SUFFIX= \ -DCMAKE_SUPPRESS_REGENERATION=ON \ -DCMAKE_BUILD_TYPE=Release \ -DPACKAGE_VENDOR=Debian \ -DENABLE_LINKER_BUILD_ID=ON \ -DLLVM_TARGETS_TO_BUILD=Native \ -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \ -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \ -DLLVM_ENABLE_PIC=ON \ -DLLVM_ENABLE_RTTI=ON \ -DLLVM_BUILD_DOCS=OFF \ -DLLVM_INCLUDE_GO_TESTS=OFF \ -DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \ -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ -DLLVM_ENABLE_CURL=ON \ -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \ -DCLANG_PLUGIN_SUPPORT=OFF \ -DCLANG_BUILD_EXAMPLES=OFF \ -DCLANG_DEFAULT_LINKER=ld \ -DCLANG_DEFAULT_RTLIB=libgcc \ -DCOMPILER_RT_BUILD_XRAY=OFF \ -DCOMPILER_RT_INCLUDE_TESTS=OFF \ -DCOMPILER_RT_USE_LIBCXX=OFF \ -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \ -DLIBUNWIND_INSTALL_LIBRARY=OFF \ -DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \ -DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \ -DLIBOMP_ENABLE_RTTI=OFF \ -DLIBOMP_OMPT_SUPPORT=OFF \ -DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \ -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \ -DLLVM_LIBC_GPU_BUILD=ON \ -DCMAKE_C_COMPILER_LAUNCHER=/opt/sccache//sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/opt/sccache//sccache -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON \ -DCLANG_ENABLE_BOOTSTRAP=ON ```

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:


-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?

@llvmbot
Copy link
Collaborator

llvmbot commented May 1, 2024

@llvm/issue-subscribers-offload

Author: Sylvestre Ledru (sylvestre)

with ``` cmake -S ../llvm/ -B build-llvm \ -G Ninja \ -DCMAKE_INSTALL_PREFIX=/usr/lib/llvm-19 \ -DLLVM_VERSION_SUFFIX= \ -DCMAKE_SUPPRESS_REGENERATION=ON \ -DCMAKE_BUILD_TYPE=Release \ -DPACKAGE_VENDOR=Debian \ -DENABLE_LINKER_BUILD_ID=ON \ -DLLVM_TARGETS_TO_BUILD=Native \ -DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra;lld;cross-project-tests;mlir;bolt;flang;polly;openmp;lldb" \ -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libc;libunwind;offload" \ -DLLVM_ENABLE_PIC=ON \ -DLLVM_ENABLE_RTTI=ON \ -DLLVM_BUILD_DOCS=OFF \ -DLLVM_INCLUDE_GO_TESTS=OFF \ -DLLVM_USE_RELATIVE_PATHS_IN_FILES=ON \ -DLLVM_TEMPORARILY_ALLOW_OLD_TOOLCHAIN=ON \ -DLLVM_ENABLE_CURL=ON \ -DLLVM_USE_RELATIVE_PATHS_IN_DEBUG_INFO=ON \ -DCLANG_PLUGIN_SUPPORT=OFF \ -DCLANG_BUILD_EXAMPLES=OFF \ -DCLANG_DEFAULT_LINKER=ld \ -DCLANG_DEFAULT_RTLIB=libgcc \ -DCOMPILER_RT_BUILD_XRAY=OFF \ -DCOMPILER_RT_INCLUDE_TESTS=OFF \ -DCOMPILER_RT_USE_LIBCXX=OFF \ -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \ -DLIBUNWIND_INSTALL_LIBRARY=OFF \ -DOPENMP_ENABLE_LIBOMP_PROFILING=OFF \ -DOPENMP_ENABLE_LIBOMPTARGET_PROFILING=OFF \ -DLIBOMP_ENABLE_RTTI=OFF \ -DLIBOMP_OMPT_SUPPORT=OFF \ -DCLANG_FORCE_MATCHING_LIBCLANG_SOVERSION=ON \ -DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \ -DLLVM_LIBC_GPU_BUILD=ON \ -DCMAKE_C_COMPILER_LAUNCHER=/opt/sccache//sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/opt/sccache//sccache -DLLVM_USE_LINKER=gold -DLLVM_BINUTILS_INCDIR=/usr/include/ -DLIBCXXABI_USE_LLVM_UNWINDER=ON -DLIBOMP_LIBFLAGS="-lm" -DOPENMP_USE_LLVM_UNWINDER=ON \ -DCLANG_ENABLE_BOOTSTRAP=ON ```

and

cmake --build build-llvm -j 13 --target stage2

The second stage fails with:


-- Configuring done (33.6s)
-- Generating done (1.3s)
-- Build files have been written to: /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins
[3159/3170] cd /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins && /usr/bin/cmake --build .
Change Dir: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-bins'

Run Build Command(s): /usr/bin/ninja -v
ninja: error: '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomp.so', needed by '/home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/lib/libomptarget.so.19.0', missing and no known rule to make it

FAILED: runtimes/runtimes-stamps/runtimes-build /home/sylvestre/dev/debian/pkg-llvm/llvm-toolchain/branches/llvm-project/build/build-llvm/runtimes/runtimes-stamps/runtimes-build

@jdoerfert @jhuber6 rings a bell ?

@sylvestre sylvestre mentioned this issue May 1, 2024
Merged
@jhuber6
Copy link
Contributor

jhuber6 commented May 1, 2024

I think it's expected to put openmp in runtimes instead. The old code all expected that openmp was in the same CMake invocation, but now that they're split it's not going to have access to the same targets.

@sylvestre
Copy link
Collaborator Author

ok, maybe we should detect this case and break cmake then :)

@jhuber6
Copy link
Contributor

jhuber6 commented May 1, 2024

I think we should be able to make this work in any case, since openmp should be a valid project now that it's just libomp.

@shiltian
Copy link
Contributor

shiltian commented May 1, 2024

I think we should be able to make this work in any case, since openmp should be a valid project now that it's just libomp.

Yeah, we still need to make sure libomp works as before, no matter how libomptarget changes.

@estewart08
Copy link
Contributor

One problem is with https://github.com/llvm/llvm-project/blob/main/llvm/runtimes/CMakeLists.txt#L561. This adds the runtimes to clang-bootstrap-deps. If openmp is on the LLVM_ENABLE_PROJECTS list instead of LLVM_ENABLE_RUNTIMES, then the configuration of the runtimes will occur before the openmp project. See #90793 for possible fix.

@sylvestre sylvestre mentioned this issue May 12, 2024
Closed
@jhuber6
Copy link
Contributor

jhuber6 commented May 16, 2024

Is this still an issue since making offload a no-op on unsupported architectures?

@jhuber6
Copy link
Contributor

jhuber6 commented May 29, 2024

Closing because it should be fixed and I haven't heard anything back.

@jhuber6 jhuber6 closed this as completed May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
build-problem cmake Build system in general and CMake in particular offload openmp
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@sylvestre @shiltian @estewart08 @jhuber6 @llvmbot