Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LLVM12: Building LLVM+clang for llvmtools fails #17

Closed
dslm4515 opened this issue Jun 28, 2021 · 14 comments
Closed

LLVM12: Building LLVM+clang for llvmtools fails #17

dslm4515 opened this issue Jun 28, 2021 · 14 comments
Assignees
Labels
bug Something isn't working

Comments

@dslm4515
Copy link
Owner

is this issue related to build llvm in llvmtools?

For LLVM 12, I just fine while building. See #12 (comment)

Now moving to chroot steps. I'll report if an issue appeared.

Originally posted by @owl4ce in #14 (comment)

So I switched back to building CMLFS with LLVM12. Still not successful. I accidentally used ninja instead of make:

 [1619/3634] Building CXX object projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.o
FAILED: projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.o 
/cgnutools/bin/x86_64-cmlfs-linux-musl-clang++  -DHAVE_RPC_XDR_H=0 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -Iprojects/compiler-rt/lib/sanitizer_common -I../projects/compiler-rt/lib/sanitizer_common -Iinclude -I../include -I../projects/compiler-rt/lib/sanitizer_common/.. -g -g1  -fPIC -fvisibility-inlines-hidden -Werror=date-time -Werror=unguarded-availability-new -Wall -Wextra -Wno-unused-parameter -Wwrite-strings -Wcast-qual -Wmissing-field-initializers -pedantic -Wno-long-long -Wimplicit-fallthrough -Wcovered-switch-default -Wno-noexcept-type -Wnon-virtual-dtor -Wdelete-non-virtual-dtor -Wsuggest-override -Wstring-conversion -fdiagnostics-color -ffunction-sections -fdata-sections -Wall -std=c++14 -Wno-unused-parameter -O3 -DNDEBUG    -m64 -fPIC -fno-builtin -fno-exceptions -fomit-frame-pointer -funwind-tables -fno-stack-protector -fno-sanitize=safe-stack -fvisibility=hidden -fno-lto -O3 -gline-tables-only -Wno-gnu -Wno-variadic-macros -Wno-c99-extensions -nostdinc++ -fno-rtti -Wframe-larger-than=570 -Wglobal-constructors -std=c++14 -MD -MT projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.o -MF projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.o.d -o projects/compiler-rt/lib/sanitizer_common/CMakeFiles/RTSanitizerCommon.x86_64.dir/sanitizer_linux.cpp.o -c ../projects/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp
../projects/compiler-rt/lib/sanitizer_common/sanitizer_linux.cpp:30:10: fatal error: 'asm/param.h' file not found
#include <asm/param.h>
         ^~~~~~~~~~~~~
1 error generated.
[1620/3634] Building CXX object utils/yaml-bench/CMakeFiles/yaml-bench.dir/YAMLBench.cpp.o
ninja: build stopped: subcommand failed.
@dslm4515 dslm4515 changed the title LLVM12: Building LLVM+clang for llvmtools fails LLVM11: Building LLVM+clang for llvmtools fails Jun 28, 2021
@dslm4515 dslm4515 changed the title LLVM11: Building LLVM+clang for llvmtools fails LLVM12: Building LLVM+clang for llvmtools fails Jun 28, 2021
@dslm4515
Copy link
Owner Author

I used this command to configure the source:

CC=x86_64-cmlfs-linux-musl-clang \
CXX=x86_64-cmlfs-linux-musl-clang++ \
cmake -B build -G Ninja       \
      -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=/llvmtools \
      -DLLVM_LINK_LLVM_DYLIB=ON \
      -DLLVM_BUILD_LLVM_DYLIB=ON \
      -DLLVM_BUILD_TESTS=OFF \
      -DLLVM_ENABLE_LIBEDIT=OFF  \
      -DLLVM_ENABLE_LIBXML2=OFF \
      -DLLVM_ENABLE_LIBCXX=ON \
      -DLLVM_INCLUDE_GO_TESTS=OFF  \
      -DLLVM_INCLUDE_TESTS=OFF \
      -DLLVM_INCLUDE_DOCS=OFF \
      -DLLVM_INCLUDE_EXAMPLES=OFF \
      -DLLVM_INCLUDE_BENCHMARKS=OFF \
      -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-musl  \
      -DLLVM_HOST_TRIPLE="x86_64-pc-linux-musl" \
      -DLLVM_TARGET_ARCH="X86" \
      -DLLVM_TARGETS_TO_BUILD="X86"  \
      -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE="x86_64-pc-linux-musl" \
      -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
      -DCOMPILER_RT_BUILD_XRAY=OFF \
      -DCOMPILER_RT_BUILD_PROFILE=OFF \
      -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
      -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON \
      -DCLANG_DEFAULT_CXX_STDLIB=libc++ \
      -DCLANG_DEFAULT_UNWINDLIB=libunwind \
      -DCLANG_DEFAULT_RTLIB=compiler-rt \
      -DCLANG_DEFAULT_LINKER="/llvmtools/bin/ld.lld" \
      -DDEFAULT_SYSROOT="/llvmtools" \
      -DLLVM_ENABLE_LLD=ON \
      -DLLVM_ENABLE_RTTI=ON \
      -DLLVM_ENABLE_ZLIB=OFF \
      -DBacktrace_INCLUDE_DIR="/llvmtools/include" \
      -DBacktrace_LIBRARY="/llvmtools/lib/libexecinfo.so" \
      -DCMAKE_CXX_COMPILER_AR="/cgnutools/bin/llvm-ar" \
      -DCMAKE_C_COMPILER_AR="/cgnutools/bin/llvm-ar" \
      -DCMAKE_CXX_COMPILER_RANLIB="/cgnutools/bin/llvm-ranlib" \
      -DCMAKE_C_COMPILER_RANLIB="/cgnutools/bin/llvm-ranlib" \
      -DCMAKE_INSTALL_OLDINCLUDEDIR="/llvmtools/include" \
      -DCMAKE_LINKER="/cgnutools/bin/ld.lld" \
      -DCMAKE_NM="/cgnutools/bin/llvm-nm" \
      -DCMAKE_OBJCOPY="/cgnutools/bin/llvm-objcopy" \
      -DCMAKE_READELF="/cgnutools/bin/llvm-readelf" \
      -DCMAKE_STRIP="/cgnutools/bin/llvm-strip" \
      -DICONV_LIBRARY_PATH="/llvmtools/lib/libc.so"

@dslm4515 dslm4515 self-assigned this Jun 28, 2021
@dslm4515 dslm4515 added the bug Something isn't working label Jun 28, 2021
@owl4ce
Copy link
Contributor

owl4ce commented Jun 28, 2021

My complete preference. I wrap every command with time for every build and install 😄

popd

# Rename the LLVM source directory to ${LLVMSRC}.
mv -v llvm-12.0.0.src "$LLVMSRC" && pushd "$LLVMSRC"

# Decompress clang, lld, and compiler-rt to correct directories.
pushd "${LLVMSRC}/projects/" && \
    tar xf ../../pkgs/compiler-rt-12.0.0.src.tar.xz && mv -v compiler-rt-12.0.0.src compiler-rt
popd

pushd "${LLVMSRC}/tools/" && \
    tar xf ../../pkgs/clang-12.0.0.src.tar.xz && mv -v clang-12.0.0.src clang
    tar xf ../../pkgs/lld-12.0.0.src.tar.xz   && mv -v lld-12.0.0.src lld
popd

# Apply patches (from Void Linux).
../patches/llvm-12/stage1-appatch

# Disable sanitizers for musl, fixing "early build failure".
sed -i 's|set(COMPILER_RT_HAS_SANITIZER_COMMON TRUE)|set(COMPILER_RT_HAS_SANITIZER_COMMON FALSE)|' \
projects/compiler-rt/cmake/config-ix.cmake

# Set default compiler to new symlink from Stage-0 Clang/LLVM.
# Sets C and C++ compiler's build flags to reduce debug symbols.
CC="${CMLFS_TARGET}-clang" CFLAGS="-g -g1"
CXX="${CMLFS_TARGET}-clang++" CXXFLAGS="-g -g1"
export CC CXX CFLAGS CXXFLAGS

# Update host/target triple detection.
cp -v ../files/config.guess cmake/

# Configure source.
cmake -B build \
    -DCMAKE_BUILD_TYPE=Release                                  \
    -DCMAKE_INSTALL_PREFIX="/llvmtools"                         \
    -DLLVM_INSTALL_TOOLCHAIN_ONLY=ON                            \
    -DLLVM_LINK_LLVM_DYLIB=ON                                   \
    -DLLVM_BUILD_LLVM_DYLIB=ON                                  \
    -DLLVM_BUILD_TESTS=OFF                                      \
    -DLLVM_ENABLE_LIBEDIT=OFF                                   \
    -DLLVM_ENABLE_LIBXML2=OFF                                   \
    -DLLVM_ENABLE_LIBCXX=ON                                     \
    -DLLVM_INCLUDE_GO_TESTS=OFF                                 \
    -DLLVM_INCLUDE_TESTS=OFF                                    \
    -DLLVM_INCLUDE_DOCS=OFF                                     \
    -DLLVM_INCLUDE_EXAMPLES=OFF                                 \
    -DLLVM_INCLUDE_BENCHMARKS=OFF                               \
    -DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-pc-linux-musl"         \
    -DLLVM_HOST_TRIPLE="x86_64-pc-linux-musl"                   \
    -DLLVM_TARGET_ARCH="X86"                                    \
    -DLLVM_TARGETS_TO_BUILD="X86"                               \
    -DCOMPILER_RT_DEFAULT_TARGET_TRIPLE="x86_64-pc-linux-musl"  \
    -DCOMPILER_RT_BUILD_SANITIZERS=OFF                          \
    -DCOMPILER_RT_BUILD_XRAY=OFF                                \
    -DCOMPILER_RT_BUILD_PROFILE=OFF                             \
    -DCOMPILER_RT_BUILD_LIBFUZZER=OFF                           \
    -DCOMPILER_RT_USE_BUILTINS_LIBRARY=ON                       \
    -DCLANG_DEFAULT_CXX_STDLIB=libc++                           \
    -DCLANG_DEFAULT_UNWINDLIB=libunwind                         \
    -DCLANG_DEFAULT_RTLIB=compiler-rt                           \
    -DCLANG_DEFAULT_LINKER="/llvmtools/bin/ld.lld"              \
    -DDEFAULT_SYSROOT="/llvmtools"                              \
    -DLLVM_ENABLE_LLD=ON                                        \
    -DLLVM_ENABLE_RTTI=ON                                       \
    -DLLVM_ENABLE_ZLIB=OFF                                      \
    -DBacktrace_INCLUDE_DIR="/llvmtools/include"                \
    -DBacktrace_LIBRARY="/llvmtools/lib/libexecinfo.so"         \
    -DCMAKE_CXX_COMPILER_AR="/cgnutools/bin/llvm-ar"            \
    -DCMAKE_C_COMPILER_AR="/cgnutools/bin/llvm-ar"              \
    -DCMAKE_CXX_COMPILER_RANLIB="/cgnutools/bin/llvm-ranlib"    \
    -DCMAKE_C_COMPILER_RANLIB="/cgnutools/bin/llvm-ranlib"      \
    -DCMAKE_INSTALL_OLDINCLUDEDIR="/llvmtools/include"          \
    -DCMAKE_LINKER="/cgnutools/bin/ld.lld"                      \
    -DCMAKE_NM="/cgnutools/bin/llvm-nm"                         \
    -DCMAKE_OBJCOPY="/cgnutools/bin/llvm-objcopy"               \
    -DCMAKE_READELF="/cgnutools/bin/llvm-readelf"               \
    -DCMAKE_STRIP="/cgnutools/bin/llvm-strip"                   \
    -DICONV_LIBRARY_PATH="/llvmtools/lib/libc.so"

# Build.
time { make -C build; }

# Install, but some binaries are not installed, make sure to install important binaries.
time {
    pushd build/ && \
        cmake -DCMAKE_INSTALL_PREFIX="/llvmtools" -P cmake_install.cmake && \
        cp -v bin/llvm-as /llvmtools/bin/                                && \
        cp -v bin/llvm-readobj /llvmtools/bin/                           && \
        ln -sv llvm-readobj /llvmtools/bin/llvm-readelf                  && \
    popd && rm -rf build
}

# Since Binutils won't be used, create a symlink to LLVM tools and set lld as default toolchain linker.
for B in as ar ranlib readelf nm objcopy objdump size strip; do
    ln -sv llvm-${B} /llvmtools/bin/${B}
done
ln -sv lld /llvmtools/bin/ld

# Configure Stage-1 Clang to build binaries with "/llvmtools/lib/ld-musl-x86_64.so.1" instead of "/lib/ld-musl-x86_64.so.1".
ln -sv clang-12 /llvmtools/bin/x86_64-pc-linux-musl-clang   && \
ln -sv clang-12 /llvmtools/bin/x86_64-pc-linux-musl-clang++ && \
cat > /llvmtools/bin/x86_64-pc-linux-musl.cfg << "EOF"
-Wl,-dynamic-linker /llvmtools/lib/ld-musl-x86_64.so.1
EOF

# Unset exported flags and set the new PATH since "/cgnutools" isn't used anymore.
unset B CFLAGS CXXFLAGS
export PATH="/llvmtools/bin:/llvmtools/usr/bin:/bin:/usr/bin"

# Configure new Stage-1 Clang/LLVM environment.
sed -i "s|PATH=.*|PATH=\"${PATH}\"|" ~/.bashrc
sed -i '/unset CFLAGS CXXFLAGS/d' ~/.bashrc
cat >> ~/.bashrc << "EOF"
# Compiler environment
CC="${TARGET_TRUPLE}-clang"
CXX="${TARGET_TRUPLE}-clang++"
AR="llvm-ar"
AS="llvm-as"
RANLIB="llvm-ranlib"
LD="ld.lld"
STRIP="llvm-strip"
export CC CXX AR AS RANLIB LD STRIP
EOF
source ~/.bashrc

popd

I created script like you for patch from Void Linux.

#!/usr/bin/env bash
# Stage-1 Clang/LLVM.

pushd "${LLVMSRC}/projects/" && \
    for P in \
        compiler-rt-aarch64-ucontext.patch \
        compiler-rt-sanitizer-ppc64-musl.patch \
        compiler-rt-size_t.patch \
        compiler-rt-xray-ppc64-musl.patch
    do patch -Np1 -i ../../patches/llvm-12/${P}
    done; unset P
popd

pushd "${LLVMSRC}/tools/" && \
    for P in \
        clang-001-fix-unwind-chain-inclusion.patch \
        clang-002-add-musl-triples.patch \
        clang-003-ppc64-dynamic-linker-path.patch \
        clang-004-ppc64-musl-elfv2.patch
    do patch -Np1 -i ../../patches/llvm-12/${P}
    done; unset P
popd

pushd "${LLVMSRC}/../" && \
    for P in \
        llvm-001-musl.patch \
        llvm-002-musl-ppc64-elfv2.patch \
        llvm-003-ppc-secureplt.patch \
        llvm-004-override-opt.patch \
        llvm-005-ppc-bigpic.patch \
        llvm-006-aarch64-mf_exec.patch
    do patch  -Np1 -i ./patches/llvm-12/${P}
    done; unset P
popd

@owl4ce
Copy link
Contributor

owl4ce commented Jun 28, 2021

Also I use the latest config.guess file.
https://git.savannah.gnu.org/cgit/config.git/plain/config.guess

@owl4ce
Copy link
Contributor

owl4ce commented Jun 28, 2021

Have you already disabled the sanitizers?

# Disable sanitizers for musl, fixing "early build failure".
sed -i 's|set(COMPILER_RT_HAS_SANITIZER_COMMON TRUE)|set(COMPILER_RT_HAS_SANITIZER_COMMON FALSE)|' \
projects/compiler-rt/cmake/config-ix.cmake

@dslm4515
Copy link
Owner Author

Have you already disabled the sanitizers?

# Disable sanitizers for musl, fixing "early build failure".
sed -i 's|set(COMPILER_RT_HAS_SANITIZER_COMMON TRUE)|set(COMPILER_RT_HAS_SANITIZER_COMMON FALSE)|' \
projects/compiler-rt/cmake/config-ix.cmake

I thought I did... I’ll do it again!

@dslm4515
Copy link
Owner Author

@owl4ce , what software sorcery is this?? Lol. WOW. I used the cmake command you posted and LLVM built without error!!!

@owl4ce
Copy link
Contributor

owl4ce commented Jun 29, 2021

I'm going to rebuild llvmtools with purpose removes this line when configuring LLVM.

-DLLVM_INSTALL_TOOLCHAIN_ONLY=ON

@owl4ce
Copy link
Contributor

owl4ce commented Jul 4, 2021

I wonder, this is ambigous flags. May I ask, is this correct?

-DDEFAULT_SYSROOT="/llvmtools" \

-DDEFAULT_SYSROOT="/usr"

CMLFS/3-chroot/026-llvm

Lines 87 to 91 in 52fd160

-DCMAKE_AR="/llvmtools/bin/llvm-ar" \
-DCMAKE_INSTALL_OLDINCLUDEDIR="/usr/include" \
-DCMAKE_LINKER="/llvmtools/bin/ld.lld" \
-DCMAKE_AR="/llvmtools/bin/llvm-ar" \
-DCMAKE_LINKER="/llvmtools/bin/ld.lld" \

@dslm4515
Copy link
Owner Author

dslm4515 commented Jul 4, 2021

-DDEFAULT_SYSROOT="/llvmtools" \

No, I believe this does not set the ‘root’ for the llvmtools tools to resolve its libraries and binaries. This is the root that LLVM will install and resolve its libraries and binaries... I have yet to test to confirm. Should be set to /usr in chroot.

CMLFS/3-chroot/026-llvm

Lines 87 to 91 in 52fd160

-DCMAKE_AR="/llvmtools/bin/llvm-ar" \
-DCMAKE_INSTALL_OLDINCLUDEDIR="/usr/include" \
-DCMAKE_LINKER="/llvmtools/bin/ld.lld" \
-DCMAKE_AR="/llvmtools/bin/llvm-ar" \
-DCMAKE_LINKER="/llvmtools/bin/ld.lld" \

This is set to force the build to use LLVM’s binary tools. This is important as I chose to install binutils and GCC in /usr Instead of /opt/gnu and in /llvmtools instead of /llvmtools/gnu

@owl4ce
Copy link
Contributor

owl4ce commented Jul 5, 2021

-DDEFAULT_SYSROOT="/llvmtools" \

No, I believe this does not set the ‘root’ for the llvmtools tools to resolve its libraries and binaries. This is the root that LLVM will install and resolve its libraries and binaries... I have yet to test to confirm. Should be set to /usr in chroot.

So, you should remove one of two DEFAULT_SYSROOT. But I still don't know why double AR and LINKER flags? While in llvmtools is uses like below.

-DCMAKE_CXX_COMPILER_AR="/cgnutools/bin/llvm-ar" \
-DCMAKE_C_COMPILER_AR="/cgnutools/bin/llvm-ar" \

@dslm4515
Copy link
Owner Author

dslm4515 commented Jul 5, 2021

So, you should remove one of two DEFAULT_SYSROOT.

yes, you are right. I did not notice.

@dslm4515
Copy link
Owner Author

dslm4515 commented Jul 5, 2021

But I still don't know why double AR and LINKER flags? While in llvmtools is uses like below.

yeah, I need to check that and remove doubles

@dslm4515
Copy link
Owner Author

dslm4515 commented Jul 5, 2021

I do appreciate you catching those!

@dslm4515
Copy link
Owner Author

dslm4515 commented Jul 9, 2021

So, you should remove one of two DEFAULT_SYSROOT.

yes, you are right. I did not notice.

Yes. I just confirmed: Setting -DDEFAULT_SYSROOT="/llvmtols" will produce clang that will search for headers in /llvmtools/include and libraries in /llvmtools/lib.

Therefore -DDEFAULT_SYSROOT="/usr" is what sets the built LLVM to /usr to search for headers in /usr/include and libraries in /usr/lib

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants