Build LLVM with the Repo Compiler

This page shows how to build the llvm-project-prepo project from source then use the resulting compiler to re-build LLVM on Linux.

Table of Contents

• Building the initial repo compiler
• Using the initial repo compiler to build LLVM
• Running unit and regression tests
• Current build status

Building the initial repo compiler

1. Clone the llvm-project-prepo repository.

   $ REPO_DIR="/the/directory/into/which/the/LLVM/source/code/will/be/cloned"
   $ cd $REPO_DIR
   $ git clone https://github.com/SNSystems/llvm-project-prepo.git

2. Clone pstore.

   $ cd $REPO_DIR/llvm-project-prepo
   $ git clone https://github.com/SNSystems/pstore.git

3. Build LLVM enabling the clang and pstore subprojects.

   $ cd $REPO_DIR/llvm-project-prepo
   $ mkdir build
   $ cd build
   $ cmake -G Ninja \
           -DCMAKE_BUILD_TYPE=Release \
           -DLLVM_ENABLE_PROJECTS="clang;pstore" \
           -DLLVM_TARGETS_TO_BUILD=X86 \
           -DLLVM_TOOL_CLANG_TOOLS_EXTRA_BUILD=OFF \
           ../llvm
   $ ninja

Using the initial repo compiler to build LLVM

The process of using the compiler that we just built to compile LLVM consists of the following steps:

1. Compile the llvm-project-prepo project targeting the program repository using the initial repo compiler.
2. Convert the repository ticket files to ELF object files using repo2obj.
3. Link the ELF object files to generate an executable.
4. Repeat the above steps once to verify the database (i.e. all fragments are in the database).

These are now described in detail:

1. Copy repo.json to any parent directory of $REPO_DIR/llvm-project-prepo/build.

    $ cd $REPO_DIR/llvm-project-prepo
    $ cp ./llvm/utils/repo/repo.json .
    $ cat repo.json
    {
    	"ar": "/usr/bin/ar",
    	"link": "/usr/bin/clang++",
    	"repo2obj": "/point/to/the/repo2obj/tool/in/llvm-pstore/build/directory"
    }
   

2. Modify the repo2obj option to point to the full path of the repo2obj tool in your build directory.

   Continue to use your default ar and clang++ for the archiving and linking; modify the repo2obj option to point to the repo2obj tool in your build directory. Note: this must be an absolute path. e.g.:

    "repo2obj": "/home/username/llvm-project-prepo/build/bin/repo2obj"
   

3. Generate a release build using the repo.cmake toolchain file.

    $ cd $REPO_DIR/llvm-project-prepo
    $ mkdir build_repo_release
    $ cd build_repo_release
    $ cmake -G Ninja \
            -DCMAKE_BUILD_TYPE=Release \
            -DLLVM_TARGETS_TO_BUILD=X86 \
            -DLLVM_ENABLE_PROJECTS="clang;pstore" \
            -DLLVM_TOOL_CLANG_TOOLS_EXTRA_BUILD=OFF \
            -DPSTORE_ENABLE_BROKER=OFF \
            -DCMAKE_TOOLCHAIN_FILE=$REPO_DIR/llvm-project-prepo/llvm/utils/repo/repo.cmake \
            -DCMAKE_CXX_COMPILER=$REPO_DIR/llvm-project-prepo/build/bin/clang++ \
            -DCMAKE_C_COMPILER=$REPO_DIR/llvm-project-prepo/build/bin/clang \
            -Dutils_dir=$REPO_DIR/llvm-project-prepo/llvm/utils/repo \
            ../llvm

   The CMake variables being defined here are:

   1. -DPSTORE_ENABLE_BROKER=OFF. Disable the pstore broker projects since they rely on exception handling which the repo compiler does not currently support.
   2. -DCMAKE_TOOLCHAIN_FILE=…/repo.cmake. CMake uses the toolchain to describe the tools and how they should be run; we use it here to set compiler switches such as the target-triple and to run repo2obj before linking.
   3. -DCMAKE_CXX_COMPILER=…/clang++. Tell CMake where to find the repo C++ compiler that we built.
   4. -DCMAKE_C_COMPILER=…/clang. As above for the C compiler.
   5. -Dutils_dir=…/llvm/utils/repo. Specify the repo utilities directory, which contains additional scripts required for the build.

4. Build LLVM.

   $ ninja

5. To rebuild, you can clean/delete all ticket files. Code fragments created by the original build will be retained within the database.

   $ ninja clean
   $ ninja

Running unit and regression tests

Currently, for the release build (-O3 -fno-exceptions -fno-rtti), all tests are passed.

1. Run the tests.

   $ ninja check-all

2. Rerun the tests.

   $ ninja clean
   $ ninja check-all

Current build status

llvm-project-prepo has been built and run with the tests twice in the release configuration.

Release build with compiler switches: -O3 -fno-exceptions -fno-rtti.

Configuration	1st build	1st run test	2nd build	2nd run test
Release	✔	100% passed ✔	✔	100% passed ✔