[BOLT] [NFC] Remove special DWARF expressions handling from LLVM

.dockerignore

@@ -0,0 +1 @@
+.git

.github/workflows/Dockerfile

@@ -0,0 +1,31 @@
+FROM ubuntu:20.04 AS builder
+
+ARG DEBIAN_FRONTEND=noninteractive
+ENV TZ=UTC
+
+RUN apt-get update && \
+    apt-get install -y --no-install-recommends ca-certificates git \
+      build-essential cmake ninja-build python3 libjemalloc-dev && \
+    rm -rf /var/lib/apt/lists
+
+WORKDIR /home/bolt
+
+COPY . llvm-bolt
+
+WORKDIR build
+
+RUN cmake -G Ninja ../llvm-bolt/llvm \
+      -DLLVM_ENABLE_PROJECTS="bolt" \
+      -DLLVM_TARGETS_TO_BUILD="X86;AArch64" \
+      -DCMAKE_BUILD_TYPE=Release \
+      -DLLVM_ENABLE_ASSERTIONS=ON \
+      -DCMAKE_EXE_LINKER_FLAGS="-Wl,--push-state -Wl,-whole-archive -ljemalloc_pic -Wl,--pop-state -lpthread -lstdc++ -lm -ldl" \
+      -DCMAKE_INSTALL_PREFIX=/home/bolt/install
+
+RUN ninja check-bolt && \
+    ninja install-llvm-bolt install-perf2bolt install-merge-fdata \
+      install-llvm-boltdiff install-bolt_rt
+
+FROM ubuntu:20.04
+
+COPY --from=builder /home/bolt/install /usr/local

.github/workflows/docker-image.yml

@@ -0,0 +1,16 @@
+name: Docker Image CI
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+    branches: [ main ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v2
+    - name: Build the Docker image and test
+      run: docker build . --file .github/workflows/Dockerfile --tag ubuntu-bolt:$(date +%s)

README.md

@@ -1,106 +1,210 @@
-# The LLVM Compiler Infrastructure
+# BOLT
 
-This directory and its sub-directories contain source code for LLVM,
-a toolkit for the construction of highly optimized compilers,
-optimizers, and run-time environments.
+BOLT is a post-link optimizer developed to speed up large applications.
+It achieves the improvements by optimizing application's code layout based on
+execution profile gathered by sampling profiler, such as Linux `perf` tool.
+An overview of the ideas implemented in BOLT along with a discussion of its
+potential and current results is available in
+[CGO'19 paper](https://research.fb.com/publications/bolt-a-practical-binary-optimizer-for-data-centers-and-beyond/).
 
-The README briefly describes how to get started with building LLVM.
-For more information on how to contribute to the LLVM project, please
-take a look at the
-[Contributing to LLVM](https://llvm.org/docs/Contributing.html) guide.
+## Input Binary Requirements
 
-## Getting Started with the LLVM System
+BOLT operates on X86-64 and AArch64 ELF binaries. At the minimum, the binaries
+should have an unstripped symbol table, and, to get maximum performance gains,
+they should be linked with relocations (`--emit-relocs` or `-q` linker flag).
 
-Taken from https://llvm.org/docs/GettingStarted.html.
+BOLT disassembles functions and reconstructs the control flow graph (CFG)
+before it runs optimizations. Since this is a nontrivial task,
+especially when indirect branches are present, we rely on certain heuristics
+to accomplish it. These heuristics have been tested on a code generated with
+Clang and GCC compilers. The main requirement for C/C++ code is not to rely
+on code layout properties, such as function pointer deltas.
+Assembly code can be processed too. Requirements for it include a clear
+separation of code and data, with data objects being placed into data
+sections/segments. If indirect jumps are used for intra-function control
+transfer (e.g., jump tables), the code patterns should be matching those
+generated by Clang/GCC.
 
-### Overview
+NOTE: BOLT is currently incompatible with the `-freorder-blocks-and-partition`
+compiler option. Since GCC8 enables this option by default, you have to
+explicitly disable it by adding `-fno-reorder-blocks-and-partition` flag if
+you are compiling with GCC8.
+
+PIE and .so support has been added recently. Please report bugs if you
+encounter any issues.
 
-Welcome to the LLVM project!
+## Installation
 
-The LLVM project has multiple components. The core of the project is
-itself called "LLVM". This contains all of the tools, libraries, and header
-files needed to process intermediate representations and convert them into
-object files.  Tools include an assembler, disassembler, bitcode analyzer, and
-bitcode optimizer.  It also contains basic regression tests.
+### Docker Image
 
-C-like languages use the [Clang](http://clang.llvm.org/) front end.  This
-component compiles C, C++, Objective-C, and Objective-C++ code into LLVM bitcode
--- and from there into object files, using LLVM.
+You can build and use the docker image containing BOLT using our [docker file](./bolt/utils/docker/Dockerfile).
+Alternatively, you can build BOLT manually using the steps below.
+
+### Manual Build
+
+BOLT heavily uses LLVM libraries, and by design, it is built as one of LLVM
+tools. The build process is not much different from a regular LLVM build.
+The following instructions are assuming that you are running under Linux.
+
+Start with cloning LLVM and BOLT repos:
+
+```
+> git clone https://github.com/facebookincubator/BOLT llvm-bolt
+> mkdir build
+> cd build
+> cmake -G Ninja ../llvm-bolt/llvm -DLLVM_TARGETS_TO_BUILD="X86;AArch64" -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON -DLLVM_ENABLE_PROJECTS="bolt"
+> ninja
+```
 
-Other components include:
-the [libc++ C++ standard library](https://libcxx.llvm.org),
-the [LLD linker](https://lld.llvm.org), and more.
+`llvm-bolt` will be available under `bin/`. Add this directory to your path to
+ensure the rest of the commands in this tutorial work.
 
-### Getting the Source Code and Building LLVM
+Note that we use a specific snapshot of LLVM monorepo as we currently
+rely on a set of patches that are not yet upstreamed.
+
+## Optimizing BOLT's Performance
 
-The LLVM Getting Started documentation may be out of date.  The [Clang
-Getting Started](http://clang.llvm.org/get_started.html) page might have more
-accurate information.
+BOLT runs many internal passes in parallel. If you foresee heavy usage of
+BOLT, you can improve the processing time by linking against one of memory
+allocation libraries with good support for concurrency. E.g. to use jemalloc:
 
-This is an example work-flow and configuration to get and build the LLVM source:
+```
+> sudo yum install jemalloc-devel
+> LD_PRELOAD=/usr/lib64/libjemalloc.so llvm-bolt ....
+```
+Or if you rather use tcmalloc:
+```
+> sudo yum install gperftools-devel
+> LD_PRELOAD=/usr/lib64/libtcmalloc_minimal.so llvm-bolt ....
+```
+
+## Usage
 
-1. Checkout LLVM (including related sub-projects like Clang):
+For a complete practical guide of using BOLT see [Optimizing Clang with BOLT](./bolt/docs/OptimizingClang.md).
 
-     * ``git clone https://github.com/llvm/llvm-project.git``
+### Step 0
 
-     * Or, on windows, ``git clone --config core.autocrlf=false
-    https://github.com/llvm/llvm-project.git``
+In order to allow BOLT to re-arrange functions (in addition to re-arranging
+code within functions) in your program, it needs a little help from the linker.
+Add `--emit-relocs` to the final link step of your application. You can verify
+the presence of relocations by checking for `.rela.text` section in the binary.
+BOLT will also report if it detects relocations while processing the binary.
+
+### Step 1: Collect Profile
+
+This step is different for different kinds of executables. If you can invoke
+your program to run on a representative input from a command line, then check
+**For Applications** section below. If your program typically runs as a
+server/service, then skip to **For Services** section.
 
-2. Configure and build LLVM and Clang:
+The version of `perf` command used for the following steps has to support
+`-F brstack` option. We recommend using `perf` version 4.5 or later.
+
+#### For Applications
+
+This assumes you can run your program from a command line with a typical input.
+In this case, simply prepend the command line invocation with `perf`:
+```
+$ perf record -e cycles:u -j any,u -o perf.data -- <executable> <args> ...
+```
+
+#### For Services
+
+Once you get the service deployed and warmed-up, it is time to collect perf
+data with LBR (branch information). The exact perf command to use will depend
+on the service. E.g., to collect the data for all processes running on the
+server for the next 3 minutes use:
+```
+$ perf record -e cycles:u -j any,u -a -o perf.data -- sleep 180
+```
 
-     * ``cd llvm-project``
+Depending on the application, you may need more samples to be included with
+your profile. It's hard to tell upfront what would be a sweet spot for your
+application. We recommend the profile to cover 1B instructions as reported
+by BOLT `-dyno-stats` option. If you need to increase the number of samples
+in the profile, you can either run the `sleep` command for longer and use
+`-F<N>` option with `perf` to increase sampling frequency.
 
-     * ``cmake -S llvm -B build -G <generator> [options]``
+Note that for profile collection we recommend using cycle events and not
+`BR_INST_RETIRED.*`. Empirically we found it to produce better results.
 
-        Some common build system generators are:
+If the collection of a profile with branches is not available, e.g., when you run on
+a VM or on hardware that does not support it, then you can use only sample
+events, such as cycles. In this case, the quality of the profile information
+would not be as good, and performance gains with BOLT are expected to be lower.
 
-        * ``Ninja`` --- for generating [Ninja](https://ninja-build.org)
-          build files. Most llvm developers use Ninja.
-        * ``Unix Makefiles`` --- for generating make-compatible parallel makefiles.
-        * ``Visual Studio`` --- for generating Visual Studio projects and
-          solutions.
-        * ``Xcode`` --- for generating Xcode projects.
+#### With instrumentation (experimental)
 
-        Some Common options:
+If perf record is not available to you, you may collect profile by first
+instrumenting the binary with BOLT and then running it.
+```
+llvm-bolt <executable> -instrument -o <instrumented-executable>
+```
+
+After you run instrumented-executable with the desired workload, its BOLT
+profile should be ready for you in `/tmp/prof.fdata` and you can skip
+**Step 2**.
+
+Run BOLT with the `-help` option and check the category "BOLT instrumentation
+options" for a quick reference on instrumentation knobs. Instrumentation is
+experimental and currently does not work for PIEs/SOs.
+
+### Step 2: Convert Profile to BOLT Format
+
+NOTE: you can skip this step and feed `perf.data` directly to BOLT using
+experimental `-p perf.data` option.
 
-        * ``-DLLVM_ENABLE_PROJECTS='...'`` --- semicolon-separated list of the LLVM
-          sub-projects you'd like to additionally build. Can include any of: clang,
-          clang-tools-extra, libcxx, libcxxabi, libunwind, lldb, compiler-rt, lld,
-          polly, or cross-project-tests.
+For this step, you will need `perf.data` file collected from the previous step and
+a copy of the binary that was running. The binary has to be either
+unstripped, or should have a symbol table intact (i.e., running `strip -g` is
+okay).
 
-          For example, to build LLVM, Clang, libcxx, and libcxxabi, use
-          ``-DLLVM_ENABLE_PROJECTS="clang;libcxx;libcxxabi"``.
+Make sure `perf` is in your `PATH`, and execute `perf2bolt`:
+```
+$ perf2bolt -p perf.data -o perf.fdata <executable>
+```
+
+This command will aggregate branch data from `perf.data` and store it in a
+format that is both more compact and more resilient to binary modifications.
+
+If the profile was collected without LBRs, you will need to add `-nl` flag to
+the command line above.
+
+### Step 3: Optimize with BOLT
+
+Once you have `perf.fdata` ready, you can use it for optimizations with
+BOLT. Assuming your environment is setup to include the right path, execute
+`llvm-bolt`:
+```
+$ llvm-bolt <executable> -o <executable>.bolt -data=perf.fdata -reorder-blocks=cache+ -reorder-functions=hfsort -split-functions=2 -split-all-cold -split-eh -dyno-stats
+```
 
-        * ``-DCMAKE_INSTALL_PREFIX=directory`` --- Specify for *directory* the full
-          path name of where you want the LLVM tools and libraries to be installed
-          (default ``/usr/local``).
+If you do need an updated debug info, then add `-update-debug-sections` option
+to the command above. The processing time will be slightly longer.
 
-        * ``-DCMAKE_BUILD_TYPE=type`` --- Valid options for *type* are Debug,
-          Release, RelWithDebInfo, and MinSizeRel. Default is Debug.
-
-        * ``-DLLVM_ENABLE_ASSERTIONS=On`` --- Compile with assertion checks enabled
-          (default is Yes for Debug builds, No for all other build types).
-
-      * ``cmake --build build [-- [options] <target>]`` or your build system specified above
-        directly.
-
-        * The default target (i.e. ``ninja`` or ``make``) will build all of LLVM.
-
-        * The ``check-all`` target (i.e. ``ninja check-all``) will run the
-          regression tests to ensure everything is in working order.
-
-        * CMake will generate targets for each tool and library, and most
-          LLVM sub-projects generate their own ``check-<project>`` target.
-
-        * Running a serial build will be **slow**.  To improve speed, try running a
-          parallel build.  That's done by default in Ninja; for ``make``, use the option
-          ``-j NNN``, where ``NNN`` is the number of parallel jobs, e.g. the number of
-          CPUs you have.
-
-      * For more information see [CMake](https://llvm.org/docs/CMake.html)
-
-Consult the
-[Getting Started with LLVM](https://llvm.org/docs/GettingStarted.html#getting-started-with-llvm)
-page for detailed information on configuring and compiling LLVM. You can visit
-[Directory Layout](https://llvm.org/docs/GettingStarted.html#directory-layout)
-to learn about the layout of the source code tree.
+For a full list of options see `-help`/`-help-hidden` output.
+
+The input binary for this step does not have to 100% match the binary used for
+profile collection in **Step 1**. This could happen when you are doing active
+development, and the source code constantly changes, yet you want to benefit
+from profile-guided optimizations. However, since the binary is not precisely the
+same, the profile information could become invalid or stale, and BOLT will
+report the number of functions with a stale profile. The higher the
+number, the less performance improvement should be expected. Thus, it is
+crucial to update `.fdata` for release branches.
+
+## Multiple Profiles
+
+Suppose your application can run in different modes, and you can generate
+multiple profiles for each one of them. To generate a single binary that can
+benefit all modes (assuming the profiles don't contradict each other) you can
+use `merge-fdata` tool:
+```
+$ merge-fdata *.fdata > combined.fdata
+```
+Use `combined.fdata` for **Step 3** above to generate a universally optimized
+binary.
+
+## License
+
+BOLT is licensed under the [Apache License v2.0 with LLVM Exceptions](./LICENSE.TXT).

bolt/CMakeLists.txt

@@ -0,0 +1,29 @@
+include(ExternalProject)
+
+set(BOLT_SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR})
+set(BOLT_BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR})
+set(CMAKE_CXX_STANDARD 14)
+
+ExternalProject_Add(bolt_rt
+  SOURCE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/runtime"
+  STAMP_DIR ${CMAKE_CURRENT_BINARY_DIR}/bolt_rt-stamps
+  BINARY_DIR ${CMAKE_CURRENT_BINARY_DIR}/bolt_rt-bins
+  CMAKE_ARGS -DCMAKE_C_COMPILER=${CMAKE_C_COMPILER}
+             -DCMAKE_CXX_COMPILER=${CMAKE_CXX_COMPILER}
+             -DCMAKE_BUILD_TYPE=Release
+             -DCMAKE_MAKE_PROGRAM=${CMAKE_MAKE_PROGRAM}
+             -DCMAKE_INSTALL_PREFIX=${LLVM_BINARY_DIR}
+  # You might want to set this to True if actively developing bolt_rt, otherwise
+  # cmake will not rebuild it after source code changes
+  BUILD_ALWAYS True
+  )
+
+install(CODE "execute_process\(COMMAND \${CMAKE_COMMAND} -DCMAKE_INSTALL_PREFIX=\${CMAKE_INSTALL_PREFIX} -P ${CMAKE_CURRENT_BINARY_DIR}/bolt_rt-bins/cmake_install.cmake \)"
+  COMPONENT bolt_rt)
+
+add_llvm_install_targets(install-bolt_rt
+  DEPENDS bolt_rt
+  COMPONENT bolt_rt)
+
+add_subdirectory(src)
+add_subdirectory(test)