Skip to content

Commit a67208e

Browse files
ilovepiilovepi
committed
[llvm] Preliminary fat-lto-objects support
Fat LTO objects contain both LTO compatible IR, as well as generated object code. This allows users to defer the choice of whether to use LTO or not to link-time. This is a feature available in GCC for some time, and makes the existing -ffat-lto-objects flag functional in the same way as GCC's. Within LLVM, we add a new EmbedBitcodePass that serializes the module to the object file, and expose a new pass pipeline for compiling fat objects. The new pipeline initially clones the module and runs the selected (Thin)LTOPrelink pipeline, after which it will serialize the module into a `.llvm.lto` section of an ELF file. When compiling for (Thin)LTO, this normally the point at which the compiler would emit a object file containing the bitcode and metadata. After that point we compile the original module using the PerModuleDefaultPipeline used for non-LTO compilation. We generate standard object files at the end of this pipeline, which contain machine code and the new `.llvm.lto` section containing bitcode. Since the two pipelines operate on different copies of the module, we can be sure that the bitcode in the `.llvm.lto` section and object code in `.text` are congruent with the existing output produced by the default and LTO pipelines. Original RFC: https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977 Reviewed By: tejohnson, MaskRay, nikic Differential Revision: https://reviews.llvm.org/D146776
1 parent 7e5d7d2 commit a67208e

14 files changed

+301
-1
lines changed

‎llvm/docs/FatLTO.rst

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
===================
2+
FatLTO
3+
===================
4+
.. contents::
5+
:local:
6+
:depth: 2
7+
8+
.. toctree::
9+
:maxdepth: 1
10+
11+
Introduction
12+
============
13+
14+
FatLTO objects are a special type of `fat object file
15+
<https://en.wikipedia.org/wiki/Fat_binary>`_ that contain LTO compatible IR in
16+
addition to generated object code, instead of containing object code for
17+
multiple target architectures. This allows users to defer the choice of whether
18+
to use LTO or not to link-time, and has been a feature available in other
19+
compilers, like `GCC
20+
<https://gcc.gnu.org/onlinedocs/gccint/LTO-Overview.html>`_, for some time.
21+
22+
Under FatLTO the compiler can emit standard object files which contain both the
23+
machine code in the ``.text`` section and LLVM bitcode in the ``.llvm.lto``
24+
section.
25+
26+
Overview
27+
========
28+
29+
Within LLVM, FatLTO is supported by choosing the ``FatLTODefaultPipeline``.
30+
This pipeline will:
31+
32+
#) Clone the IR module.
33+
#) Run the pre-link (Thin)LTO pipeline using the cloned module.
34+
#) Embed the pre-link bitcode in a special ``.llvm.lto`` section.
35+
#) Optimize the unmodified copy of the module using the normal compilation pipeline.
36+
#) Emit the object file, including the new ``.llvm.lto`` section.
37+
38+
.. NOTE
39+
40+
At the time of writing, we conservatively run independent pipelines to
41+
generate the bitcode section and the object code, which happen to be
42+
identical to those used outside of FatLTO. This results in compiled
43+
artifacts that are identical to those produced by the default and (Thin)LTO
44+
pipelines. However, this is not a guarantee, and we reserve the right to
45+
change this at any time. Explicitly, users should not rely on the produced
46+
bitcode or object code to mach their non-LTO counterparts precisely. They
47+
will exhibit similar performance characteristics, but may not be bit-for-bit
48+
the same.
49+
50+
Internally, the ``.llvm.lto`` section is created by running the
51+
``EmbedBitcodePass`` at the start of the ``PerModuleDefaultPipeline``. This
52+
pass is responsible for cloning and optimizing the module with the appropriate
53+
LTO pipeline and emitting the ``.llvm.lto`` section. Afterwards, the
54+
``PerModuleDefaultPipeline`` runs normally and the compiler can emit the fat
55+
object file.
56+
57+
Limitations
58+
===========
59+
60+
Linkers
61+
-------
62+
63+
Currently, using LTO with LLVM fat lto objects is supported by LLD and by the
64+
GNU linkers via :doc:`GoldPlugin`. This may change in the future, but
65+
extending support to other linkers isn't planned for now.
66+
67+
.. NOTE
68+
For standard linking the fat object files should be usable by any
69+
linker capable of using ELF objects, since the ``.llvm.lto`` section is
70+
marked ``SHF_EXLUDE``.
71+
72+
Supported File Formats
73+
----------------------
74+
75+
The current implementation only supports ELF files. At time of writing, it is
76+
unclear if it will be useful to support other object file formats like ``COFF``
77+
or ``Mach-O``.

‎llvm/docs/ReleaseNotes.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,6 +80,12 @@ Changes to LLVM infrastructure
8080
* InstructionSimplify APIs now require instructions be inserted into a
8181
parent function.
8282

83+
* A new FatLTO pipeline was added to support generating object files that have
84+
both machine code and LTO compatible bitcode. See the :doc:`FatLTO`
85+
documentation and the original
86+
`RFC <https://discourse.llvm.org/t/rfc-ffat-lto-objects-support/63977>`_
87+
for more details.
88+
8389
Changes to building LLVM
8490
------------------------
8591

‎llvm/docs/UserGuides.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ intermediate LLVM representation.
3232
DebuggingJITedCode
3333
DirectXUsage
3434
Docker
35+
FatLTO
3536
ExtendingLLVM
3637
GoldPlugin
3738
HowToBuildOnARM

‎llvm/include/llvm/Passes/PassBuilder.h

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -234,6 +234,18 @@ class PassBuilder {
234234
ModulePassManager buildPerModuleDefaultPipeline(OptimizationLevel Level,
235235
bool LTOPreLink = false);
236236

237+
/// Build a fat object default optimization pipeline.
238+
///
239+
/// This builds a pipeline that runs the LTO/ThinLTO pre-link pipeline, and
240+
/// emits a section containing the pre-link bitcode along side the object code
241+
/// generated by running the PerModuleDefaultPipeline, used when compiling
242+
/// without LTO. It clones the module and runs the LTO/non-LTO pipelines
243+
/// separately to avoid any inconsistencies with an ad-hoc pipeline that tries
244+
/// to approximate the PerModuleDefaultPipeline from the pre-link LTO
245+
/// pipelines.
246+
ModulePassManager buildFatLTODefaultPipeline(OptimizationLevel Level,
247+
bool ThinLTO, bool EmitSummary);
248+
237249
/// Build a pre-link, ThinLTO-targeting default optimization pipeline to
238250
/// a pass manager.
239251
///

‎llvm/include/llvm/Transforms/IPO/EmbedBitcodePass.h

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
//===-- EmbedBitcodePass.h - Embeds bitcode into global ---------*- C++ -*-===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
/// \file
9+
///
10+
/// This file provides a pass which clones the current module and runs the
11+
/// provided pass pipeline on the clone. The optimized module is stored into a
12+
/// global variable in the `.llvm.lto` section. Primarily, this pass is used
13+
/// to support the FatLTO pipeline, but could be used to generate a bitcode
14+
/// section for any arbitrary pass pipeline without changing the current module.
15+
///
16+
//===----------------------------------------------------------------------===//
17+
//
18+
#ifndef LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
19+
#define LLVM_TRANSFORMS_IPO_EMBEDBITCODEPASS_H
20+
21+
#include "llvm/IR/PassManager.h"
22+
23+
namespace llvm {
24+
class Module;
25+
class ModulePass;
26+
class Pass;
27+
28+
struct EmbedBitcodeOptions {
29+
EmbedBitcodeOptions() : EmbedBitcodeOptions(false, false) {}
30+
EmbedBitcodeOptions(bool IsThinLTO, bool EmitLTOSummary)
31+
: IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary) {}
32+
bool IsThinLTO;
33+
bool EmitLTOSummary;
34+
};
35+
36+
/// Pass embeds a copy of the module optimized with the provided pass pipeline
37+
/// into a global variable.
38+
class EmbedBitcodePass : public PassInfoMixin<EmbedBitcodePass> {
39+
bool IsThinLTO;
40+
bool EmitLTOSummary;
41+
ModulePassManager MPM;
42+
43+
public:
44+
EmbedBitcodePass(EmbedBitcodeOptions Opts)
45+
: EmbedBitcodePass(Opts.IsThinLTO, Opts.EmitLTOSummary,
46+
ModulePassManager()) {}
47+
EmbedBitcodePass(bool IsThinLTO, bool EmitLTOSummary, ModulePassManager &&MPM)
48+
: IsThinLTO(IsThinLTO), EmitLTOSummary(EmitLTOSummary),
49+
MPM(std::move(MPM)) {}
50+
51+
PreservedAnalyses run(Module &M, ModuleAnalysisManager &);
52+
53+
static bool isRequired() { return true; }
54+
};
55+
56+
} // end namespace llvm.
57+
58+
#endif

‎llvm/lib/Object/ObjectFile.cpp

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -79,7 +79,7 @@ uint32_t ObjectFile::getSymbolAlignment(DataRefImpl DRI) const { return 0; }
7979
bool ObjectFile::isSectionBitcode(DataRefImpl Sec) const {
8080
Expected<StringRef> NameOrErr = getSectionName(Sec);
8181
if (NameOrErr)
82-
return *NameOrErr == ".llvmbc";
82+
return *NameOrErr == ".llvmbc" || *NameOrErr == ".llvm.lto";
8383
consumeError(NameOrErr.takeError());
8484
return false;
8585
}

‎llvm/lib/Passes/PassBuilder.cpp

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -102,6 +102,7 @@
102102
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
103103
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
104104
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
105+
#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
105106
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
106107
#include "llvm/Transforms/IPO/FunctionAttrs.h"
107108
#include "llvm/Transforms/IPO/FunctionImport.h"
@@ -738,6 +739,26 @@ Expected<HWAddressSanitizerOptions> parseHWASanPassOptions(StringRef Params) {
738739
return Result;
739740
}
740741

742+
Expected<EmbedBitcodeOptions> parseEmbedBitcodePassOptions(StringRef Params) {
743+
EmbedBitcodeOptions Result;
744+
while (!Params.empty()) {
745+
StringRef ParamName;
746+
std::tie(ParamName, Params) = Params.split(';');
747+
748+
if (ParamName == "thinlto") {
749+
Result.IsThinLTO = true;
750+
} else if (ParamName == "emit-summary") {
751+
Result.EmitLTOSummary = true;
752+
} else {
753+
return make_error<StringError>(
754+
formatv("invalid EmbedBitcode pass parameter '{0}' ", ParamName)
755+
.str(),
756+
inconvertibleErrorCode());
757+
}
758+
}
759+
return Result;
760+
}
761+
741762
Expected<MemorySanitizerOptions> parseMSanPassOptions(StringRef Params) {
742763
MemorySanitizerOptions Result;
743764
while (!Params.empty()) {

‎llvm/lib/Passes/PassBuilderPipelines.cpp

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,7 @@
4646
#include "llvm/Transforms/IPO/CrossDSOCFI.h"
4747
#include "llvm/Transforms/IPO/DeadArgumentElimination.h"
4848
#include "llvm/Transforms/IPO/ElimAvailExtern.h"
49+
#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
4950
#include "llvm/Transforms/IPO/ForceFunctionAttrs.h"
5051
#include "llvm/Transforms/IPO/FunctionAttrs.h"
5152
#include "llvm/Transforms/IPO/GlobalDCE.h"
@@ -1464,7 +1465,18 @@ PassBuilder::buildPerModuleDefaultPipeline(OptimizationLevel Level,
14641465

14651466
if (LTOPreLink)
14661467
addRequiredLTOPreLinkPasses(MPM);
1468+
return MPM;
1469+
}
14671470

1471+
ModulePassManager
1472+
PassBuilder::buildFatLTODefaultPipeline(OptimizationLevel Level, bool ThinLTO,
1473+
bool EmitSummary) {
1474+
ModulePassManager MPM;
1475+
MPM.addPass(EmbedBitcodePass(ThinLTO, EmitSummary,
1476+
ThinLTO
1477+
? buildThinLTOPreLinkDefaultPipeline(Level)
1478+
: buildLTOPreLinkDefaultPipeline(Level)));
1479+
MPM.addPass(buildPerModuleDefaultPipeline(Level));
14681480
return MPM;
14691481
}
14701482

‎llvm/lib/Passes/PassRegistry.def

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,13 @@ MODULE_PASS_WITH_PARAMS("ipsccp",
170170
},
171171
parseIPSCCPOptions,
172172
"no-func-spec;func-spec")
173+
MODULE_PASS_WITH_PARAMS("embed-bitcode",
174+
"EmbedBitcodePass",
175+
[](EmbedBitcodeOptions Opts) {
176+
return EmbedBitcodePass(Opts);
177+
},
178+
parseEmbedBitcodePassOptions,
179+
"thinlto;emit-summary")
173180
#undef MODULE_PASS_WITH_PARAMS
174181

175182
#ifndef CGSCC_ANALYSIS

‎llvm/lib/Transforms/IPO/CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ add_llvm_component_library(LLVMipo
1111
CrossDSOCFI.cpp
1212
DeadArgumentElimination.cpp
1313
ElimAvailExtern.cpp
14+
EmbedBitcodePass.cpp
1415
ExtractGV.cpp
1516
ForceFunctionAttrs.cpp
1617
FunctionAttrs.cpp

‎llvm/lib/Transforms/IPO/EmbedBitcodePass.cpp

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
//===- EmbedBitcodePass.cpp - Pass that embeds the bitcode into a global---===//
2+
//
3+
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
4+
// See https://llvm.org/LICENSE.txt for license information.
5+
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
6+
//
7+
//===----------------------------------------------------------------------===//
8+
9+
#include "llvm/Transforms/IPO/EmbedBitcodePass.h"
10+
#include "llvm/Bitcode/BitcodeWriter.h"
11+
#include "llvm/Bitcode/BitcodeWriterPass.h"
12+
#include "llvm/IR/Constants.h"
13+
#include "llvm/IR/PassManager.h"
14+
#include "llvm/InitializePasses.h"
15+
#include "llvm/Pass.h"
16+
#include "llvm/Passes/PassBuilder.h"
17+
#include "llvm/Support/ErrorHandling.h"
18+
#include "llvm/Support/MemoryBuffer.h"
19+
#include "llvm/TargetParser/Triple.h"
20+
#include "llvm/Transforms/IPO/ThinLTOBitcodeWriter.h"
21+
#include "llvm/Transforms/Utils/Cloning.h"
22+
#include "llvm/Transforms/Utils/ModuleUtils.h"
23+
24+
using namespace llvm;
25+
26+
PreservedAnalyses EmbedBitcodePass::run(Module &M, ModuleAnalysisManager &AM) {
27+
if (M.getGlobalVariable("llvm.embedded.module", /*AllowInternal=*/true))
28+
report_fatal_error("Can only embed the module once",
29+
/*gen_crash_diag=*/false);
30+
31+
Triple T(M.getTargetTriple());
32+
if (T.getObjectFormat() != Triple::ELF)
33+
report_fatal_error(
34+
"EmbedBitcode pass currently only supports ELF object format",
35+
/*gen_crash_diag=*/false);
36+
37+
std::unique_ptr<Module> NewModule = CloneModule(M);
38+
MPM.run(*NewModule, AM);
39+
40+
std::string Data;
41+
raw_string_ostream OS(Data);
42+
if (IsThinLTO)
43+
ThinLTOBitcodeWriterPass(OS, /*ThinLinkOS=*/nullptr).run(*NewModule, AM);
44+
else
45+
BitcodeWriterPass(OS, /*ShouldPreserveUseListOrder=*/false, EmitLTOSummary)
46+
.run(*NewModule, AM);
47+
48+
embedBufferInModule(M, MemoryBufferRef(Data, "ModuleData"), ".llvm.lto");
49+
50+
return PreservedAnalyses::all();
51+
}

‎llvm/test/Transforms/EmbedBitcode/embed-multiple.ll

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
; RUN: not opt --mtriple x86_64-unknown-linux-gnu < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
2+
3+
@a = global i32 1
4+
@llvm.embedded.module = private constant [4 x i8] c"BC\C0\DE"
5+
6+
; CHECK: LLVM ERROR: Can only embed the module once

‎llvm/test/Transforms/EmbedBitcode/embed-unsupported-object-format.ll

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
; RUN: not opt --mtriple powerpc64-unknown-aix < %s -passes=embed-bitcode -S 2>&1 | FileCheck %s
2+
3+
@a = global i32 1
4+
5+
; CHECK: LLVM ERROR: EmbedBitcode pass currently only supports ELF object format

‎llvm/test/Transforms/EmbedBitcode/embed.ll

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode" -S | FileCheck %s
2+
; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto>" -S | FileCheck %s
3+
; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<emit-summary>" -S | FileCheck %s
4+
; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S | FileCheck %s
5+
6+
@a = global i32 1
7+
8+
; CHECK: @a = global i32 1
9+
;; Make sure the module is in the correct section.
10+
; CHECK: @llvm.embedded.object = private constant {{.*}}, section ".llvm.lto", align 1
11+
12+
;; Ensure that the metadata is in llvm.compiler.used.
13+
; CHECK: @llvm.compiler.used = appending global [1 x ptr] [ptr @llvm.embedded.object], section "llvm.metadata"
14+
15+
;; Make sure the metadata correlates to the .llvm.lto section.
16+
; CHECK: !llvm.embedded.objects = !{!1}
17+
; CHECK: !0 = !{}
18+
; CHECK: !{ptr @llvm.embedded.object, !".llvm.lto"}
19+
20+
21+
;; Ensure that the .llvm.lto section has SHT_EXCLUDE set.
22+
; RUN: opt --mtriple x86_64-unknown-linux-gnu < %s -passes="embed-bitcode<thinlto;emit-summary>" -S \
23+
; RUN: | llc --mtriple x86_64-unknown-linux-gnu -filetype=obj \
24+
; RUN: | llvm-readobj - --sections --elf-output-style=JSON --pretty-print \
25+
; RUN: | FileCheck %s --check-prefix=EXCLUDE
26+
27+
; EXCLUDE: "Name": ".llvm.lto",
28+
; EXCLUDE-NEXT: "Value": 7
29+
; EXCLUDE-NEXT: },
30+
; EXCLUDE-NEXT: "Type": {
31+
; EXCLUDE-NEXT: "Name": "SHT_PROGBITS",
32+
; EXCLUDE-NEXT: "Value": 1
33+
; EXCLUDE-NEXT: },
34+
; EXCLUDE-NEXT: "Flags": {
35+
; EXCLUDE-NEXT: "Value": 2147483648,
36+
; EXCLUDE-NEXT: "Flags": [
37+
; EXCLUDE-NEXT: {
38+
; EXCLUDE-NEXT: "Name": "SHF_EXCLUDE",
39+
; EXCLUDE-NEXT: "Value": 2147483648
40+
; EXCLUDE-NEXT: }
41+
; EXCLUDE-NEXT: ]
42+
; EXCLUDE-NEXT: },
43+

0 commit comments

Comments
 (0)