-
Notifications
You must be signed in to change notification settings - Fork 11.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[LTO] Enable adding custom pass instrumentation callbacks #71268
Conversation
@llvm/pr-subscribers-lto Author: Igor Kudrin (igorkudrin) ChangesThe hook allows the calling code to register instrumentation callbacks for the LTO optimization pipeline. In particular, a custom pass filter can be added to skip certain passes from the default pipeline. Full diff: https://github.com/llvm/llvm-project/pull/71268.diff 2 Files Affected:
diff --git a/llvm/include/llvm/LTO/Config.h b/llvm/include/llvm/LTO/Config.h
index 6fb55f1cf1686a5..fd22c17bc12a38c 100644
--- a/llvm/include/llvm/LTO/Config.h
+++ b/llvm/include/llvm/LTO/Config.h
@@ -257,6 +257,11 @@ struct Config {
const DenseSet<GlobalValue::GUID> &GUIDPreservedSymbols)>;
CombinedIndexHookFn CombinedIndexHook;
+ /// This hook is called when the optimization pipeline is being built.
+ using CustomizeOptPipelineHookFn =
+ std::function<void(PassInstrumentationCallbacks &)>;
+ CustomizeOptPipelineHookFn CustomizeOptPipeline;
+
/// This is a convenience function that configures this Config object to write
/// temporary files named after the given OutputFileName for each of the LTO
/// phases to disk. A client can use this function to implement -save-temps.
diff --git a/llvm/lib/LTO/LTOBackend.cpp b/llvm/lib/LTO/LTOBackend.cpp
index ccc4276e36dacf0..7e1b35319a71d59 100644
--- a/llvm/lib/LTO/LTOBackend.cpp
+++ b/llvm/lib/LTO/LTOBackend.cpp
@@ -265,6 +265,8 @@ static void runNewPMPasses(const Config &Conf, Module &Mod, TargetMachine *TM,
ModuleAnalysisManager MAM;
PassInstrumentationCallbacks PIC;
+ if (Conf.CustomizeOptPipeline)
+ Conf.CustomizeOptPipeline(PIC);
StandardInstrumentations SI(Mod.getContext(), Conf.DebugPassManager,
Conf.VerifyEach);
SI.registerCallbacks(PIC, &MAM);
|
Ping |
Can you add a test, which looks like it will also require adding an LLVM mechanism for setting this hook. |
the commit message is confusing, this isn't customizing the optimization pipeline, it's customizing |
I don't think we have this sort of functionality elsewhere for the default optimization pipelines, why do you specifically just want this for the LTO pipeline? |
86ce902
to
c63c479
Compare
Added a unittest that uses the hook.
Updated. Is it better now?
We are working on a downstream tool that pre-links bitcode files, runs optimization passes, and produces bitcode file output that can be used later for a final link with LTO, so we run LTO for input files but interrupt the process before codegen. Some passes in the default optimization pipeline, like |
The hook allows the calling code to register instrumentation callbacks for the LTO optimization pipeline. In particular, a custom pass filter can be added to skip certain passes from the default pipeline.
c63c479
to
fdb36c4
Compare
Interesting, essentially you are doing 2 rounds of LTO? Is the EliminateAvailableExternallyPass the only one you want to disable? Another approach would be to add an internal option to enable or skip that pass, like many of the other -enable-* flags in PassBuilderPipelines.cpp. There are existing options to register a custom opt pipeline completely, but I suppose you want something very close to an existing pass pipeline minus a handful of passes? |
That's right.
These are command-line options and their variables are local to the TU. There is no API to access them from other components. Designing a global filter could be a solution, but for now, it seems too complex for our needs. Moreover, using such an API would mean that components communicate through a global state, which is usually not considered a good program design.
That's right again. We want to run the typical LTO pass pipelines with only minor customizations. |
I'm not sure what you mean here. How are you invoking your LTO link? You can pass internal options via the linker or other tools that are invoking LTO. E.g. for an lld LTO link: -Wl,-mllvm,-enable-elim-avail-extern=false (making up a possible name for the internal option). |
Our tool uses the LLVM libraries directly, similar to |
Can you clarify what you meant by "These are command-line options and their variables are local to the TU. There is no API to access them from other components." Because if you have a tool that is built like llvm-lto2 etc using LLVM libraries, you should be able to parse the internal command line options passed to it. I'm not sure what is TU local about them? |
Let's take static cl::opt<bool> EnableSyntheticCounts("enable-npm-synthetic-counts", ...); This is a command line option, so a user may be able to specify it on the command line when running the tool. This is suitable for developer-level tools like The variable for this option is If we make the variable |
Ah got it. The typical way then would be to have a flag in the PipelineTuningOptions struct that can be set from your tool and queried by the PM builder. |
A setting in the |
Sure, that makes sense. I will let @aeubanks comment more on the original approach (or any other ideas) now that you have clarified the problem. |
Gently ping |
This seems ok, but I feel like what you really want is IIUC, you want optimizations, but more like the ThinLTO pre-link phase (e.g. we don't run |
The hook allows the calling code to register instrumentation callbacks for the LTO optimization pipeline. In particular, a custom pass filter can be added to skip certain passes from the default pipeline.