[libLTO] add thinlto caching flags to libLTO

[w2yehia]

On AIX, the linker's release cadence is once per year and it doesn't backport non-critical fixes to previous releases.
We would like to get thinLTO caching accessible for current customers, so this PR adds the cache flags as cl::opt options.
We can surround the code with #ifdef AIX if the patch is unacceptable as is.

[llvmbot]

@llvm/pr-subscribers-lto

Author: Wael Yehia (w2yehia)

Changes

On AIX, the linker's release cadence is once per year and it doesn't backport non-critical fixes to previous releases.
We would like to get thinLTO caching accessible for current customers, so this PR adds the cache flags as cl::opt options.
We can surround the code with #ifdef AIX if the patch is unacceptable as is.

---

Full diff: https://github.com/llvm/llvm-project/pull/168567.diff

1 Files Affected:

• (modified) llvm/tools/lto/lto.cpp (+42)

diff --git a/llvm/tools/lto/lto.cpp b/llvm/tools/lto/lto.cpp
index 467a4da27dcd8..6efefc7977e4a 100644
--- a/llvm/tools/lto/lto.cpp
+++ b/llvm/tools/lto/lto.cpp
@@ -24,6 +24,7 @@
 #include "llvm/LTO/legacy/LTOCodeGenerator.h"
 #include "llvm/LTO/legacy/LTOModule.h"
 #include "llvm/LTO/legacy/ThinLTOCodeGenerator.h"
+#include "llvm/Support/FileSystem.h"
 #include "llvm/Support/MemoryBuffer.h"
 #include "llvm/Support/Signals.h"
 #include "llvm/Support/TargetSelect.h"
@@ -44,6 +45,29 @@ static cl::opt<bool> EnableFreestanding(
     "lto-freestanding", cl::init(false),
     cl::desc("Enable Freestanding (disable builtins / TLI) during LTO"));
 
+static cl::opt<std::string> ThinLTOCacheDir(
+    "thinlto-cache-dir",
+    cl::desc("Experimental option, enable ThinLTO caching. Note: the cache "
+             "currently does not take the mcmodel setting into account, so you "
+             "might get false hits if different mcmodels are used in different "
+             "builds using the same cache directory."));
+
+static cl::opt<int> ThinLTOCachePruningInterval(
+    "thinlto-cache-pruning-interval", cl::init(1200),
+    cl::desc("Set ThinLTO cache pruning interval (seconds)."));
+
+static cl::opt<uint64_t> ThinLTOCacheMaxSizeBytes(
+    "thinlto-cache-max-size-bytes",
+    cl::desc("Set ThinLTO cache pruning directory maximum size in bytes."));
+
+static cl::opt<int> ThinLTOCacheMaxSizeFiles(
+    "thinlto-cache-max-size-files", cl::init(1000000),
+    cl::desc("Set ThinLTO cache pruning directory maximum number of files."));
+
+static cl::opt<unsigned> ThinLTOCacheEntryExpiration(
+    "thinlto-cache-entry-expiration", cl::init(604800) /* 1w */,
+    cl::desc("Set ThinLTO cache entry expiration time (seconds)."));
+
 #ifdef NDEBUG
 static bool VerifyByDefault = false;
 #else
@@ -543,6 +567,24 @@ thinlto_code_gen_t thinlto_create_codegen(void) {
     assert(CGOptLevelOrNone);
     CodeGen->setCodeGenOptLevel(*CGOptLevelOrNone);
   }
+  if (!ThinLTOCacheDir.empty()) {
+    auto Err = llvm::sys::fs::create_directories(ThinLTOCacheDir.getValue());
+    if (Err)
+      report_fatal_error(Twine("Unable to create thinLTO cache directory: ") +
+                         Err.message());
+    bool result;
+    Err = llvm::sys::fs::is_directory(ThinLTOCacheDir.getValue(), result);
+    if (Err || !result)
+      report_fatal_error(Twine("Unable to get status of thinLTO cache path or "
+                               "path is not a directory: ") +
+                         Err.message());
+    CodeGen->setCacheDir(ThinLTOCacheDir);
+  }
+  CodeGen->setCachePruningInterval(ThinLTOCachePruningInterval);
+  CodeGen->setCacheEntryExpiration(ThinLTOCacheEntryExpiration);
+  CodeGen->setCacheMaxSizeFiles(ThinLTOCacheMaxSizeFiles);
+  CodeGen->setCacheMaxSizeBytes(ThinLTOCacheMaxSizeBytes);
+
   return wrap(CodeGen);
 }
 

[cachemeifyoucan]

Can you explain why you can't add flags to linker to call the API that sets the cache directly but want to rely on cl:opt?

Also if you want to rely on it in production, please add test, even we end up running on AIX only.

[github-actions]

🐧 Linux x64 Test Results

• 186372 tests passed
• 4855 tests skipped

[w2yehia]

Can you explain why you can't add flags to linker to call the API that sets the cache directly but want to rely on cl:opt?

Technically we can teach the platform linker to call the API, but users will only get it in the next OS release which is at least a year away. We have users on existing in-service OS releases that we want to provide this functionality to, and they are stuck with the platform linker they have.

[cachemeifyoucan]

I am not a big fan of adding this if this is only used for one year but I am also not strongly against it.

Please add a test to exercise the options.

[w2yehia]

Thanks Steven (@cachemeifyoucan ) for the review

Please add a test to exercise the options.

Since these options are added to lto.cpp which is only used to build libLTO.so, I don't think we test libLTO.so in LIT, do we?

[cachemeifyoucan]

Sure. Adding a new tool to test libLTO is probably overkill for this change. SGTM