[AIX] Raise soft memory limits to hard limits

[daltenty]

AIX out-of-box memory soft limits are often
insufficient to run LLVM on reasonably size
inputs. Thus, we often encounter users who run
into spurious out of memory errors.

This change raises the memory soft limits
to the hard limits at LLVM startup to prevent
these types of issues.

[llvmbot]

@llvm/pr-subscribers-llvm-support

Author: David Tenty (daltenty)

Changes

AIX out-of-box memory soft limits are often
insufficient to run LLVM on reasonably size
inputs. Thus, we often encounter users who run
into spurious out of memory errors.

This change raises the memory soft limits
to the hard limits at LLVM startup to prevent
these types of issues.

---

Full diff: https://github.com/llvm/llvm-project/pull/167928.diff

1 Files Affected:

• (modified) llvm/lib/Support/InitLLVM.cpp (+30)

diff --git a/llvm/lib/Support/InitLLVM.cpp b/llvm/lib/Support/InitLLVM.cpp
index b8fbfd21c4f28..5c0f206d389de 100644
--- a/llvm/lib/Support/InitLLVM.cpp
+++ b/llvm/lib/Support/InitLLVM.cpp
@@ -32,6 +32,35 @@
 #endif
 #endif
 
+static void RaiseLimits() {
+#ifdef _AIX
+  // AIX has restrictive memory soft-limits out-of-box, so raise them if needed.
+  auto RaiseLimit = [](int resource) {
+    struct rlimit r;
+    getrlimit(resource, &r);
+
+    // Increase the soft limit to the hard limit, if necessary and
+    // possible.
+    if (r.rlim_cur != RLIM_INFINITY &&
+        r.rlim_cur != r.rlim_max) {
+      r.rlim_cur = r.rlim_max;
+      setrlimit(resource, &r);
+    }
+  };
+
+  // Address space size.
+  RaiseLimit(RLIMIT_AS);
+  // Heap size.
+  RaiseLimit(RLIMIT_DATA);
+  // Stack size.
+  RaiseLimit(RLIMIT_STACK);
+#ifdef RLIMIT_RSS
+  // Resident set size.
+  RaiseLimit(RLIMIT_RSS);
+#endif
+#endif
+}
+
 void CleanupStdHandles(void *Cookie) {
   llvm::raw_ostream *Outs = &llvm::outs(), *Errs = &llvm::errs();
   Outs->flush();
@@ -67,6 +96,7 @@ InitLLVM::InitLLVM(int &Argc, const char **&Argv,
   StackPrinter.emplace(Argc, Argv);
   sys::PrintStackTraceOnErrorSignal(Argv[0]);
   install_out_of_memory_new_handler();
+  RaiseLimits();
 
 #ifdef __MVS__
 

[github-actions]

✅ With the latest revision this PR passed the C/C++ code formatter.

[daltenty]

Note: writing a lit test for this is difficult (i.e. lit internal shell doesn't support setting separate soft/hard limits) and flaky (the soft vs hard limits needed to exhibit the out-of-memory behaviour change may shift over time), so I've opt-ed to forgo that.

[xingxue-ibm]

LGTM, thanks!