[lit] Ensure ulimit does not persist across tests

[boomanaiden154]

When constructing a container in a default argument in Python, we actually end up with a reference to the same default container for every invocation of the function. Given this happened with the ulimit variable in ShellEnvironment, we ended up persisting limits across tests. This would cause some LLVM tests to fail, particularly jitlink and dsymutil tests, if they ended up running after the one clang test that uses ulimit -v to set a maximum amount of memory that can be allocated. This patch fixes that behavior by constructing the dict inside the function to ensure we get a new instance and adds test coverage.

[llvmbot]

@llvm/pr-subscribers-testing-tools

Author: Aiden Grossman (boomanaiden154)

Changes

When constructing a container in a default argument in Python, we actually end up with a reference to the same default container for every invocation of the function. Given this happened with the ulimit variable in ShellEnvironment, we ended up persisting limits across tests. This would cause some LLVM tests to fail, particularly jitlink and dsymutil tests, if they ended up running after the one clang test that uses ulimit -v to set a maximum amount of memory that can be allocated. This patch fixes that behavior by constructing the dict inside the function to ensure we get a new instance and adds test coverage.

---

Full diff: https://github.com/llvm/llvm-project/pull/164485.diff

3 Files Affected:

• (modified) llvm/utils/lit/lit/TestRunner.py (+2-2)
• (added) llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_reset.txt (+3)
• (modified) llvm/utils/lit/tests/shtest-ulimit.py (+5-2)

diff --git a/llvm/utils/lit/lit/TestRunner.py b/llvm/utils/lit/lit/TestRunner.py
index a7e2705f609af..f88314547bb3f 100644
--- a/llvm/utils/lit/lit/TestRunner.py
+++ b/llvm/utils/lit/lit/TestRunner.py
@@ -92,12 +92,12 @@ class ShellEnvironment(object):
     we maintain a dir stack for pushd/popd.
     """
 
-    def __init__(self, cwd, env, umask=-1, ulimit={}):
+    def __init__(self, cwd, env, umask=-1, ulimit=None):
         self.cwd = cwd
         self.env = dict(env)
         self.umask = umask
         self.dirStack = []
-        self.ulimit = ulimit
+        self.ulimit = ulimit if ulimit else {}
 
     def change_dir(self, newdir):
         if os.path.isabs(newdir):
diff --git a/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_reset.txt b/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_reset.txt
new file mode 100644
index 0000000000000..011d6db6c127f
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_reset.txt
@@ -0,0 +1,3 @@
+# RUN: %{python} %S/print_limits.py
+# Fail the test so that we can assert on the output.
+# RUN: not echo return
diff --git a/llvm/utils/lit/tests/shtest-ulimit.py b/llvm/utils/lit/tests/shtest-ulimit.py
index e84327772d3a1..9a8febd3333c4 100644
--- a/llvm/utils/lit/tests/shtest-ulimit.py
+++ b/llvm/utils/lit/tests/shtest-ulimit.py
@@ -5,9 +5,9 @@
 # as well.
 # UNSUPPORTED: system-windows, system-solaris
 
-# RUN: not %{lit} -a -v %{inputs}/shtest-ulimit | FileCheck %s
+# RUN: not %{lit} -a -v %{inputs}/shtest-ulimit --order=lexical | FileCheck %s
 
-# CHECK: -- Testing: 2 tests{{.*}}
+# CHECK: -- Testing: 3 tests{{.*}}
 
 # CHECK-LABEL: FAIL: shtest-ulimit :: ulimit-bad-arg.txt ({{[^)]*}})
 # CHECK: ulimit -n
@@ -16,3 +16,6 @@
 # CHECK-LABEL: FAIL: shtest-ulimit :: ulimit_okay.txt ({{[^)]*}})
 # CHECK: ulimit -n 50
 # CHECK: RLIMIT_NOFILE=50
+
+# CHECK-LABEL: FAIL: shtest-ulimit :: ulimit_reset.txt ({{[^)]*}})
+# CHECK-NOT: RLIMIT_NOFILE=50

[arichardson]

Good catch! I wonder if we should run tools like ruff over the lit files to catch other potential issues?

[boomanaiden154]

Good catch! I wonder if we should run tools like ruff over the lit files to catch other potential issues?

Maybe. I feel like getting a linter with a reasonable set of rules clean on a codebase like lit's is going to be a decent amount of work though.