[lit] Add LIT_CURRENT_TESTCASE environment variable when running tests

[ndrewh]

I'm not aware of any way for %run wrapper scripts like iosssim_run.py (ref) to know what testcase they are currently running. This can be useful if these wrappers need to create a (potentially remote) temporary directory for each test case.

This adds the LIT_CURRENT_TESTCASE environment variable to both the internal shell and the external shell, containing the full name of the current test being run.

[llvmbot]

@llvm/pr-subscribers-testing-tools

Author: Andrew Haberlandt (ndrewh)

Changes

I'm not aware of any way for %run wrapper scripts like iosssim_run.py (ref) to know what testcase they are currently running. This can be useful if these wrappers need to create a (potentially remote) temporary directory for each test case.

This adds the LIT_CURRENT_TESTCASE environment variable to both the internal shell and the external shell, containing the full name of the current test being run.

---

Full diff: https://github.com/llvm/llvm-project/pull/168762.diff

1 Files Affected:

• (modified) llvm/utils/lit/lit/TestRunner.py (+5-1)

diff --git a/llvm/utils/lit/lit/TestRunner.py b/llvm/utils/lit/lit/TestRunner.py
index 64148c6098327..9525320f133c6 100644
--- a/llvm/utils/lit/lit/TestRunner.py
+++ b/llvm/utils/lit/lit/TestRunner.py
@@ -1226,6 +1226,8 @@ def executeScriptInternal(
     results = []
     timeoutInfo = None
     shenv = ShellEnvironment(cwd, test.config.environment)
+    shenv.env["LIT_CURRENT_TESTCASE"] = test.getFullName()
+
     exitCode, timeoutInfo = executeShCmd(
         cmd, shenv, results, timeout=litConfig.maxIndividualTestTime
     )
@@ -1425,11 +1427,13 @@ def executeScript(test, litConfig, tmpBase, commands, cwd):
             # run on clang with no real loss.
             command = litConfig.valgrindArgs + command
 
+    env = dict(test.config.environment)
+    env["LIT_CURRENT_TESTCASE"] = test.getFullName()
     try:
         out, err, exitCode = lit.util.executeCommand(
             command,
             cwd=cwd,
-            env=test.config.environment,
+            env=env,
             timeout=litConfig.maxIndividualTestTime,
         )
         return (out, err, exitCode, None)

[boomanaiden154]

This seems like a reasonable enough feature to me for the stated use case.

It would be good if this had a test though, at least for the internal shell.

[github-actions]

🐧 Linux x64 Test Results

• 186484 tests passed
• 4859 tests skipped

[ndrewh]

@boomanaiden154 I added a testcase.

Apparently env (with no args) does not correctly chain with | FileCheck %s, so every testcase in env-no-subcommand.txt is broken (passes no matter the output). I don't know how to fix that easily, so I'm just going out to bash here for this test.

[boomanaiden154]

Apparently env (with no args) does not correctly chain with | FileCheck %s, so every testcase in env-no-subcommand.txt is broken (passes no matter the output). I don't know how to fix that easily, so I'm just going out to bash here for this test.

Yeah, that's a known issue with how we handle builtins in lit. It's on my TODO list to fix, but hasn't been super high priority. Bash in this case seems fine to me.

[boomanaiden154]

LGTM, thanks for the tests.

[jh7370]

@boomanaiden154 I added a testcase.

Apparently env (with no args) does not correctly chain with | FileCheck %s, so every testcase in env-no-subcommand.txt is broken (passes no matter the output). I don't know how to fix that easily, so I'm just going out to bash here for this test.

No idea if this would work, but could you do env > %t.txt followed by FileCheck %s --input-file %t.txt on the next line or something like that?

[boomanaiden154]

No idea if this would work, but could you do env > %t.txt followed by FileCheck %s --input-file %t.txt on the next line or something like that?

I still don't think that works. The issue is that env is executed as a builtin and doesn't have support for writing to the stdin file descriptor of whatever comes after it.