Skip to content

Conversation

boomanaiden154
Copy link
Contributor

Some of the ulimit limits do not work on some POSIX platforms. THe motivating example here is ulimit -v on MacOS as the relevant system calls are not implemented in XNU. Splitting the tests lets us keep test coverage on POSIX (non-Linux) platforms.

Some of the ulimit limits do not work on some POSIX platforms. THe
motivating example here is ulimit -v on MacOS as the relevant system
calls are not implemented in XNU. Splitting the tests lets us keep test
coverage on POSIX (non-Linux) platforms.
@llvmbot
Copy link
Member

llvmbot commented Sep 13, 2025

@llvm/pr-subscribers-testing-tools

Author: Aiden Grossman (boomanaiden154)

Changes

Some of the ulimit limits do not work on some POSIX platforms. THe motivating example here is ulimit -v on MacOS as the relevant system calls are not implemented in XNU. Splitting the tests lets us keep test coverage on POSIX (non-Linux) platforms.


Full diff: https://github.com/llvm/llvm-project/pull/158390.diff

4 Files Affected:

  • (added) llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/lit.cfg (+8)
  • (added) llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/ulimit_okay.txt (+4)
  • (modified) llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_okay.txt (-1)
  • (modified) llvm/utils/lit/tests/shtest-ulimit.py (+7-8)
diff --git a/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/lit.cfg b/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/lit.cfg
new file mode 100644
index 0000000000000..c7bdc7e7b6bc0
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/lit.cfg
@@ -0,0 +1,8 @@
+import lit.formats
+
+config.name = "shtest-ulimit"
+config.suffixes = [".txt"]
+config.test_format = lit.formats.ShTest(execute_external=False)
+config.test_source_root = None
+config.test_exec_root = None
+config.substitutions.append(("%{python}", '"%s"' % (sys.executable)))
diff --git a/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/ulimit_okay.txt b/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/ulimit_okay.txt
new file mode 100644
index 0000000000000..dbdd0037e70a7
--- /dev/null
+++ b/llvm/utils/lit/tests/Inputs/shtest-ulimit-linux/ulimit_okay.txt
@@ -0,0 +1,4 @@
+# RUN: ulimit -v 1048576
+# RUN: %{python} %S/../shtest-ulimit/print_limits.py
+# Fail the test so that we can assert on the output.
+# RUN: not echo return
diff --git a/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_okay.txt b/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_okay.txt
index ad353b5d7c459..4edf1c303a092 100644
--- a/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_okay.txt
+++ b/llvm/utils/lit/tests/Inputs/shtest-ulimit/ulimit_okay.txt
@@ -1,4 +1,3 @@
-# RUN: ulimit -v 1048576
 # RUN: ulimit -n 50
 # RUN: %{python} %S/print_limits.py
 # Fail the test so that we can assert on the output.
diff --git a/llvm/utils/lit/tests/shtest-ulimit.py b/llvm/utils/lit/tests/shtest-ulimit.py
index b86578a21f661..444937f21eb02 100644
--- a/llvm/utils/lit/tests/shtest-ulimit.py
+++ b/llvm/utils/lit/tests/shtest-ulimit.py
@@ -3,13 +3,8 @@
 # ulimit does not work on non-POSIX platforms.
 # UNSUPPORTED: system-windows
 
-# TODO(boomanaiden154): The test fails on some non-Linux POSIX
-# platforms (like MacOS) due to the underlying system not supporting
-# ulimit -v. This test needs to be carved up so we keep full test
-# coverage on Linux and as much as possible on other platforms.
-# REQUIRES: system-linux
-
 # RUN: not %{lit} -a -v %{inputs}/shtest-ulimit | FileCheck %s
+# RUN: %if system-linux %{ not %{lit} -a -v %{inputs}/shtest-ulimit-linux | FileCheck %s --check-prefix CHECK-LINUX %}
 
 # CHECK: -- Testing: 2 tests{{.*}}
 
@@ -18,7 +13,11 @@
 # CHECK: 'ulimit' requires two arguments
 
 # CHECK-LABEL: FAIL: shtest-ulimit :: ulimit_okay.txt ({{[^)]*}})
-# CHECK: ulimit -v 1048576
 # CHECK: ulimit -n 50
-# CHECK: RLIMIT_AS=1073741824
 # CHECK: RLIMIT_NOFILE=50
+
+# CHECK-LINUX: -- Testing: 1 tests{{.*}}
+
+# CHECK-LINUX-LABEL: FAIL: shtest-ulimit :: ulimit_okay.txt ({{[^)]*}})
+# CHECK-LINUX: ulimit -v 1048576
+# CHECK-LINUX: RLIMIT_AS=1073741824

@boomanaiden154 boomanaiden154 merged commit 87cca0c into llvm:main Sep 14, 2025
10 of 11 checks passed
@boomanaiden154 boomanaiden154 deleted the lit-ulimit-linux-tests branch September 14, 2025 07:04
@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 14, 2025

LLVM Buildbot has detected a new failure on builder lldb-remote-linux-ubuntu running on as-builder-9 while building llvm at step 16 "test-check-lldb-api".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/195/builds/14561

Here is the relevant piece of the build log for the reference
Step 16 (test-check-lldb-api) failure: Test just built components: check-lldb-api completed (failure)
...
UNSUPPORTED: lldb-api :: functionalities/postmortem/FreeBSDKernel/TestFreeBSDKernelVMCore.py (568 of 1305)
UNSUPPORTED: lldb-api :: functionalities/postmortem/elf-core/expr/TestExpr.py (569 of 1305)
PASS: lldb-api :: functionalities/plugins/python_os_plugin/TestPythonOSPlugin.py (570 of 1305)
PASS: lldb-api :: functionalities/postmortem/elf-core/TestLinuxCore.py (571 of 1305)
PASS: lldb-api :: functionalities/postmortem/elf-core/gcore/TestGCore.py (572 of 1305)
PASS: lldb-api :: functionalities/pointer_num_children/TestPointerNumChildren.py (573 of 1305)
PASS: lldb-api :: functionalities/postmortem/elf-core/thread_crash/TestLinuxCoreThreads.py (574 of 1305)
PASS: lldb-api :: functionalities/postmortem/minidump/TestMiniDump.py (575 of 1305)
PASS: lldb-api :: functionalities/postmortem/minidump-new/TestMiniDumpNew.py (576 of 1305)
UNRESOLVED: lldb-api :: functionalities/postmortem/netbsd-core/TestNetBSDCore.py (577 of 1305)
******************** TEST 'lldb-api :: functionalities/postmortem/netbsd-core/TestNetBSDCore.py' FAILED ********************
Script:
--
/usr/bin/python3.12 /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/llvm-project/lldb/test/API/dotest.py -u CXXFLAGS -u CFLAGS --env LLVM_LIBS_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib --env LLVM_INCLUDE_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include --env LLVM_TOOLS_DIR=/home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin --libcxx-include-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include/c++/v1 --libcxx-include-target-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/include/aarch64-unknown-linux-gnu/c++/v1 --libcxx-library-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib/aarch64-unknown-linux-gnu --arch aarch64 --build-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex --lldb-module-cache-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex/module-cache-lldb/lldb-api --clang-module-cache-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/lldb-test-build.noindex/module-cache-clang/lldb-api --executable /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin/lldb --compiler /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/bin/clang --dsymutil /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin/dsymutil --make /usr/bin/gmake --llvm-tools-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./bin --lldb-obj-root /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/tools/lldb --lldb-libs-dir /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/build/./lib --cmake-build-type Release --platform-url connect://jetson-agx-2198.lab.llvm.org:1234 --platform-working-dir /home/ubuntu/lldb-tests --sysroot /mnt/fs/jetson-agx-ubuntu --env ARCH_CFLAGS=-mcpu=cortex-a78 --platform-name remote-linux --skip-category=lldb-server /home/buildbot/worker/as-builder-9/lldb-remote-linux-ubuntu/llvm-project/lldb/test/API/functionalities/postmortem/netbsd-core -p TestNetBSDCore.py
--
Exit Code: -6

Command Output (stdout):
--
lldb version 22.0.0git (https://github.com/llvm/llvm-project.git revision 87cca0c32c4e3ce51b53cf5c3ee0a430c1a6cb8f)
  clang revision 87cca0c32c4e3ce51b53cf5c3ee0a430c1a6cb8f
  llvm revision 87cca0c32c4e3ce51b53cf5c3ee0a430c1a6cb8f
Setting up remote platform 'remote-linux'
Connecting to remote platform 'remote-linux' at 'connect://jetson-agx-2198.lab.llvm.org:1234'...
Connected.
Setting remote platform working directory to '/home/ubuntu/lldb-tests'...
Skipping the following test categories: ['lldb-server', 'msvcstl', 'dsym', 'gmodules', 'debugserver', 'objc', 'lldb-dap']

--
Command Output (stderr):
--
WARNING:root:Custom libc++ is not supported for remote runs: ignoring --libcxx arguments
double free or corruption (fasttop)
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace and instructions to reproduce the bug.
Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it):
0  _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a76698082
1  _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a76694f7f
2  _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a766950cc
3  libc.so.6                             0x00007c4a7d845330
4  libc.so.6                             0x00007c4a7d89eb2c pthread_kill + 284
5  libc.so.6                             0x00007c4a7d84527e gsignal + 30
6  libc.so.6                             0x00007c4a7d8288ff abort + 223
7  libc.so.6                             0x00007c4a7d8297b6
8  libc.so.6                             0x00007c4a7d8a8ff5
9  libc.so.6                             0x00007c4a7d8ab32a
10 libc.so.6                             0x00007c4a7d8addae __libc_free + 126
11 _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a7644e44f
12 _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a75bd075e
13 _lldb.cpython-312-x86_64-linux-gnu.so 0x00007c4a760c3d9a

@llvm-ci
Copy link
Collaborator

llvm-ci commented Sep 14, 2025

LLVM Buildbot has detected a new failure on builder clang-solaris11-sparcv9 running on solaris11-sparcv9 while building llvm at step 5 "ninja check 1".

Full details are available at: https://lab.llvm.org/buildbot/#/builders/13/builds/9547

Here is the relevant piece of the build log for the reference
Step 5 (ninja check 1) failure: stage 1 checked (failure)
******************** TEST 'lit :: shtest-ulimit.py' FAILED ********************
Exit Code: 1

Command Output (stdout):
--
# RUN: at line 6
not env -u FILECHECK_OPTS "/usr/bin/python3.11" /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/utils/lit/lit.py -j1 --order=lexical -a -v Inputs/shtest-ulimit | FileCheck /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/shtest-ulimit.py
# executed command: not env -u FILECHECK_OPTS /usr/bin/python3.11 /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/utils/lit/lit.py -j1 --order=lexical -a -v Inputs/shtest-ulimit
# .---command stderr------------
# | lit.py: /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/llvm/llvm/utils/lit/lit/run.py:142: warning: Failed to raise process limit: module 'resource' has no attribute 'RLIMIT_NPROC'
# | 
# | 1 warning(s) in tests
# `-----------------------------
# executed command: FileCheck /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/shtest-ulimit.py
# .---command stderr------------
# | /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/shtest-ulimit.py:16:10: error: CHECK: expected string not found in input
# | # CHECK: RLIMIT_NOFILE=50
# |          ^
# | <stdin>:26:13: note: scanning from here
# | ulimit -n 50
# |             ^
# | <stdin>:33:5: note: possible intended match here
# | # | RLIMIT_NOFILE=48
# |     ^
# | 
# | Input file: <stdin>
# | Check file: /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/shtest-ulimit.py
# | 
# | -dump-input=help explains the following input dump.
# | 
# | Input was:
# | <<<<<<
# |             .
# |             .
# |             .
# |            21: Exit Code: 1 
# |            22:  
# |            23: Command Output (stdout): 
# |            24: -- 
# |            25: # RUN: at line 1 
# |            26: ulimit -n 50 
# | check:16'0                 X error: no match found
# |            27: # executed command: ulimit -n 50 
# | check:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            28: # RUN: at line 2 
# | check:16'0     ~~~~~~~~~~~~~~~~~
# |            29: "/usr/bin/python3.11" /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/Inputs/shtest-ulimit/print_limits.py 
# | check:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
# |            30: # executed command: /usr/bin/python3.11 /opt/llvm-buildbot/home/solaris11-sparcv9/clang-solaris11-sparcv9/stage1/utils/lit/tests/Inputs/shtest-ulimit/print_limits.py 
# | check:16'0     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants