Write test for virtual memory space limits

[norov]

During the testing of arm64/ilp32 it was discovered a bug related to how kernel internal machinery handles limits of userspace memory.

https://lkml.org/lkml/2019/5/13/129

$ ulimit -s
8192
$ ./hello.ilp32
Hello World!
$ ulimit -s unlimited
$ ./hello.ilp32
Segmentation fault
$ strace ./hello.ilp32
execve("./hello.ilp32", ["./hello.ilp32"], 0xfffff10548f0 /* 77 vars */) = -1 ENOMEM (Cannot allocate memory)
+++ killed by SIGSEGV +++
Segmentation fault (core dumped)

I use LTP to test my branch, and it seems there is no testcase that triggers an alarm. The discovered reason for a bug is in functionality introduced at the end of 2018 - DEFAULT_MAP_WINDOW. Adding the new ABI to arm64 requires a small modification of the macro:
diff --git a/arch/arm64/include/asm/processor.h b/arch/arm64/include/asm/processor.h
index 5bdf357169d8..c509f83fa506 100644
--- a/arch/arm64/include/asm/processor.h
+++ b/arch/arm64/include/asm/processor.h
@@ -63,7 +63,7 @@
TASK_SIZE_32 : TASK_SIZE_64)
#define TASK_SIZE_OF(tsk) (is_compat_thread(tsk) ?
TASK_SIZE_32 : TASK_SIZE_64)
-#define DEFAULT_MAP_WINDOW (test_thread_flag(TIF_32BIT) ?
+#define DEFAULT_MAP_WINDOW (is_compat_task() ?
TASK_SIZE_32 : DEFAULT_MAP_WINDOW_64)
#else
#define TASK_SIZE TASK_SIZE_64

Otherwise, for the new ilp32 runtime DEFAULT_MAP_WINDOW would be just like for lp64 which leads to over-the-boundary address assignment in the case of 'ulimit -s unlimited'.

In unlimited case, execve() creates a vdso area close to the DEFAULT_MAP_WINDOW which was erroneously set out of 32-bit space. This is the exact scenario of how the bug was spotted out by Andreas Schwab.

Because of a lack for a testcase it took almost half a year to discover the bug (and 1 day to fix it). I'd like to encourage LTP community to introduce a testcase for this scenario.

Thanks,
Yury

[metan-ucw]

Looking at the report do I get it right that you compiled LTP with ilp32 ABI and all that we need to do is to write a test that sets stack limit to unlimited then does fork() + execve()?

[norov]

Yes. It would reproduce the bug as Andreas reported, on this branch.
https://github.com/norov/linux/tree/ilp32-5.1

I think it would make sense in scenario to take a set of relevant existing tests and run them again with ulimit -s unlimited.

[coolgw]

https://patchwork.ozlabs.org/project/ltp/patch/20231021122958.13458-2-wegao@suse.com/