If a user process tries to access a stack page for the first time using a system call, it may cause a page fault.
This problem was seen with the make program while building linux but the setup to reproduce the problem would be impractical, so I created a small test program. It took quite a bit of investigation and testing to isolate the specific problem.
The following code can be used to trigger the problem. Admittedly, this test is not something anyone would run into casually, but I do believe it this is "valid" code that should work and represents what I think make was doing when it faulted.
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/stat.h>
#include <errno.h>
int main() {
char filename[3670] = "stacktest";
struct stat statinfo;
/*
puts("hello");
__asm__ (
"mov $0x6A,%eax\t\n"
"mov $0xbffff00a,%ebx\t\n"
"mov $0xbfffefc8,%ecx\t\n"
"int $0x80"
);
printf("my size is %u\n", statinfo.st_size);
*/
printf("filename is 0x%08x\n", (unsigned int) filename);
printf("statinfo is 0x%08x\n", (unsigned int) &statinfo);
}
You may need to do perform some manual adjustments to make this work.
First, compile and run the program as is:
cc stacktest.c -o stacktest
./stacktest
filename is 0xbffff00a
statinfo is 0xbfffefc8
Note the address of filename and statinfo. It is important that filename and statinfo are in different memory pages. You may need to adjust the size of filename in order to make this happen.
Now change the assembly code (if necessary) to match those addresses for setting %ebx and %ecx, respectively, and uncomment that code.
You should be able to run the program now and get the correct size of the file:
cc stacktest.c -o stacktest
./stacktest
hello
my size is 586769
filename is 0xbffff00a
statinfo is 0xbfffefc8
The assembly code is executing a stat system call and is the equivalent of stat(filename, &statinfo);.
Now, edit the code and remove the line puts("hello");. Compile and run again. For me, this produces a segfault.
The issue is that the stack page for statinfo is not mapped in.
Normally, any access of an unmapped stack page from user land would cause the page to be mapped in by a page fault. The puts("hello"); call triggers this mechanism when it pushes a call frame onto the stack. However, when that is removed, the program is using a system call to write to that memory for the first time. In this case, the fault occurs in the kernel and it appears the normal mechanism for growing the stack does not trigger and a page fault terminates the program instead.
It seems there are a couple of ways to resolve this. At first, I thought the problem was simply that a lack of sufficient stack space was being mapped for the user program. I developed a solution with that in mind. I wrote a change in which the user program
is mapped a significant chunk of free stack from the start of execution. (See attached PR). This indeed resolves the problem.
After trying to create a test case I came to understand the current mechanism for growing the user stack dynamically on a fault, which examines the user process %esp register. It might be possible to create a similar mechanism for growing the stack when a
fault occurs during a system call. On the other hand, I don't see a downside to just setting up more stack memory in the vma to begin with.
If a user process tries to access a stack page for the first time using a system call, it may cause a page fault.
This problem was seen with the
makeprogram while building linux but the setup to reproduce the problem would be impractical, so I created a small test program. It took quite a bit of investigation and testing to isolate the specific problem.The following code can be used to trigger the problem. Admittedly, this test is not something anyone would run into casually, but I do believe it this is "valid" code that should work and represents what I think
makewas doing when it faulted.You may need to do perform some manual adjustments to make this work.
First, compile and run the program as is:
Note the address of filename and statinfo. It is important that filename and statinfo are in different memory pages. You may need to adjust the size of filename in order to make this happen.
Now change the assembly code (if necessary) to match those addresses for setting %ebx and %ecx, respectively, and uncomment that code.
You should be able to run the program now and get the correct size of the file:
The assembly code is executing a
statsystem call and is the equivalent ofstat(filename, &statinfo);.Now, edit the code and remove the line
puts("hello");. Compile and run again. For me, this produces a segfault.The issue is that the stack page for statinfo is not mapped in.
Normally, any access of an unmapped stack page from user land would cause the page to be mapped in by a page fault. The
puts("hello");call triggers this mechanism when it pushes a call frame onto the stack. However, when that is removed, the program is using a system call to write to that memory for the first time. In this case, the fault occurs in the kernel and it appears the normal mechanism for growing the stack does not trigger and a page fault terminates the program instead.It seems there are a couple of ways to resolve this. At first, I thought the problem was simply that a lack of sufficient stack space was being mapped for the user program. I developed a solution with that in mind. I wrote a change in which the user program
is mapped a significant chunk of free stack from the start of execution. (See attached PR). This indeed resolves the problem.
After trying to create a test case I came to understand the current mechanism for growing the user stack dynamically on a fault, which examines the user process %esp register. It might be possible to create a similar mechanism for growing the stack when a
fault occurs during a system call. On the other hand, I don't see a downside to just setting up more stack memory in the vma to begin with.