runtime: x_cgo_init determines stacklo incorrectly for some Solaris versions

[binarycrusader]

This is basically a continuation of issue #12210.

As @rsc guessed, Solaris grows the stack as faults happen. This behavior is described in section 5.3.5, on page 136 of Mauro, J., & McDougall, R. (2001). Solaris internals: Core kernel components:

The process stack is mapped into the address space with an initial allocation and then grows downward. The stack, like the heap, grows on demand, but no library grows the stack; instead, a different mechanism triggers this growth.

Initially, a single page is allocated for the stack, and as the process executes and calls functions, it pushes the program counter, arguments, and local variables onto the stack. When the stack grows larger than one page, the process causes a page fault, and the kernel notices that this is a stack segment page fault and grows the stack segment.

Prior to Solaris 11 (e.g. OpenSolaris and derivatives), when the stack size was set to unlimited, libc_init() (which is responsible for the initialization of the main stack) would set ss_size = 0 and ss_sp = ul_stktop initially and would then update that as stack faults were encountered.

A simple (naive) test program on a pre-Solaris 11 system:

#include <ucontext.h>
#include <stdio.h>
#include <sys/resource.h>
#include <inttypes.h>

int main(int argc, char **argv) {
  ucontext_t uc;
  struct rlimit lim;
  stack_t ss;

  getcontext(&uc);
  printf("ss_sp = %p\n", uc.uc_stack.ss_sp);
  printf("ss_size = %" PRId64 "\n", uc.uc_stack.ss_size);

  getrlimit(RLIMIT_STACK, &lim);
  printf("RLIMIT_STACK = %" PRId64 "\n", lim.rlim_cur);
}

...produces these results:

$ gcc -m64 -o s main.c ; ./s
ss_sp = fffffd7fffdfb000
ss_size = 8192
RLIMIT_STACK = -3

There's nothing wrong with the results since posix doesn't require specific behaviors in regards to these values and in fact, getcontext(), et al. are considered obsolete (they were removed from the 2013 version of the posix standard: http://pubs.opengroup.org/onlinepubs/9699919799/xrat/V4_xsh_chap03.html).

Now, on Solaris 11 and later, the same test program when executed produces different results:

ss_sp = ffff800000000000
ss_size = 1098437885952
RLIMIT_STACK = -3

As such, for Solaris 11+, the current code in gcc_solaris_amd64.c in x_cgo_init() should generally work fine for determining stacklo in the unlimited stack size case.

However, for older releases the correct answer is to actually call getrlimit(RLIMIT_STACK, ...) and calculate the low address using rlim_cur (unless it returns -3, RLIM_INFINITY), in which case it can safely assume 2GB.

[rsc]

Remind me which systems are affected by this and what the effect is?

If it's the case that only older Solarises are affected and that the effect is that less stack is used than is actually available (e.g., only 1 MB even though there is arguably 2 GB), then it seems like maybe we can just ignore it and wait for people who need multiple MB of system stack to update to the newer Solaris.

[binarycrusader]

All versions of Solaris before Solaris 11.0 and potentially any OpenSolaris-based derivatives (SmartOS, Illumos, Tribblix, OpenIndiana, etc.).

I have verified that the latest supported release of Solaris 10 exhibits this issue, I am unable to verify which versions of OpenSolaris-based derivatives exhibit this issue, although I would note that there is at least one version of SmartOS where this problem was seen as described in issue #12210.

[agrellum]

agrellum@claudville:$ uname -a
SunOS claudville 5.11 illumos-a3bcc60de1 i86pc i386 i86pc
agrellum@claudville:$ cat /etc/release
OpenIndiana Hipster 2017.04 (powered by illumos)
OpenIndiana Project, part of The Illumos Foundation (C) 2010-2017
Use is subject to license terms.
Assembled 01 May 2017
agrellum@claudville:~$ gcc -m64 -o s main.c ; ./s
ss_sp = fffffd7fff400000
ss_size = 10485760
RLIMIT_STACK = 10485760