Skip to content

runtime: Fix sigaltstack call with musl on certain Intel CPUs - #21

Merged
nmote merged 1 commit into
5.3.0-semgrepfrom
nat/sigstksz
Jul 14, 2026
Merged

runtime: Fix sigaltstack call with musl on certain Intel CPUs#21
nmote merged 1 commit into
5.3.0-semgrepfrom
nat/sigstksz

Conversation

@nmote

@nmote nmote commented Jul 14, 2026

Copy link
Copy Markdown
Member

The problem

On Friday we started sporadically seeing some test failures in CI with the error message Fatal error: Failed to allocate signal stack for domain 0. It turned out that in order to mitigate a capacity incident, Depot added Intel runners to their fleet, which was previously AMD only. These errors occurred consistently when we were allocated an Intel runner.

The OCaml runtime calls sigaltstack with the size of the stack set to SIGSTKSZ. Musl libc defines that as a build-time constant.

Musl 1.2.6 includes a change that errors if the size passed to sigaltstack is less than sysconf(_SC_MINSIGSTKSZ). This value is determined at runtime based on AT_MINSIGSTKSZ. The kernel determines AT_MINSIGSTKSZ based in part on constants provided by the CPU. As a result, sysconf(_SC_MINSIGSTKSZ) can be greater than SIGSTKSZ, leading to a fatal error in the OCaml runtime when linked against musl 1.2.6 or later.

The production Semgrep image is currently based on Alpine 3.23, which ships with musl 1.2.5, which lacks the enforcement of the minimum. Thus, we are currently safe from this issue in production. Alpine 3.24 ships with musl 1.2.6, so we need to address this issue before upgrading.

The fix

The fix is fairly straightforward. We dynamically choose the size of the signal stack based on sysconf(_SC_MINSIGSTKSZ), when available. This logic is gated behind an #ifdef because on some platforms (e.g. macOS) it is not available.

Test plan

I put together a set of CI jobs that illustrates the issue and tests this change (Semgrep-internal): https://github.com/semgrep/semgrep-proprietary/actions/runs/29355721766

  • The -4 runners were hardcoded by Depot for us to always land on Intel, and -8 were hardcoded to always land on AMD.
  • The first failed job is a simple C program to illustrate the issue. On a Intel(R) Xeon(R) Platinum 8488C CPU, sysconf(_SC_MINSIGSTKSZ) = 12976 and SIGSTKSZ=8192. Calling sigaltstack with SIGSTKSZ succeeds on Alpine 3.23 (musl 1.2.5), but fails on Alpine 3.24 (musl 1.2.6) because the size is below the minimum.
  • The second failed job illustrates the failure within the OCaml runtime on the same Intel chip and musl 1.2.6: Fatal error: Failed to allocate signal stack for domain 0: Out of memory using OCaml 5.3.0-semgrep.
  • In contrast, both the Intel and AMD variants of the hello world program pass when we use OCaml from this branch.

@nmote
nmote requested a review from dijkstracula July 14, 2026 18:29
@nmote

nmote commented Jul 14, 2026

Copy link
Copy Markdown
Member Author

I'll port this to our 5.4.0 branch, our tsan branches, and I'll put an upstream PR up once we've validated it a bit with Semgrep.

@dijkstracula dijkstracula left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, nice job debugging this.

Comment thread runtime/signals.c
size_t size = SIGSTKSZ;
#ifdef _SC_MINSIGSTKSZ
/* glibc/musl only */
long min = sysconf(_SC_MINSIGSTKSZ);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want/need to handle sysconf returning -1 (and setting errno)? I think this would only happen if _SC_MINSIGSTKSZ is not a valid system limit, which you gate via the macro, so maybe not a concern but just throwing it out there.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, and we already do (see the line below this one). I believe that it's possible for _SC_MINSIGSTKSZ to be a valid constant at build time but for the built binary to link against a different libc for which it is not a valid constant. In that case, sysconf would error, but this is handled by the min > 0 check below.

@nmote
nmote merged commit 3499e57 into 5.3.0-semgrep Jul 14, 2026
28 checks passed
yosefAlsuhaibani pushed a commit to semgrep/semgrep that referenced this pull request Jul 15, 2026
…63) (semgrep/semgrep-proprietary#6752)

This pulls in semgrep/ocaml#19 and semgrep/ocaml#21. See the latter for a description of the fatal error in the runtime that this will fix.

Test plan: CI is green on this PR and the relevant jobs are green on the [test PR](semgrep/semgrep-proprietary#6753), which forces intel Depot runners, e.g. https://github.com/semgrep/semgrep-proprietary/actions/runs/29363621357/job/87191257620

synced from Pro 49302cf1304af25e9391bb02b17c121af97b45c9
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants