sandbox: kill the step before letting a refused notification go

[schivei]

Sandbox: kill the step before letting a refused notification go

A patch. stage0/, lib/ and tests/ are untouched; the code change is 25 lines in
src/sandbox.mc and src/seccomp.mc, nothing removed.

The flake

CI job The sandbox (linux/arm64), ROOT cell, case forkbomb (alt) (the --allow=threads
variant, cap 64) — twice in about ten runs (PR #27 run 33984786337, PR #29 run 33998821248, a
docs-only change):

     stdout [forked 64], want []
FAIL forkbomb (alt)

with the report line sandbox: refused: process limit (64) correct.

Reproduced first, on the Lima oracle

mc-k7, Ubuntu 26.04, kernel 7.0.0-30, aarch64, glibc; the compiler built from origin/main
(build/mc-linux-arm64-gnu), 100 runs of the single case per cell:

cell	runs with stdout	report
root	14 / 100	correct in all 100
unprivileged (kernel.apparmor_restrict_unprivileged_userns=0)	13 / 100	correct in all 100

The mechanism, measured

1. The errno names it. A copy of tests/sandbox/forkbomb.mc that prints
ld32(__errno_location()) of the fork that failed answered, on every failing run:

forked 64 errno 38          # ENOSYS

never EAGAIN (11), which is what the RLIMIT_NPROC backstop would have given.

ENOSYS is what the kernel hands a target whose pending SECCOMP_RET_USER_NOTIF is released
because the listener went away (seccomp_unotify(7)). So not answering the notification
(step C note 12) is only half of "a refused call never returns": the other half is that nobody
may close the listener while the step can still run.

2. Where the listener is closed. sb_go closes it as soon as sb_supervise() returns, and
the supervisor loop ends when the status pipe closes — about a millisecond after the kill it just
sent. The kill, meanwhile, has not arrived: sb_kill_box() SIGKILLs J and I, and the STEP (C)
dies only through zap_pid_ns_processes() in J's own exit path. A second probe put timestamped
traces in P; a failing iteration reads

[trace 91441392] refuse pid 324076
[trace 91441392] after kill_box
[trace 91441393] supervise: status pipe closed -> break
[trace 91441393] supervise: wait4(I)
[trace 91441393] sb_go: close the listener

— refusal, kill and close inside the same millisecond, with the step still alive.

3. The ordering proved from the other side. A third probe (a 2 s sleep in sb_refuse after
the kill, nothing else): 0 failures in 60 runs.

The fix

sb_refuse now does three things in one order, and the order is the whole of it:

    sb_kill_pid(ld32(sb_notifp() + SB_NF_PID));   // the task whose call this is
    sb_kill_box();                                // the box under it
    sb_wait_gone(SB_GONE_MS);                     // and only then let the notification go

• sb_kill_pid can name the task because seccomp_notif.pid is translated into the reader's
  pid namespace, which is P's — the same number process_vm_readv already takes in sb_vm_read.
• sb_wait_gone waits for POLLHUP on the listener, which is exactly filter->users == 0:
  no process under that filter is left, so no pending notification can be released into a program
  that could still act on it. It asks for no events, because POLLHUP is reported whatever
  the mask says and a notification queued behind the refused one must not wake it — answering one
  is precisely what must not happen there. Bounded at two seconds, the same grace the wall clock
  already gives the box after its own kill.

The expectation was not relaxed: forkbomb (alt) still asserts stdout [] and the report line
refused: process limit (64).

After the fix

Same oracle, same two cells, the final build, 200 iterations each:

== final build, root 200:
== 200 iterations: 0 with stdout, 0 with a wrong report
== final build, unprivileged 200:
== 200 iterations: 0 with stdout, 0 with a wrong report

Full suite, both cells:

== test-sandbox: 55 ok, 0 failed, 1 skipped      (root, Lima)
== test-sandbox: 55 ok, 0 failed, 1 skipped      (unprivileged, Lima)

sh scripts/sandbox-trace.sh --check green (the profile lists were not touched):

ok   compile: 25 calls, every call in the trace is in the table
ok   program: 22 calls, every call in the trace is in the table
ok   threads: 7 calls, every call in the trace is in the table
ok   src/sandbox_profiles.mc is what the lists generate

The refusal path costs no measurable time — the box's processes die at once, so the wait returns
on the first ppoll; the box overhead lines are plain 483 us, boxed 4187 us (root) and
plain 472 us, boxed 4084 us (unprivileged), the machine's usual figures.

Gates (macOS)

make bundle was re-run BEFORE bootstrapping (93 files, raw 1170795 -> LZ 548311, blob 549473 B).

make check RC 0, zero FAIL, 7m59s:

32/32 tests passed                      (test)
145/145 files identical (3 skipped)     (check-lex)
146/146 files identical                 (check-ast)
146/146 files identical                 (check-asm)
32/32 objects identical                 (check-obj, against the frozen seed)
  ok: build/mc2.o == build/mc3.o        (bootstrap, 1263304 bytes)
  dump-asm diff mc1 vs mc2: 0 lines
32/32 objects identical                 (check-surface)
32/32 tests passed via --exe            (test-exe)
check-pkg: 85/85
17/17 seed limits under 90%             (check-limits)
41/41 tests passed on linux/aarch64 · 39/39 on linux/x86_64
44/44 + 44/44 + 42/42 + 42/42           (the four --exe cells, musl and gnu)
42/42 and 40/40 objects cross-compiled for windows/aarch64 and windows/x86_64
check-float: ok · check-wide: ok · check-kernel · check-avr · check-desktop
docs ok: 197 symbols, 36 flags, 27 toml keys, 10 directives, 51 samples, 361 links
mcsite: 89 pages, 5 sections · mcsite --check: 89 pages, 0 link problems
== test-sandbox: 55 ok, 0 failed, 1 skipped      (delegated to Lima)

make check-linux-host RC 0, all four cells, each after its own mc2l.o == mc3l.o and with
the cross proof green:

== linux/aarch64 host, musl: ok ==   41/41 · test-exe 31/31 · check-obj 31/31
== linux/aarch64 host, gnu:  ok ==   42/42 native
== linux/x86_64  host, musl: ok ==   39/39 · test-exe 29/29
== linux/x86_64  host, gnu:  ok ==   40/40 native
ok: the Mach-O object written on linux/<arch> is byte for byte the one macOS writes  (x4)

scripts/check-inert.sh against a build/mc1 built from origin/main:

ok   33 objects identical (tests/*.mc and src/mc.mc)
ok   taught examples/api -> build/api
ok   taught examples/lang -> build/lang-demo
ok   taught examples/conc -> build/conc-demo
ok   taught examples/desktop -> build/desktop-ui
ok   taught examples/kernel -> build/kernel.bin
check-inert: everything identical

A sandbox fix emits no different byte; the goldens move for the bundle and the two new functions.

The five goldens

Rewritten once each, only after its own criterion (mc2.sha256 after the empty --dump-asm diff
and cmp build/mc2.o build/mc3.o; the Linux pair deleted and re-recorded by
make check-linux-host; the Windows pair cross-computed per tests/golden/README.md and
confirmed byte for byte by build/mc2):

6e8eccf15dd071925d13eb146a05e888cff6f25572b60a0ca7a7f191a8c641fe  build/mc2.o
3789cc6d5cb4f4109b323de831d85e6a815142a7e46ff507a42ad2cbdcba3f80  build/mc2l.o     (linux-arm64)
ace3d74575d71b4151ec0d10e6fa3caf28cb56c993b107c49820993b86fcb557  build/mc2l.o     (linux-x86_64)
babc44ca1dec8dcaf2dc0f6507a3abe9fce48a283b8887684964208740fd5701  build/mc2w.obj   (windows-arm64, 1289613 B)
f6594f109a2eb58b48073042aeaa7127125f74735c137185b336e77e1a0103de  build/mc2w.obj   (windows-x86_64, 1325521 B)

Docs

• docs/specs/M43.md § Implementation notes — the forkbomb flake (new, four numbered points with
  the measurements above)
• docs/reference/sandbox.md — the "A refused call never returns" paragraph gains its other half,
  the order
• CLAUDE.md § State — one entry

🤖 Generated with Claude Code

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[Copilot]

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

[schivei]

Merging: CI 14/14 green on 96532b9 (both sandbox cells print ok forkbomb (alt)), mergeable clean, patch (no release label). Copilot could not review this PR -- three requests (af00391, 96532b9 twice) all answered Copilot encountered an error, most likely the regenerated 1 MB src/bundle_data.mc in the diff -- so the review of the 25 source lines is the architect's: sb_wait_gone polls the listener fd for POLLHUP with no events requested (pollfd offsets 0/4/6, ppoll with a null sigmask), bounded at 2 s, after sb_kill_pid + sb_kill_box; the measured mechanism (ENOSYS from the listener close racing the kill) and the 0/200 in both cells are in the PR body.