gates(aarch64): the native boot test declares its boot_tests-kernel requirement; CLAUDE.md names the production gate - #815
Merged
Conversation
…ile kernel fast docker/qemu/run-aarch64-boot-test-native.sh unconditionally required INIT_GROUP_REFUSAL_ORACLE_LITERAL and the INIT_GROUP_WALK marker, both emitted only by #[cfg(feature = "boot_tests")] code (kernel/src/tracing/providers/teardown.rs, gated at the module level by kernel/src/test_framework/mod.rs:60-66). A production kernel built without that feature can not print either marker, so each of its 5 MAX_RETRIES boots ran into a generic "marker missing" failure with nothing saying the kernel's build profile was the cause (deduced from the cfg gate, not separately measured against the pre-fix script -- see the doc's "What was red" section). claim-lint:ok: structural claim about #[cfg(feature = "boot_tests")] resolved by direct citation at kernel/src/tracing/providers/teardown.rs:5125-5126 and :1422-1423, and kernel/src/test_framework/mod.rs:60-66; a build without that feature does not compile the cited code, so it cannot emit the marker. Added require_boot_tests_kernel(), the same shape as the strict gate's existing guard (a census of 7 boot_tests-only marker prefixes, one missing="$missing $marker" accumulation, one exit 1 in the missing-marker arm), called once before the retry loop. That arm prints the script's own ARM64 BOOT TEST: FAILED banner ahead of its exit, so a downstream harness watching for that banner still sees a labeled verdict on this failure mode instead of an unformatted early exit. CLAUDE.md's ARM64 test-script list and its Standard Workflow build command now say the native/strict gates need a --features boot_tests kernel and add the previously-unlisted production-profile gate, which builds its own no-features kernel and needs no change here. tests/strand_handoff_structure.rs::boot_tests_gates_refuse_a_wrong_profile_kernel now censuses the native gate too (a NATIVE_GATE_PATH entry alongside the existing service-sequence/strict/full-test rows), so a future aarch64 gate script shipping without this guard shape reddens that test. Confirmed by deleting the guard invocation in a scratch copy of the script (test failed: "native gate must invoke the boot-tests profile guard exactly once at top level", left: 0, right: 1) and restoring it (test passed). Proofs: a no-features production kernel run against the native gate now fails in 2.08s with the "was not built with --features boot_tests" verdict and zero QEMU processes launched, instead of 5 retries; a --features boot_tests kernel passes the native gate on attempt 1/5; the production-profile gate (run-aarch64-prod-profile-boot-test.sh) passes; all 30 tests/*_structure.rs suites are green, 562 cases, 0 failed. Detail, including what is not claimed, is in docs/planning/green-program/gates/NATIVE-GATE-GUARD-2026-09-05.md. claim-lint: scripts/claim-lint.py -> exit 0 claim-lint: scripts/claim-lint.py --files docs/planning/green-program/gates/NATIVE-GATE-GUARD-2026-09-05.md -> exit 0 Co-Authored-By: Ryan Breen Co-Authored-By: Claude Code
…claim, and three minors 6 of 6 review findings on 329ab9c (the native-gate wrong-profile-kernel guard) closed in this round. N1 (major): the script's own "No ARM64 kernel found" arm -- the first preflight check, above require_boot_tests_kernel() -- still told the operator to build a kernel WITHOUT --features boot_tests, the exact kernel the guard 60 lines below then refuses. 3 of 3 peer aarch64 gates already carried the feature in that arm; this script was the exception. Added --features boot_tests to the hint. The pre-existing ratchet (strict_gate_build_hint_enables_boot_tests) only ever checked the strict gate, so it could not have caught this; generalized it to boot_tests_gate_build_hints_enable_boot_tests, looping over both gates. Its first draft (.any(hint contains the feature)) was itself vacuous against this exact bug -- the native gate's second build-hint line, inside require_boot_tests_kernel()'s own missing-marker arm, already carried the feature, so "at least one hint has it" was already true with the top-of-script hint broken. Strengthened to .all(...). Mutation-confirmed both ways: reverting only the top-of-script hint reddens the test ("native gate every build hint must enable --features boot_tests"); restoring it returns the suite to green. N2 (major): NATIVE-GATE-GUARD-2026-09-05.md and 329ab9c's own commit body claimed "a future aarch64 gate script cannot ship without this guard passing the same census" / "reddens that test". False -- boot_tests_gates_refuse_a_wrong_profile_kernel iterates a literal four-path list, not a directory scan; a new fifth gate script is invisible to it. docker/qemu/run-aarch64-arma609-arm.sh already demonstrates the gap (its own require_boot_tests_kernel(), 5-marker census, 0 references in tests/*.rs). Narrowed the doc's claim to what the census actually covers; the commit body is left as pushed per house style (pushed history stays intact) -- this commit's own body states the correction instead. N3 (minor): the script comment ahead of require_boot_tests_kernel() claimed the ratchet proves the banner echoes print "by construction." The ratchet only proves the arm's exit-1 count; deleting the three banner echoes alone (mutation-confirmed, then reverted) leaves it green. Reworded to state only what the ratchet checks. N4 (minor): only the new missing-marker arm routed through the PASSED/FAILED banner; the script's other two preflight rejections (no kernel, no ext2 disk) still exited bare. Routed both through the same banner convention, live-proven: ran the gate with no kernel present (prints the new build hint + "FAILED (no ARM64 kernel found)"), then with a --features boot_tests kernel built but no ext2 image (prints "FAILED (ext2 disk not found)"), then end to end with both present (PASSED, attempt 1/5, zero orphaned qemu-system-aarch64 processes before or after). N5 (minor): docs/planning/ARM64_TEST_CATALOG.md's "Running Tests" recipe still built --features testing before running this gate, which now hard-fails on the guard (testing and boot_tests are independent Cargo features). Corrected to --features boot_tests, with a note that the full-test-suite line builds its own --features testing kernel internally and is unaffected. N6 (nit): the doc's "is what run-x86-prod-profile-boot-test.sh's report_gate_failure machinery enforces" sentence could read as this script satisfying that ratchet. It would not -- validate_x86_prod_profile_harness rejects any exit other than exit "$exit_code", and this script's 3 of 3 preflight arms exit via bare exit 1. Reworded to state the shared principle and disclose the gap explicitly. Verification: cargo test across strand_handoff_structure (38/38), exec_lock_order_structure (44/44), and teardown_structure (83/83); the full 30-binary tests/*_structure.rs family, 562/562, 0 failed (same total the prior commit's own proof cited). Live boots on this Mac (aarch64 native): missing-kernel arm, missing-ext2 arm, and a full --features boot_tests pass (attempt 1/5, PASSED), each confirmed by running the script directly; no qemu-system-aarch64 process was ever left running across any of them. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen Co-Authored-By: Claude Code
ryanbreen
added a commit
that referenced
this pull request
Sep 5, 2026
Pulls in PR #815 (aarch64 native-boot-test gate guard, review r157) from main; no file overlap with this branch's 7 gate scripts, doc, serials, or tests/teardown_structure.rs -- main's own changes this round touched docker/qemu/run-aarch64-boot-test-native.sh and tests/strand_handoff_structure.rs instead, so this merge carries no conflicts.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The native ARM64 boot-test gate now refuses a wrong-profile kernel fast (health-811)
Base:
mainat9b3dd4af9dc53d2950688f8094a26351703892c.What was red
docker/qemu/run-aarch64-boot-test-native.shscores a boot only throughrun_single_test's final checks, and one of those checks is unconditional:INIT_GROUP_REFUSAL_ORACLE_LITERALis the serial lineinit_group_refusal_oracle_test()prints(
kernel/src/tracing/providers/teardown.rs:5125), and that function is#[cfg(feature = "boot_tests")]. The gate also requires the[INIT_GROUP_WALK:...]marker, emitted byemit_init_group_walk()(
kernel/src/tracing/providers/teardown.rs:1422), which carries the same#[cfg(feature = "boot_tests")]gate. A kernel built without--features boot_tests— the shipped production profile — cannot printeither line, because
kernel/src/test_framework/mod.rs:60-66gates the wholeregistry/executormodule tree, where these functions are registered andinvoked, on that same feature. Since the marker can not be present in such a
kernel's serial output under any boot outcome, a boot of it deterministically
fails
run_single_teston"init-group refusal oracle counter marker missing"(or, earlier in the same run,"exec first commit not observed"if the boot is slow enough that the 24s liveness window elapses first) — this
branch did not run the pre-fix script against a production kernel to count
the failed attempts directly, since the fix lands before the retry loop is
reached, but the deduction above is what the pre-fix script's own
MAX_RETRIES=5retry loop and its genericARM64 BOOT TEST: FAILED (after 5 attempts)banner would have produced.Nothing in that banner says the kernel was built in the wrong profile; it
reads exactly like a boot regression.
docker/qemu/run-aarch64-boot-test-strict.shhad already been fixed forthis class of failure (its own
require_boot_tests_kernel(), added ahead ofthis change) but the native script had not.
The fix
(a)
require_boot_tests_kernel()guardAdded the same function name and body shape as the strict gate's guard —
same for-loop census of boot_tests-only marker prefixes, same
missing="$missing $marker"accumulation, sameif [ -n "$missing" ]; then ... exit 1; fistructure — inserted right afterthe existing
check-kernel-no-neon.shpreflight and before the ext2-diskcheck, called once at top level as
require_boot_tests_kernel "$KERNEL".The one deliberate difference from the strict gate's copy: this gate's own
The same principle — route a preflight rejection through theverdict convention is the
ARM64 BOOT TEST: PASSED/ARM64 BOOT TEST: FAILEDbanner printed at the bottom of the script on the retry-loop path.Review R157 correction: the first version of this change routed only
the new missing-marker arm through that banner, leaving this script's two
pre-existing preflight rejections (no kernel found, no ext2 disk found)
exiting bare — the same defect this whole change exists to remove, on a
different arm. Fixed in the same commit as this correction: 3 of 3
preflight arms now print the banner (each with its own parenthetical)
immediately before their
exit 1, so every preflight rejection in thisscript flows through the verdict shape this script already reports through
on its ordinary retry-exhausted path, not just the marker-census one.
script's one verdict-reporting mechanism rather than a bare
exit— iswhat
docker/qemu/run-x86-prod-profile-boot-test.sh'sreport_gate_failuremachinery enforces for that gate, in that mechanism's own idiom: a single
report_gate_failure ...; exit "$exit_code"call site, an ERR trap, and areachedflag, all checked bytests/teardown_structure.rs::validate_x86_prod_profile_harness(
docs/planning/green-program/gates/GATE-PREFLIGHT-VERDICT-802-2026-09-05.md).This native script implements the analogous idea independently, with its
own bespoke banner-before-
exit 1convention, no ERR trap, and noreachedflag, and would NOT itself satisfy that x86-specific structural test — it
rejects any
exitstatement other thanexit "$exit_code", and everypreflight arm in this script exits via a bare
exit 1. A downstreamharness grepping for the
ARM64 BOOT TEST: FAILEDbanner text gets alabeled verdict on all three preflight rejections, not an unlabeled early
exit that reads as a script crash.
The census reuses the strict gate's exact seven marker prefixes
(
[SCHED_STRAND_ORACLE:,[STRAND_INJECT_ORACLE:,[CENSUS_WIDEN_ORACLE:,[FUTEX_HANDOFF_ORACLE:,[CTX596_ORACLE:,[TOMBSTONE_JOIN_ORACLE:,[BOOT_TESTS:) rather than a native-specific pair.Six of the seven — every ORACLE marker in the list — are general
boot_tests-only profile markers this gate's own
run_single_testdoes nototherwise score;
[BOOT_TESTS:is the exception, sincerun_single_testalready checks for a
[BOOT_TESTS:FAILline (a check that is a harmlessno-op against a kernel built without the feature, since such a kernel never
emits any
[BOOT_TESTS:...]line at all). The six-marker redundancy isreused here so the guard stays a robust profile detector — a single marker
regressing to a different profile cannot quietly disarm it — instead of a
narrower check that a smaller future refactor could accidentally shrink
toward vacuousness. The gate's actual pinned boot_tests-only markers,
INIT_GROUP_REFUSAL_ORACLE_LITERALandINIT_GROUP_WALK, are named in theguard's failure message so the operator sees the concrete reason this gate
specifically needs the feature, not just an abstract marker list.
(b) CLAUDE.md
Two minimal wording changes in the "Test Scripts" / "Standard Workflow"
sections, no other edits:
need a
--features boot_testskernel, and adds the previously-unlistedrun-aarch64-prod-profile-boot-test.shas the production-profile script(it builds its own no-features kernel internally, so it needs no build
line of its own).
cargo build ... kernel-aarch64command that precedes./docker/qemu/run-aarch64-boot-test-native.shin "Standard Workflow" nowincludes
--features boot_tests, with a one-line comment saying whichscript needs the feature and which one (the prod-profile gate) does not.
Before this change, CLAUDE.md's own documented workflow built a
no-features kernel and then ran the gate that requires boot_tests markers —
the exact mismatch this whole change is about, reproduced by a reader
following CLAUDE.md verbatim.
(c) Anti-vacuity: extend the existing structure-test census
This censustests/strand_handoff_structure.rs::boot_tests_gates_refuse_a_wrong_profile_kernelalready censused the service-sequence, strict, and full-test gates for this
guard shape (function body non-empty,
for marker in '...' ...; dolinewith at least
MIN_BOOT_TESTS_PROFILE_MARKERS(6) bracketed markers,grep -aqFinspection, singlemissing="$missing $marker"accumulation,single
exit 1in the missing-marker arm, single top-level invocation,no-NEON preflight ordered first). Added a
NATIVE_GATE_PATHconstant andone entry,
("native gate", NATIVE_GATE_PATH), to that test's gate list.covers the four gate scripts the test already names, not a directory-wide
discovery: those four scripts cannot silently lose this guard shape, but a
new fifth script is invisible to this list until someone adds it.
(d) Review R157: the top-of-script build hint, and one honesty fix
N1 (major). The script's own "No ARM64 kernel found" arm — the very
first preflight check, at the top of the file, before
require_boot_tests_kernel()is even defined — still told the operator tobuild a kernel WITHOUT
--features boot_tests: the exact kernel the guard60 lines below then refuses. 3 of 3 peer boot_tests-requiring aarch64
gates (service-sequence, strict, full-test) already carried the feature in
that same arm; this script was the one exception. Fixed by adding
--features boot_teststo that hint, with a one-line comment pointing atrequire_boot_tests_kernel()as the reason.The pre-existing ratchet,
strict_gate_build_hint_enables_boot_tests, only ever checked the strictgate's build hint, so it could not have caught this. Renamed to
boot_tests_gate_build_hints_enable_boot_testsand generalized to loop overboth the strict and native gates. The first pass at the generalized
assertion (
.any(build_hints, contains "--features boot_tests")) wasitself vacuous against exactly this bug: the native gate has a second
cargo buildecho line, insiderequire_boot_tests_kernel()'s ownmissing-marker arm, which already carried
--features boot_testsbeforethis round — so "at least one hint has the feature" was already true even
with the top-of-script hint broken. Strengthened to
.all(...), whichrequires every build-hint line in the gate to carry the feature.
Mutation-proven both ways in this round: reverting only the top-of-script
hint to drop
--features boot_tests(leaving the guard's own hintuntouched) reddens
boot_tests_gate_build_hints_enable_boot_testswith"native gate every build hint must enable --features boot_tests"; restoring
it returns the suite to green (38/38 in
tests/strand_handoff_structure.rs, plustests/exec_lock_order_structure.rs44/44 and
tests/teardown_structure.rs83/83, all of which also read thisscript's text and were otherwise unaffected).
N3 (minor). The in-script comment ahead of
require_boot_tests_kernel()had claimed the ratchet proved the banner echoes print "by construction,"
citing the same
exit 1-count check the ratchet actually performs. Theratchet proves the arm has exactly one
exit 1line; it does not inspectthe echo lines above it, so deleting the three banner echoes alone (proven
by mutation, then reverted) leaves the ratchet green. Reworded the comment
to state only what the ratchet checks, and to say plainly that the banner's
presence is a maintained convention, not a ratcheted one.
Mutation proof the added census line is load-bearing
With the fix applied and
require_boot_tests_kernel "$KERNEL"temporarilydeleted from a scratch copy of the native script (not committed — restored
before this branch's tests were re-run),
cargo test --test strand_handoff_structure boot_tests_gates_refuse_a_wrong_profile_kernelfailed:
Restoring the file and re-running the same test returned
test result: ok. 1 passed.Proofs
Fail-fast on the production (no-features) kernel
Built with
cargo build --release --target aarch64-breenix-kernel.json -Z build-std=core,alloc -Z build-std-features=compiler-builtins-mem -p kernel --bin kernel-aarch64(no--features), verified withscripts/check-kernel-no-neon.sh(PASS: 0 FP/SIMD load/store instructions), then run once againstdocker/qemu/run-aarch64-boot-test-native.sh:Exit status 1, wall time 2.08s (
timeon the run).pgrep -fl qemu-system-aarch64 | wc -lreported 0 immediately before this run and 0immediately after it, so this run itself launched no QEMU process; this
branch did not run the pre-fix script against this same production kernel to
directly count how many
qemu-system-aarch64processes its 5-attempt retryloop would have launched (see "What was red" above for the deduction that it
would be 5, one per
MAX_RETRIESattempt, each on a 30stimeout).Pass on the boot_tests kernel
Rebuilt with
--features boot_testsadded (userspace ELFs built first viauserspace/programs/build.sh --arch aarch64, since the boot_tests registryembeds
simple_exit.elfat compile time), re-verified with the sameno-NEON guard, ext2 image created with
scripts/create_ext2_disk.sh --arch aarch64, then run once:Exit status 0, passed on the first attempt (no retry needed).
Pass on the production-profile gate
docker/qemu/run-aarch64-prod-profile-boot-test.sh, which builds its ownno-features kernel and boots it, run once:
Exit status 0.
Structure-test suites
Every file in
tests/*_structure.rs(30 files at this head) runindividually with
cargo test --test <file>: 30 of 30 green, 562 testcases total (
aarch64_testing_profile_structure2,block_request_lifetime_structure12,context_restore_structure97,coreproof_component_h_structure5,coreproof_coverage_structure4,coreproof_mutation_register_structure5,coreproof_sites_structure4,degenerate_transfer_fd_validation_structure4,dispatch_path_lock_free_structure4,dispatch_strand_census_structure7,dma_and_log_sink_structure4,entry_point_df_structure5,exec_lock_order_structure44,exit_tally_structure6,ext2_lock_structure36,fork_lock_order_structure10,green_program_envelope_structure14,loopback_pump_structure72,masked_binary_load_structure4,mmap_floor_structure9,net_lock_structure19,poll_tcp_gate_wiring_structure3,preempt_bracket_structure8,serial_line_atomicity_structure9,signal_eintr_predicate_structure2,strand_handoff_structure38,syscall_return_register_structure6,teardown_structure83,ttbr0_shadow_reconciliation_structure32,tty_oracle_structure14),0 failed.
Claim-lint
plus a
--files/--commit-msgrun recorded in the round notes for thisbranch's own diff and commit message.
What is NOT claimed
kernel/was nottouched; no kernel behavior changed.
boot_tests-profile detectors, not a claim that
run-aarch64-boot-test-native.shitself scores
SCHED_STRAND_ORACLE,FUTEX_HANDOFF_ORACLE,CENSUS_WIDEN_ORACLE,CTX596_ORACLE, orTOMBSTONE_JOIN_ORACLE— itdoes not. The two markers this gate does score and require are
INIT_GROUP_REFUSAL_ORACLE_LITERALandINIT_GROUP_WALK, namedseparately in the guard's own failure message.
run, one pass on
boot_tests, one pass on the production-profile gate),not a multi-boot soak. The native gate's own
MAX_RETRIES=5retrymechanism and the strict gate's iteration count remain the tools for
measuring flake rate; this change does not add or remove either.
docker/qemu/run-aarch64-boot-test-strict.sh,run-aarch64-service-sequence-gate.sh,run-aarch64-full-test.sh, or anyx86 gate script. Their preflight/verdict shapes are unchanged.
run-x86-prod-profile-boot-test.shverdict-discipline ratchet (
x86_production_profile_gate_verdict_discipline_holds)were read for the pattern this fix follows but were neither run nor
modified as part of this change.
other section of CLAUDE.md was reviewed or edited as part of this task.