fcntl: EAGAIN under process-manager contention is a census, not a verdict - #796 - #819
Merged
Conversation
STEP 1 of #796. No kernel code changes in this commit: the census table lands first so the repair that follows can be checked against it. 18 try-lock sites are in scope (13 `try_manager()` + 5 `try_lock()`) across kernel/src/syscall/ and the two architecture syscall entry points. 6 of the 18 turn a lock try-failure into an errno returned to userspace; 12 of the 18 degrade silently and are listed too, so the split is checkable against the three greps recorded in the doc. Of the 6, three are not POSIX-legal: - handlers.rs:3875 sys_fcntl -> EAGAIN (the #796 signature: the preamble runs before the cmd dispatch, so it covers 6 of the 6 implemented commands, and POSIX only permits EAGAIN in fcntl's F_SETLK record-locking arm, which this kernel does not implement) - signal.rs:1368 and signal.rs:2152 sys_sigsuspend -> ESRCH (POSIX lists EINTR only) The other three are graphics.rs's ErrorCode::Busy on Breenix-private framebuffer syscalls, which have no POSIX errno list and already implement a bounded spin; they are marked no-change. The lock-discipline section records the STOP-condition answer: 13 of 13 PM accesses in the interrupt and exception files are tabulated, 9 non-blocking and 4 blocking, and 0 of the 4 blocking acquisitions is in asynchronous IRQ context -- all four are synchronous traps from userspace, the same context class as a syscall. The 7 asynchronous IRQ-context accesses are all non-blocking try_manager() with a refuse-and-re-arm degrade path. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… contention STEP 2 of #796, applied to the 3 census rows whose errno is not POSIX-legal. The 3 graphics.rs rows are left alone: they are Breenix-private syscalls with no POSIX errno list and already spin before returning a retryable Busy. sys_fcntl (kernel/src/syscall/handlers.rs): the process-lookup preamble ran before the cmd dispatch, so a failed try_manager() returned EAGAIN for any of the 6 implemented commands. POSIX allows EAGAIN out of fcntl only in the F_SETLK record-locking arm, which this kernel does not implement, and the reported failure was fcntl(F_SETFD) returning it before set_fd_flags -- whose only error is EBADF -- was reached. sys_sigsuspend, both architecture bodies (kernel/src/syscall/signal.rs): the same try-lock returned ESRCH. POSIX gives sigsuspend one error, EINTR. All three take the blocking crate::process::manager() instead. Option (a) of the repair menu, argued per site in the commit comment at each acquisition and in the doc: - these bodies are reachable only from the syscall dispatcher, i.e. on a trap taken from EL0/ring 3, which is the context class in which aarch64/exception.rs:2331 already takes the blocking manager() for a CoW abort, with the same justification -- a CPU that was in userspace when it trapped does not already own PROCESS_MANAGER; - 0 of the 4 blocking PM acquisitions in the interrupt and exception files is in asynchronous IRQ context, and the 7 that are use try_manager() with a refuse-and-re-arm degrade path, so waiting here cannot wedge an ISR; - a holder is not preempted away from the waiter (aarch64 masks DAIF for the guard's lifetime; x86's dispatch path refuses to switch while PM is held), so the wait is bounded by the holder's own window; - sys_dup and sys_dup2, directly above sys_fcntl and mutating the same process.fd_table from the same context, have taken the blocking manager() all along -- fcntl was the outlier; - each window drops its guard before any scheduler call, so the PM->SCHEDULER order marker (SCHED_AFTER_PM_VIOLATIONS) is untouched. Neither sigsuspend guard scope changes: both still end before with_scheduler(block_current_for_signal_with_context). claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ger lock STEP 3 of #796. A boot_tests-profile oracle that creates the contention that produced the reported EAGAIN, instead of simulating it: a kthread pinned to a peer CPU takes PROCESS_MANAGER with try_manager() and holds it for a 3 ms window read off CNTVCT_EL0, while the driving test thread issues 64 fcntl(F_SETFD) calls through the production sys_fcntl and counts EAGAIN. Two readings keep a green verdict out of reach without contention: - pm_busy_probe: an independent try-lock read taken immediately before the measured call, so "contended" is observed rather than assumed from the peer's timing; - first_wait_us: how long the first call waited, floored at 1000 us of the 3000 us window, so a call that sailed through an uncontended lock scores FAIL rather than PASS. first_errno=9 (EBADF) is expected and is disclosed in the marker: the driving thread is a kthread with no process row, so a repaired kernel reaches the process lookup and fails there. The oracle measures which arm the syscall takes when the lock is busy; tty_oracle's cloexec_exec arm stays the witness that F_SETFD succeeds under production load. x86 boots -smp 1, where two threads do not contend for this lock at all -- the dispatch path refuses to switch while it is held, so the holder is not preempted and no second thread runs. That arm emits [FCNTL_PM_CONTENTION_ORACLE:x86:...:SKIP] with the reason named, returns false, and is emitted once from the marker-only stage path, mirroring the census widening oracle's x86 SKIP exactly. The holder deliberately uses try_manager() rather than manager(): try_manager() does not touch DAIF, so the holding CPU keeps taking timer interrupts and the CPU-0-only global tick counter is not stopped by this oracle. Its release deadline is read from CNTVCT_EL0, which advances while any CPU is masked, so the window closes even while the driver waits for the lock with interrupts off. Registered last in SYSCALL_TESTS, which runs after PROCESS_TESTS, and followed by a 20 ms settle, so the window in which no CPU can commit a dispatch does not ride into the scheduler censuses in the process subsystem. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
aarch64 strict gate (the kernel-merge gate):
- FCNTL_PM_CONTENTION_ORACLE_PATTERN pins the armed verdict field by field:
armed=1, pm_busy_probe=1, calls=64, eagain=0, first_errno=9, joined=1, PASS.
attempts is allowed 1-3 because the oracle re-arms if the peer's window
closed before the measured call; the other fields are exact.
- the marker is added to require_boot_tests_kernel's literal census, so a
kernel built in the wrong feature profile is still refused by name rather
than failing every boot on "marker missing".
- a separate FAIL scan runs before the pattern check, in the same
presence-then-absence-of-FAIL shape the neighbouring oracles use, so a
failing verdict is reported by its own line instead of as a missing marker.
x86 boot-tests gate:
- FCNTL_PM_CONTENTION_ORACLE_LITERAL pins the SKIP line, including its stated
reason and online_cpus=1. Pinning it keeps the emitter alive on the arch
that cannot arm the oracle: without this, deleting the oracle outright
would leave this gate green.
No roster deltas. The oracle adds no userspace process and no process row, so
EXPECTED_USERSPACE_EXITS and the row-count arithmetic in either gate are
unchanged; its one extra kthread is joined and retired inside the test.
claim-lint: scripts/claim-lint.py -> exit 0
Co-Authored-By: Ryan Breen <ryan.breen@gmail.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r fixes Every one of these was found by running the oracle, not by reading it, and each is recorded here because the first version of this oracle would have shipped a flaky gate. 1. It reddened census_widen_oracle. Subsystem test threads run concurrently, so registering this oracle in SYSCALL_TESTS put its window -- the interval in which no CPU can commit a dispatch, because it holds the lock every dispatch needs -- alongside census_widen_oracle's baseline, which requires no thread queued on a non-dispatching CPU. 2 of 2 branch smoke boots reddened it while plain origin/main scored 3 of 3 green, so the interference was this oracle's, not a pre-existing condition. Fixed by registering in PROCESS_TESTS after census_widen_oracle: tests inside one subsystem run in sequence, so that baseline is now taken strictly before this window exists. 2. joined=0 on 3 of 3 boots. The peer was chosen as "the first CPU that is not me", which picked a CPU this boot had stopped dispatching on. The scheduler's unschedulable-queue reclaim then migrated the probe off the CPU it was placed on -- holder_cpu came back as a CPU that was never the target -- and the boot-test affinity retain check bounced it from the CPU that had it, so it armed but never reached its exit. Fixed with live_peer_cpu_for_test(), the complement of the existing stale_peer_cpu_for_test(), plus a release_cpu_affine_thread_for_test() before the exit wait, which is what the census widening oracle already does with its own probe. 3. armed=0 on 1 of 3 boots. The window was 3 ms and the timer tick is 5 ms, and the driver waited for the arm with arch_halt(), so it could sleep through the entire hold: the peer acquired, held and released between two of the driver's wakeups. Fixed by spinning instead of halting in the arm wait -- the peer runs on its own CPU, so spinning does not delay it -- and by widening the window to 8 ms, comfortably more than one tick. hold_done joins the marker so a future failure of this shape is legible: it separates "the holder never finished its window" from "the holder finished but its exit was not observed". The strict gate pins hold_done=1. Result at these bytes: aarch64 strict 6/6, with [FCNTL_PM_CONTENTION_ORACLE:aarch64:attempts=1:armed=1:holder_cpu=1:pm_busy_probe=1:calls=64:eagain=0:first_errno=9:first_wait_us=8116:hold_done=1:joined=1:PASS] claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Two more defects the oracle's own boots found, and the reason both are the same defect: this oracle drives a call that now blocks for a lock with no timeout, by construction, so anything that can stop the holder from reaching its release hangs the boot instead of failing the test. 1. 1 of 6 strict boots hung with [TEST:syscall:fcntl_pm_contention_oracle:START] and no verdict: [STAGE:proc:ADVANCE] never completed, the syscall and process subsystem threads both stopped, and the boot ran on emitting censuses until the gate killed it. The holder took the lock with try_manager(), which leaves interrupts enabled, so it could be taken off its CPU between the acquire and the release. Fixed by holding inside arch_impl::aarch64::cpu::without_interrupts -- the same mask-then-hold shape crate::process::manager() itself uses on this architecture -- so nothing can take the CPU away for the length of the window. The acquire stays a bounded try-loop rather than a blocking manager(): an unbounded masked wait for a lock this oracle does not own would put the same hang on the other side. Its bound drops from 200 ms to 20 ms because that spin is now masked. 2. The driver could migrate onto the peer between choosing it and issuing the contended call, and would then be waiting -- with interrupts masked inside the blocking acquisition -- for a holder that cannot be dispatched on the CPU the driver is occupying. The measured section now runs inside a preempt_disable/preempt_enable bracket. CPU 0 became the second choice rather than the first, via live_peer_cpu_for_test_excluding_cpu0(): it is the only writer of the global tick counter on this architecture, and the holder now masks interrupts. Also moved the registration from PROCESS_TESTS to SYSCALL_TESTS at TestStage::ProcessContext. PROCESS_TESTS was wrong twice over: placing it after census_widen_oracle broke the strand_handoff_structure ratchet that pins that oracle as the last registration in its array, and placing it before would put a kthread-stack allocation inside the accounting window that ratchet exists to protect. Stages are barriers -- advance_to_stage joins every subsystem thread before returning -- so ProcessContext puts this window strictly after every PostScheduler verdict, census_widen_oracle's baseline included, without touching that array at all. Result at these bytes: aarch64 strict 10/10, with [FCNTL_PM_CONTENTION_ORACLE:aarch64:attempts=1:armed=1:holder_cpu=2:pm_busy_probe=1:calls=64:eagain=0:first_errno=9:first_wait_us=7980:hold_done=1:joined=1:PASS] and tests/strand_handoff_structure.rs back to 38 passed. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rofiles The oracle holds the process-manager lock on a peer CPU on purpose. It is boot_tests-only by construction -- kernel/src/lib.rs compiles test_framework only under boot_tests or btrt -- but "by construction" is exactly the kind of claim these two gates exist to turn into a reading. aarch64 production-profile gate: FCNTL_PM_ORACLE_LITERAL joins the three boot_tests-only markers already asserted absent, with its count printed on both the success and the diagnostic paths. x86 production-profile gate: '[FCNTL_PM_CONTENTION_ORACLE:' joins TEST_ONLY_MARKERS, next to '[CENSUS_WIDEN_ORACLE:' whose x86 SKIP line this oracle's x86 arm mirrors. Measured with these gate bytes: aarch64 production profile 3/3 PASS, each reporting "Observed fcntl contention oracle marker count: 0" alongside "Observed TTY oracle failure count: 0" -- the #796 signature's own production witness, clean. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t claimed Completes docs/planning/green-program/syscalls/796-FCNTL-EAGAIN-2026-09-05.md, whose census section landed first in 79b793d. STEP 2 records the per-site argument for taking the blocking acquisition, including the four points the census section supports (trap-from-userspace context, 0 of 4 blocking PM acquisitions in asynchronous IRQ context, bounded holds, and sys_dup/sys_dup2 as the in-file precedent), and why option (b) was not used: there is no correct errno to return at the end of a bound, because POSIX has no "the kernel was busy" answer for F_SETFD. STEP 3 records the oracle's shape, its two anti-vacuity readings, the gate wiring on both rosters with the roster deltas stated (0 fields changed), and both halves of the red-to-green record with the verdict lines quoted: main + the oracle, 3 boots: eagain=64:first_errno=11, 3 of 3 FAIL branch, 3 boots: eagain=0:first_errno=9:first_wait_us~8000, 3 of 3 PASS Plain origin/main with no oracle scored 3 of 3 green on the same gate, which is what makes those failures attributable to the measured property rather than to the branch's presence. STEP 4 records the three build profiles with the toolchain notice disclosed rather than filtered, 30 of 30 structure suites (544 cases) with the one ratchet that went red mid-round and why, the aarch64 strict and production boots with kernel sha256s, and the x86 leg. The closing section states six things this branch does NOT claim, including that the production-profile EAGAIN rate is not measured, that the oracle does not witness a successful F_SETFD, that the x86 arm is not evidence about contention, and that sigsuspend's repair is not covered by any oracle here. claim-lint: scripts/claim-lint.py -> exit 0 claim-lint: scripts/claim-lint.py --files docs/planning/green-program/syscalls/796-FCNTL-EAGAIN-2026-09-05.md -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The x86 build and both x86 gates were re-run at de83845 -- the head that adds the production-profile absence assertion -- rather than at the earlier head the first x86 pass used, so the recorded numbers and the shipped gate bytes are the same bytes. Build grep empty, RC 0, booted binary sha256 03b4fc45978df3e8fcdbd3e7683fb6e399672defe221478d9fb62340a966e7e8. run-x86-boot-tests.sh 1 -> exit 0 with the SKIP literal present in the serial. run-x86-prod-profile-boot-test.sh -> exit 0, reporting test-only marker '[FCNTL_PM_CONTENTION_ORACLE:': 0 claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…loor with one holder Six review findings, three of which say this branch's own prose was wrong. F1. "No asynchronous interrupt handler on either architecture blocks on this lock" is false and is deleted from sys_fcntl's comment, from both sigsuspend comments and from the census. net/udp.rs:146 deliver_to_socket takes the blocking with_process_manager() and runs on the NetRx softirq, which do_softirq() dispatches at IRQ exit on both arches (aarch64/exception.rs:2212, per_cpu.rs:723). The conclusion survives for a narrower, per-CPU reason the branch had not stated: aarch64 manager() masks DAIF before the mutex and unmasks in Drop, and x86 runs the syscall preempt-disabled while irq_exit() dispatches softirqs only at preempt_count()==0. New census point 3b writes that out. Residual filed as #812: the DAIF argument covers manager() holders only, and an aarch64 try_manager() holder in a preempt-disabled syscall sits in a window where that softirq can run on its own CPU and block on the lock it owns. F2. Census point 4's "2 of 2 dispatch entry paths refuse identically" is deleted. The x86 comment's "both entry paths" are two x86 paths; the aarch64 TtbrResult::PmLockBusy remedy redirects this CPU to idle and requeues the incoming thread, leaving no lock-holding context running. Point 4 now gives each arch its own mechanism and says the DAIF bound excludes try_manager(). F3. first_wait_us had one defence and now has three, each mutation-driven. The gate pattern floors it at four digits; a FCNTL_PM_WAIT_SELFCHECK block proves that pattern rejects first_wait_us=0 and accepts 8032 before any boot is scored (3 of 3 legs run: shipped exits 0, [0-9]+ exits 1 on the zero leg, [0-9]{9,} exits 1 on the 8032 leg); and tests/fcntl_pm_contention_gate_structure.rs holds the in-kernel conjunct and the self-check's presence by census. Three mutations redden that ratchet and were reverted. F4. The repair's cost is disclosed for the first time: on aarch64 both sites move to a DAIF-masked window covering both the wait and the critical section, including log::debug!/log::info! calls that reach COM2, and the wait's worst case is exec's ELF load. The withdrawn "does no I/O" claim is withdrawn in place. F5. interrupts.rs:965 is handle_stack_growth, the user-stack-growth arm of the page-fault handler, not signal delivery. F6. The census's grep set named a file that does not exist (aarch64/irq.rs) and then discussed one that was not in the set (gic.rs). The files are named individually and the timer/GIC result is a measured 0-of-3. Two by-catch repairs, both found by running the battery rather than reading it: * teardown_structure's x86_production_profile_gate_ratchet_is_not_vacuous was RED at the branch head. Its "array emptied" mutation was a hardcoded chain of replacen calls, and de83845 added a 20th TEST_ONLY_MARKERS element the chain does not name, so the mutation stopped applying. The previous round's "30 of 30 green" was therefore wrong. Fixed with a census-derived empty_shell_array(). * poll_tcp_gate_wiring_structure panicked on scripts/__pycache__/claim-lint.cpython-314.pyc -- an artifact this project's own mandated claim-lint step writes into a git-ignored directory. Its readers now skip files that are not UTF-8 text. #813 filed for four POSIX-illegal EAGAIN arms on blocking descriptors. Battery: aarch64 strict 3/3 and production PASS at the shipped kernels (45ca7fe9 boot_tests, 07c1b8c3 production, both no-neon PASS); 4/5 strict and production PASS at the interim kernels. Structure suites 31 of 31, 546 cases. The one red is attributed to #599 with its serial committed under docs/planning/green-program/syscalls/serials/796-r157/ and posted to that issue. UNATTRIBUTED: 0. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Beast, own clone at /root/breenix-p796 with BREENIX_GATE_TMP=/root/breenix-p796-tmp (R20/R18), branch fetched from GitHub at 6bf0b64. pgrep for qemu-system-x86_64 read 0 before the run. * cargo build --release --features testing,external_test_bins --bin qemu-uefi, piped through grep -E "^(warning|error)": 0 lines. * run-x86-boot-tests.sh 1: PASS, exit 0, "x86 userspace gate: PASS - exited=110 expected>=105 nonzero=0 allowlist=0". * run-x86-prod-profile-boot-test.sh: PASS, exit 0. The booted binary's sha256 is 03b4fc45978df3e8fcdbd3e7683fb6e399672defe221478d9fb62340a966e7e8, rebuilt at 2026-09-05 11:08:00 UTC. That is the same value the previous round recorded, which is what a comment-only kernel change predicts; it is recorded as an observation and supports no claim about the repair. Marker counts read from the serials rather than from the gate verdicts: 1 occurrence of FCNTL_PM_CONTENTION_ORACLE in the boot-tests serial_user.txt (the pinned x86 SKIP literal), 0 in both production-profile serials. claim-lint: scripts/claim-lint.py -> exit 0 Co-Authored-By: Ryan Breen <ryan.breen@gmail.com> Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Both branches independently patched the same latent panic in tests/poll_tcp_gate_wiring_structure.rs (a non-UTF8 scripts/__pycache__/*.pyc reaching scripts_asserting_oracle_fail's read closure): origin/main (#811 by-catch) inlined a read_to_string(...).map(...).unwrap_or(false) guard; this branch (6bf0b64) added a shared script_text() helper used at both read sites. Conflict resolution: scripts_asserting_oracle_fail keeps origin/main's already-landed inline form verbatim. That leaves script_text() with a single remaining caller (missing_ready_lost_wiring), so the helper is removed and its non-panicking read is inlined at that one call site instead, preserving the branch's R157 fix there with the smaller diff against origin/main. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…l oracle broke both_aarch64_gates_fail_on_an_untagged_publish replays 03-strict-boot1-serial.txt through run-aarch64-boot-test-strict.sh's scoring-only mode. This branch's score_serial now requires the FCNTL_PM_CONTENTION_ORACLE line, and that fixture predates the marker, so the suite scored FAIL. Re-recorded from a single boot_tests-kernel boot at merge commit ae9b2cd (BUILD_ID 006a9c04ac0a15); the strict scorer accepts it and both mutation legs (fcntl line deleted, ASID tag altered) fail as expected. 04-prod-boot1-serial.txt is untouched: the production scorer only asserts the marker's absence, which the pre-existing fixture already satisfies. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Brings in PR #818 (gates/verdict-discipline-widened): the six gate scripts run-x86-boot-tests.sh, run-aarch64-tty-oracle-gate.sh, run-coreproof-gate.sh, run-ext2-lock-race-gate.sh, run-fs-fault-gate.sh, and run-x86-tty-oracle-gate.sh now reject through their ERR-trap verdict path (echo + bare `false`) rather than a silent `exit`, and tests/teardown_structure.rs gained the matching verdict-trap shape tests. git auto-merged: 0 of 39 files touched across both sides landed with conflict markers, per `git status --short` after the merge. The two files this branch also touches -- docker/qemu/run-x86-boot-tests.sh and tests/teardown_structure.rs -- had disjoint hunks on each side: main's preflight-rejection and per-run FAIL-line idiom change landed alongside this branch's own FCNTL_PM_CONTENTION_ORACLE literal and gate-shape additions. Verified post-merge: all nine touched gate scripts pass `bash -n`, and run-x86-boot-tests.sh's preflight check still exits nonzero through the ERR-trap verdict line when given a relative BREENIX_GATE_TMP.
Adds a "Landing re-smoke" table for the merge with origin/main at aa7e0fd (PR #818's verdict-discipline widening): 31 of 31 tests/*_structure.rs suites green (567 cases), scripts/test_claim_lint.py exit 0, and PASS on 4 of 4 boot gates -- aarch64 strict and production, x86 boot-tests and production -- quoting the FCNTL_PM_CONTENTION_ORACLE lines each gate scores.
ryanbreen
added a commit
that referenced
this pull request
Sep 5, 2026
…ness branch before landing origin/main advanced (through #819, the fcntl EAGAIN census) while slice 3d was in review. Auto-merge with no conflicts in any path, kernel/ included.
This was referenced Sep 5, 2026
ryanbreen
added a commit
that referenced
this pull request
Sep 5, 2026
…, after #627 landed first origin/main advanced 19 more commits between this branch's first R16 merge and its PR being opened, including PR #837 (fix/627-futex-oracle-anchor), which independently re-recorded the same shared fixture (docs/planning/green-program/aarch64-testing/serials/slice3d/01-strict-boot1-serial.txt) to add an arm_delay_us field to FUTEX_HANDOFF_ORACLE_PATTERN. That capture carries the IRQ_HOLD_ORACLE line and arm_delay_us field but reverts FCNTL_PM_CONTENTION_ORACLE to the pre-#819 attempts=1:armed=1:... shape (it predates #819's merge to main), so this branch's second git merge --no-ff origin/main conflicted on the fixture again, this time three-way: 0 of 2 copies (this branch's, origin/main's) carries the 3 required lines -- IRQ_HOLD_ORACLE (#812), the FCNTL_PM_CONTENTION_ORACLE rendezvous shape (#819), and arm_delay_us (#627) -- together. Re-recorded from a strict-gate boot at the second merge's head (BUILD_ID 006a9c7cb301c7), carrying all three required lines together: the rendezvous-shape FCNTL_PM_CONTENTION_ORACLE PASS line, the IRQ_HOLD_ORACLE PASS line, and FUTEX_HANDOFF_ORACLE's arm_delay_us=16, plus 4 of 4 PINNED_HOME_CPU_UNAVAILABLE census lines reading count=0 and 14 of 14 TTBR0_ASID_CENSUS lines reading untagged=0. Both tests/loopback_pump_structure.rs::both_aarch64_gates_fail_on_a_pinned_placement_refusal and tests/ttbr0_shadow_reconciliation_structure.rs::both_aarch64_gates_fail_on_an_untagged_publish pass against the re-recorded fixture; all 32 of 32 tests/*_structure.rs suites pass (588 cases), including the new tests/qemu_host_lock_structure.rs PR #835 added in the same 19 commits. This capture used PR #835's new docker/qemu/lib/qemu-host-lock.sh mechanism rather than a manual pgrep check. README.md in the same directory records the full four-re-record provenance.
ryanbreen
added a commit
that referenced
this pull request
Sep 5, 2026
…ndezvous fcntl oracle: arming is a deadline rendezvous, not three attempts - the #819 strict-gate arming flake
This was referenced Sep 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tty_oracle'scloexec_execarm observed[TTY_ORACLE:FAIL:cloexec_exec:fcntl_setfd_failed:EAGAIN]on the production-profile aarch64 gate (1 boot in 10 on main
bdb5be90, #796).sys_fcntl's process-lookup preamble turned a failedtry_manager()try-lockinto
SyscallResult::Err(11)(EAGAIN) before theF_SETFDdispatch arm wasever reached --
FileDescriptorTable::set_fd_flagshas noEAGAINreturn pathof its own.
A census of the 18 try-lock sites under
kernel/src/syscall/and the twoarchitecture syscall entry points found 6 of 18 turn a failed try-lock into an
errno returned to userspace; 3 of those 6 (
sys_fcntl, and both architecturebodies of
sys_sigsuspend) take the blockingcrate::process::manager()instead, following the precedent already set by
sys_dup/sys_dup2in thesame file.
sys_fcntl(F_SETFD)'sEAGAINunder process-manager contention isa census artifact of an incomplete migration, not a POSIX-legal error this
syscall should ever surface -- hence the PR title.
A new
FCNTL_PM_CONTENTION_ORACLEforces the contention deterministically(one thread holds
PROCESS_MANAGER, a second callsfcntl) and is wired intoboth aarch64 boot gates: the strict gate's
score_serialnow requires theoracle's
PASSpattern, and the production gate asserts the marker's absence(it does not run under the injected-hold arm). The x86 side is a disclosed
SKIP-- these gates boot-smp 1, so there is no second thread to contendfor the lock -- pinned as a literal so deleting the oracle would not silently
leave the gate green.
The repair's cost is disclosed rather than left implicit: on aarch64 both
sites move from an interrupts-enabled
try_manager()to a DAIF-masked windowcovering the wait and the critical section (several
log::debug!/log::info!calls now execute inside it), bounded in the worst case byexec's ELF load under PM. This is the same costsys_dup/sys_dup2alreadypay on the same lock from the same context -- not a new exposure, but now a
named one.
Filed rather than fixed in this scope: #812 (a NetRx softirq bottom half can
still block on
PROCESS_MANAGERon aarch64, pre-existing and untouched here)and #813 (four other sites return
EAGAINon blocking file descriptors by adifferent, unimplemented-blocking mechanism).
Full census tables, the per-site repair argument, the oracle's construction,
and the R157 review-round repair are in
docs/planning/green-program/syscalls/796-FCNTL-EAGAIN-2026-09-05.md.#796.
Landing
Merge with
origin/mainorigin/mainhad moved to0b3b908e(PR #818,gates/verdict-discipline-widened):six gate scripts (
run-x86-boot-tests.sh, both tty-oracle gates,run-fs-fault-gate.sh,run-ext2-lock-race-gate.sh,run-coreproof-gate.sh) now fail through theirERR-trap verdict path (
echo+ barefalse) instead of a silentexit, andtests/teardown_structure.rsgained the matching verdict-trap shape tests.git merge --no-ff origin/mainresolved with 0 conflicted paths out of 39files touched across both sides. This branch's two files that overlap with
#818 --
docker/qemu/run-x86-boot-tests.shandtests/teardown_structure.rs-- had disjoint hunks against main's changes to the same files (main touched
the preflight rejection and the per-run
FAILlines; this branch touched theFCNTL_PM_CONTENTION_ORACLEliteral and onescore_serialcondition furtherup the file), so
gitcombined both sides without conflict markers. Post-merge,git diff --stat origin/main..HEADlists exactly the 16 files this branch'sown pre-merge diff against the merge-base already listed, byte-identical.
Fixture re-record
ttbr0_shadow_reconciliation_structure.rs'sboth_aarch64_gates_fail_on_an_untagged_publishreplays two committed serials through
run-aarch64-boot-test-strict.shandrun-aarch64-prod-profile-boot-test.shin their scoring-only modes. Thisround's
FCNTL_PM_CONTENTION_ORACLE_PATTERNaddition to the strict gate'sscore_serialrequires that pattern's line in any serial the strict scorer isto accept, and the committed
03-strict-boot1-serial.txtpredates the marker,so it scored
FAIL-- the strict gate did not regress; the scorer it isreplayed against grew a new required line. The fixture is re-recorded from a
single boot of this branch's own
boot_testskernel, which carries[FCNTL_PM_CONTENTION_ORACLE:aarch64:...:PASS], and the strict scorer acceptsthe re-recorded file.
04-prod-boot1-serial.txtis unchanged: the productionscorer's addition this round only asserts the marker's absence, and the
pre-existing fixture already reads 0 because it predates the marker's addition.
Landing re-smoke (merged head
aa7e0fdd)tests/*_structure.rs(31 suites)scripts/test_claim_lint.py--features boot_tests) +check-kernel-no-neon.shrun-aarch64-boot-test-strict.sh 1run-aarch64-prod-profile-boot-test.sh--features testing,external_test_bins)run-x86-boot-tests.sh 1(beast)x86 frame-custody gate run 1: PASSrun-x86-prod-profile-boot-test.sh(beast)PASS: x86 production profile reached steady state with the teardown census at restThe aarch64 strict boot's serial carries:
the x86 boot-tests serial carries:
and the x86 production gate reports
test-only marker '[FCNTL_PM_CONTENTION_ORACLE:': 0.0 reds out of the 8 rows in the table above. Full detail (host-load counts,
BREENIX_GATE_TMPhandling) is in the doc's own "Landing re-smoke" section.🤖 Generated with Claude Code