diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index db09fae..965221e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -561,16 +561,15 @@ jobs: run: make ci-smoke-mem512 scheduler-baseline: - name: Scheduler perf/stability baseline (ADR-0022 prereq 2, report-only) + name: Scheduler perf/stability baseline (ADR-0022 prereq 2) runs-on: ubuntu-latest needs: [build] - # ADR-0022 prerequisite 2: RECORD the scheduler baseline (preempt/switch per - # 1000 guest ticks, max reschedule gap, run-count spread) so a future scheduler- - # constant change is measured, not blind (ADR-0016). REPORT-ONLY in this PR: it - # asserts only a non-vacuous liveness floor (the scheduler is multiplexing a real - # roster), so it reddens on a dead/wedged scheduler but arms no perf band yet. - # A follow-up arms the calibration-selected scalar (preempt-per-1000-ticks) with a - # SCHED_QUANTUM_TICKS revert-confirm. Not a release blocker. + # ADR-0022 prerequisite 2: measure the scheduler baseline so a future scheduler- + # constant change is measured, not blind (ADR-0016). ASSERTS the calibration- + # selected scalar — preemptions per 1000 guest ticks (~200 at the default quantum) + # within [100, 600] — plus a non-vacuous liveness floor. Guest-tick-normalized, so + # host-CPU-invariant on a slow runner. Revert-confirmed: SCHED_QUANTUM_TICKS 5 -> 20 + # drops the rate to ~50 and reddens the floor. Not a release blocker. steps: - name: Checkout code uses: actions/checkout@v4 @@ -578,7 +577,7 @@ jobs: run: | sudo apt-get update sudo apt-get install -y build-essential qemu-system-x86 nasm python3 - - name: Boot and record the scheduler baseline (liveness floor asserted) + - name: Boot and assert the scheduler baseline (preemption-rate band + liveness) run: make ci-smoke-sched quiet-boot: diff --git a/CHANGELOG.md b/CHANGELOG.md index e564355..cb67607 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -180,6 +180,21 @@ floor of ~100 (= q5-median × 0.5), revert-confirmed by a `SCHED_QUANTUM_TICKS` 20 (q=80 grinds the harness itself, so 20 is the practical lever). Build clean under `-Werror`; format/cppcheck/api-consistency green. +### ADR-0022 prereq-2 — arm the scheduler gate, **ADR Accepted** + +`scripts/test_qos_sched.py` flips from report-only to a real gate: it asserts +**preemptions-per-1000-guest-ticks in `[100, 600]`** (the PR-1-calibrated scalar; ~200 at +the default quantum) on top of the liveness floor. Everything asserted is a delta over +guest ticks, so the band is host-CPU-invariant — a slow CI runner reads the same rate. +The floor catches a *coarsened* round-robin cadence (the scheduler getting more sluggish — +ADR-0022's regression direction); the loose ceiling catches pathological thrash without +reddening on the intended future fix (a *smaller* quantum raises the rate). **Revert- +confirmed** on WSL: at the default `SCHED_QUANTUM_TICKS=5` the gate is green (rate 199.9); +bumping it to 20 drops the rate to 49.9 and reddens the floor; restoring returns it to +green. This completes ADR-0022's second prerequisite, so the ADR flips to **Accepted** — +the deferral decision and both prerequisites (latency gate + scheduler baseline) are done; +the latency fix itself stays a deliberately deferred, now-unblocked future epic. + ## [0.5.0] — 2026-07-11 — Agent-reachable QPU, hardening, dead-code payoff Post-v0.4.0 increments landed autonomously through the panel → gate (revert-and-confirm) diff --git a/Makefile b/Makefile index 5b69b4f..a1e7ac2 100644 --- a/Makefile +++ b/Makefile @@ -2211,12 +2211,13 @@ ci-smoke-latency: kernel @echo "=== QuantumOS COM2 latency Test (ADR-0022) ===" QOS_KERNEL=$(BUILD_DIR)/kernel.elf32 python3 scripts/test_qos_latency.py -# Scheduler perf/stability baseline recorder (ADR-0022 prereq 2). REPORT-ONLY: -# emits the tick-normalized SCHED baseline (preempt/switch per 1000 guest ticks, -# max reschedule gap, run-count spread) and asserts ONLY a non-vacuous liveness -# floor (the scheduler is multiplexing a real roster), so it reddens on a -# dead/wedged scheduler but arms no perf band yet — PR-2 arms the calibration- -# selected scalar. QOS_SCHED_LOAD=1 drives COM2 PING load during sampling. +# Scheduler perf/stability baseline gate (ADR-0022 prereq 2). Emits the tick- +# normalized SCHED baseline (preempt/switch per 1000 guest ticks, max reschedule +# gap, run-count spread) and ASSERTS the calibration-selected scalar — preemptions +# per 1000 guest ticks, ~200 at the default quantum — within [100, 600], plus a +# non-vacuous liveness floor. Guest-tick-normalized, so host-CPU-invariant on slow +# CI. Revert-confirmed: SCHED_QUANTUM_TICKS 5 -> 20 drops the rate to ~50 and reddens +# the floor. QOS_SCHED_LOAD=1 drives COM2 PING load during sampling. ci-smoke-sched: kernel @echo "=== QuantumOS scheduler baseline recorder (ADR-0022 prereq 2) ===" QOS_KERNEL=$(BUILD_DIR)/kernel.elf32 python3 scripts/test_qos_sched.py diff --git a/docs/adr/0022-com2-latency-scheduler-bound.md b/docs/adr/0022-com2-latency-scheduler-bound.md index 9600f30..5114691 100644 --- a/docs/adr/0022-com2-latency-scheduler-bound.md +++ b/docs/adr/0022-com2-latency-scheduler-bound.md @@ -1,15 +1,30 @@ # 22. COM2 Round-Trip Latency Is Scheduler-Cadence-Bound Date: 2026-07-11 -Status: Proposed (root cause measured; a fix is DEFERRED — **prerequisite 1, the latency gate, now shipped**) +Status: Accepted (2026-07-13; the deferral decision + both prerequisites are complete — the latency fix itself remains a deliberately deferred future epic) > **Update (2026-07-11).** Prerequisite 1 below — a bounded latency assertion — **shipped** as > `ci-smoke-latency` (`scripts/test_qos_latency.py`, via the new `QosVM.ping()` one-hop primitive). > It asserts a bounded PING/PONG median (the pure transport + scheduler-cadence floor) and records the > STATUS baseline; it is revert-confirmed against the real regression source — bumping > `SCHED_QUANTUM_TICKS` reddens it. This gives any future latency-reduction work a proven guard and a -> recorded baseline (~0.45 s PING / ~0.90 s STATUS on the dev box). Prerequisite 2 (a scheduler perf -> baseline) and the fix itself remain open. +> recorded baseline (~0.45 s PING / ~0.90 s STATUS on the dev box). +> +> **Update (2026-07-13) — Accepted, both prerequisites complete.** Prerequisite 2 (the scheduler +> perf/stability baseline) shipped as `ci-smoke-latency`'s sibling `ci-smoke-sched` +> (`scripts/test_qos_sched.py`, CI job `scheduler-baseline`). A design panel refuted the naive metric — +> aggregate context switches are voluntary-yield-dominated, so `switch_count/tick` is both quantum- +> insensitive and host-throughput-dependent. The kernel instead grew a **dedicated `preempt_count`** +> incremented only at timer-quantum expiry (never on `SYS_YIELD`), exposed via a `SYSINFO_SCHED` sub-op +> / qsh `sched` command / `QosVM.sched()`, alongside a made-live `last_scheduled` and a per-PCB +> `sched_picks` for the fairness/tail snapshot. Calibration (WSL) confirmed **preemptions-per-1000- +> guest-ticks ≈ 1000/quantum, host-invariant** (~200 at q=5 idle+load with <0.5 % variance, ~50 at +> q=20); the gate asserts that scalar in `[100, 600]` plus a liveness floor, revert-confirmed by a +> `SCHED_QUANTUM_TICKS` 5→20 bump (rate drops to ~50, reddens the floor). Both prerequisites the fix was +> gated on now exist. **The latency fix itself (the I/O-priority-boost epic, or a measured quantum +> reduction) stays deferred** — it is now unblocked, with a proven latency gate AND a proven scheduler +> baseline to measure it against, but remains a future epic per this ADR's decision, not an autonomous +> increment. ## Context diff --git a/scripts/test_qos_sched.py b/scripts/test_qos_sched.py index d1e33f0..663ba62 100644 --- a/scripts/test_qos_sched.py +++ b/scripts/test_qos_sched.py @@ -1,26 +1,24 @@ #!/usr/bin/env python3 """ -Scheduler perf/stability baseline recorder (ADR-0022, prerequisite 2) — REPORT-ONLY. +Scheduler perf/stability baseline gate (ADR-0022, prerequisite 2). ADR-0022 root-caused the agent-tool latency floor to the scheduler cadence and DEFERS the fix behind two prerequisites. Prereq 1 (the COM2 latency gate, test_qos_latency.py) shipped. This is prereq 2: a perf/stability baseline so a future scheduler-constant change (the deferred fix) is MEASURED, not blind (ADR-0016 anti-vacuous discipline). -WHY REPORT-ONLY (PR-1): the design panel's red-team refuted the naive metric. Aggregate -context switches (`switches`) are dominated by VOLUNTARY yields — every long-lived citizen -busy-yields, resetting the quantum counter thousands of times per 10 ms tick, so the timer- -preemption path is nearly dead at rest. switches/tick is therefore BOTH quantum-insensitive -(a SCHED_QUANTUM_TICKS change does not move it -> a gate on it is vacuous) AND host-CPU- -throughput dependent (yield-loop iterations per tick scale with guest speed -> flaky on slow -CI). The kernel now exposes a DEDICATED `preempt` counter incremented ONLY on timer-quantum -expiry (never on SYS_YIELD), which is 1/quantum-paced by construction. But whether `preempt` -is non-zero enough UNDER LOAD to separate q=5 from q=80 is an EMPIRICAL question. So PR-1 -does NOT assert any perf band. It RECORDS every candidate (preempt-per-1000-ticks, switch- -per-1000-ticks, max reschedule gap, run-count spread) under idle and under COM2 PING load, -across the SCHED_QUANTUM_TICKS values, to CALIBRATE which scalar PR-2 will gate. The only -assertion here is a non-vacuous LIVENESS floor (the scheduler is actually multiplexing a -real roster) so this job still fails on a dead/wedged/frozen scheduler. +THE METRIC (design-panel-refined, calibration-selected): the design panel's red-team +refuted the naive metric. Aggregate context switches (`switches`) are dominated by +VOLUNTARY yields — every long-lived citizen busy-yields, resetting the quantum counter +thousands of times per 10 ms tick, so the timer-preemption path is nearly dead at rest. +switches/tick is therefore BOTH quantum-insensitive (a gate on it is vacuous) AND host-CPU +dependent (flaky on slow CI). The kernel instead exposes a DEDICATED `preempt` counter, +incremented ONLY on timer-quantum expiry (never on SYS_YIELD). PR-1's calibration confirmed +preempt-per-1000-guest-ticks = ~1000/quantum, host-invariant (~200 at q=5 idle+load with +<0.5% variance, ~50 at q=20). So this gate ASSERTS that scalar in [FLOOR, CEIL] (see the +constants below) — plus a non-vacuous LIVENESS floor so it also reddens on a dead/wedged/ +frozen scheduler. Everything asserted is a delta over GUEST ticks, so a slow CI runner reads +the same value (it just takes longer wall-clock to fill the window). Env: QOS_KERNEL path to kernel.elf32 (else the bridge default) @@ -46,6 +44,18 @@ LIVENESS_MIN_SWITCHES = 100 # context switches over the measured window LIVENESS_MIN_RUNNABLE = 12 # settled runnable count (observed ~20 at rest) +# Armed perf band on the calibration-selected scalar: preemptions per 1000 guest ticks. +# WSL calibration measured preempt/1000t = ~200 at SCHED_QUANTUM_TICKS=5 (idle AND load, +# <0.5% variance) and ~50 at q=20 — i.e. it tracks 1000/quantum, host-invariant (a ratio +# of guest-side counters, so a slow CI runner reads the same value). FLOOR = q5-median x +# 0.5 = 100 is load-bearing: it reddens when the round-robin cadence COARSENS (quantum +# raised past ~10, the sluggish direction ADR-0022 warns about) — a SCHED_QUANTUM_TICKS +# bump to 20 drops P to ~50 and trips it (revert-confirmed). CEIL = q5-median x 3 = 600 is +# a loose thrash cap that does NOT redden on the intended future fix (a SMALLER quantum +# raises P; q=2 -> ~500 still passes), only on pathological churn. +PREEMPT_PER_1000T_FLOOR = 100.0 +PREEMPT_PER_1000T_CEIL = 600.0 + # State-gated warmup by GUEST TICKS: the transient boot self-tests (echo, watched-svc, # quota-test, delegation-test, cpu-hog, qpu-test) run and exit inside the first few # hundred ticks; cpu-hog in particular holds whole quanta and pollutes the preemption @@ -136,19 +146,33 @@ def main(): f"switch_per_1000t={p_switch:.2f} maxgap={max_gap} spread={max_spread} " f"runnable_min={min_runnable}") - # NON-VACUOUS LIVENESS FLOOR (the only assertion in PR-1): the scheduler must be - # multiplexing a real roster. Reddens on a dead scheduler (no switches), a single- - # runnable degenerate, or a frozen guest clock — but NOT on any healthy cadence, so - # it is not a perf band (that is PR-2, armed on the calibration-selected scalar). + # NON-VACUOUS LIVENESS FLOOR: the scheduler must be multiplexing a real roster. + # Reddens on a dead scheduler (no switches), a single-runnable degenerate, or a + # frozen guest clock. Checked before the perf band so a wedged scheduler is + # attributed correctly rather than as a band miss. if d_switch < LIVENESS_MIN_SWITCHES: _fail(f"scheduler not multiplexing: {d_switch} switches over {d_ticks} ticks " f"(< {LIVENESS_MIN_SWITCHES}) — dead/wedged scheduler") if min_runnable < LIVENESS_MIN_RUNNABLE: _fail(f"runnable roster collapsed to {min_runnable} (< {LIVENESS_MIN_RUNNABLE}) " "— citizens not staying schedulable") + + # ARMED PERF BAND (the calibration-selected scalar): preemptions per 1000 guest + # ticks must sit in [FLOOR, CEIL]. The floor catches a coarsened round-robin cadence + # (the scheduler getting more sluggish — ADR-0022's regression direction); the loose + # ceiling catches pathological thrash without reddening on the intended smaller- + # quantum fix. Revert-confirmed: SCHED_QUANTUM_TICKS 5 -> 20 drops P to ~50 < FLOOR. + if p_preempt < PREEMPT_PER_1000T_FLOOR: + _fail(f"preemption rate {p_preempt:.1f}/1000t below floor {PREEMPT_PER_1000T_FLOOR:.0f} " + f"— round-robin cadence has COARSENED (quantum raised? scheduler more sluggish). " + f"dpreempt={d_preempt} over dticks={d_ticks}. See ADR-0022.") + if p_preempt > PREEMPT_PER_1000T_CEIL: + _fail(f"preemption rate {p_preempt:.1f}/1000t above ceiling {PREEMPT_PER_1000T_CEIL:.0f} " + f"— pathological reschedule thrash. dpreempt={d_preempt} over dticks={d_ticks}.") print(f"OK: scheduler live ({d_switch} switches, {min_runnable} runnable over " - f"{d_ticks} ticks). Baseline recorded — no perf band asserted (PR-1 report-only).") - print("=== scheduler baseline recorder PASSED (liveness floor; calibration data emitted) ===") + f"{d_ticks} ticks); preemption rate {p_preempt:.1f}/1000t within " + f"[{PREEMPT_PER_1000T_FLOOR:.0f}, {PREEMPT_PER_1000T_CEIL:.0f}].") + print("=== scheduler baseline gate PASSED (liveness + preemption-rate band) ===") if __name__ == "__main__":