fix(beat): the GaussianNB "4.9x" was a dev-box number - re-pin to the host that actually runs it - #2328
Merged
Merged
Conversation
… host that actually runs it PMAT-GNB-SPEED-DEFENSE-001. The beat pinned "Measured ~0.20 (apr ~4.9x faster on a 16-core box)" against a 0.50 ceiling, which reads as 2.5x of headroom. The real headroom on the machine that runs this beat is about 20%, and the gate has been tripping: 2026-07-03 ratio 0.563, 07-04 0.520, 07-28 0.601. MEASURED THIS SESSION (the 4.9x has never been observed on the CI host): lambda-vector (modern 48-thread box): 0.188 / 0.237 / 0.220 (apr 4.2-5.3x) mac-server (canonical nightly host, IDLE): 0.398 (apr 2.51x) mac-server under nightly load (run logs): 0.278 .. 0.601 (straddles 0.50) So this is NOT decay. `git log --since=2026-07-01 -S"GaussianNB" -- crates/aprender-core/src/` is empty; the last impl change was #2181 on 2026-06-22, and the ratio has been bouncing across that whole window. It is a different machine, plus contention. WHY THE HOST MOVES IT SO MUCH. apr's GaussianNB is a scalar Rust loop; sklearn's is vectorised numpy. mac-server is a Xeon W-3245 with AVX-512 (avx512f/bw/cd/dq/vl), so numpy closes much of the gap there: moving from lambda-vector to mac-server costs apr 2.3x (12.0ms -> 28.0ms) but sklearn only 1.25x (56ms -> 70.3ms). On top of that the box runs 17 registered runners on 32 threads, so a busy night pushes it the rest of the way over 0.50. WHAT THIS CHANGE DOES - and does NOT do. It does NOT weaken the gate. RATIO_CEILING stays 0.50. The idle-host reading (0.398) clears it, so a run over 0.50 remains real signal rather than expected noise. Moving the threshold to make red go away would concede a win we still hold (see the standing rule against conceding speed without evidence). It fixes the false baseline in the doc comment, recording each measurement WITH the host it came from, so nobody re-tightens the gate on a number that machine cannot produce. It adds a fit/predict phase split, printed on every run and included in the failure message. That is the missing diagnostic: contention drags both phases up together, while an algorithmic regression lands in one. With only a fused total, five red nights told you nothing actionable. Verified locally: BEAT-SKLEARN-GAUSSIANNB-SPEED: apr=11.675ms sklearn=51.795ms ratio=0.225 (apr 4.44x) BEAT-SKLEARN-GAUSSIANNB-SPEED-PHASES: fit apr=4.666ms skl=15.508ms ratio=0.301 | predict apr=7.078ms skl=37.199ms ratio=0.190 predict is where apr's edge is largest (5.3x) and fit is the weaker half (3.3x), which also says where the durable fix belongs: SIMD the predict hot loop via trueno to restore real margin on AVX-512 hosts. That is the actual defence of this beat and remains open under PMAT-GNB-SPEED-DEFENSE-001 - deliberately not attempted here, because it needs its own before/after measurement on mac-server rather than a dev box. Note this beat has been UNOBSERVABLE since 2026-07-26: the nightly lane died at step 1 on a poisoned uv env cache and fail-fast skipped every later beat (fixed in #2326). The 0.601 reading above came from re-running the lane after repairing the host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PMAT-GNB-SPEED-DEFENSE-001. The beat pinned "Measured ~0.20 (apr ~4.9× faster on a 16-core box)" against a 0.50 ceiling — which reads as 2.5× of headroom. The real headroom on the machine that runs this beat is about 20%, and the gate has been tripping: 07-030.563, 07-040.520, 07-280.601.Measured this session
This is not decay.
git log --since=2026-07-01 -S"GaussianNB" -- crates/aprender-core/src/is empty; the last impl change was #2181 (2026-06-22) and the ratio has bounced across that entire window. It's a different machine, plus contention.Why the host moves it so much
apr's GaussianNB is a scalar Rust loop; sklearn's is vectorised numpy. mac-server is a Xeon W-3245 with AVX-512 (
avx512f/bw/cd/dq/vl), so numpy closes much of the gap there: lambda → mac-server costs apr 2.3× (12.0 → 28.0 ms) but sklearn only 1.25× (56 → 70.3 ms). On top of that the box runs 17 registered runners on 32 threads, which pushes a busy night the rest of the way over 0.50.What this does — and does not do
It does NOT weaken the gate.
RATIO_CEILINGstays0.50. The idle-host reading (0.398) clears it, so a run over 0.50 remains real signal rather than expected noise. Moving the threshold to make red go away would concede a win we still hold.It fixes the false baseline, recording each measurement with the host it came from, so nobody re-tightens the gate on a number that machine cannot produce.
It adds a fit/predict phase split, printed every run and included in the failure message — the missing diagnostic. Contention drags both phases up together; an algorithmic regression lands in one. With only a fused total, five red nights told you nothing actionable.
predictis where apr's edge is largest (5.3×) andfitis the weaker half (3.3×) — which also says where the durable fix belongs: SIMD the predict hot loop via trueno to restore real margin on AVX-512 hosts. That remains open underPMAT-GNB-SPEED-DEFENSE-001, deliberately not attempted here because it needs its own before/after on mac-server rather than a dev box.🤖 Generated with Claude Code