Skip to content

test(engine): drop non-portable 5k EPS floor in DurationBound (IB-21) - #90

Merged
miladaslaner merged 1 commit into
mainfrom
fix/IB-21-durationbound-5k-floor-portable
Jul 2, 2026
Merged

test(engine): drop non-portable 5k EPS floor in DurationBound (IB-21)#90
miladaslaner merged 1 commit into
mainfrom
fix/IB-21-durationbound-5k-floor-portable

Conversation

@miladaslaner

Copy link
Copy Markdown
Owner

What & why (IB-21, Bug — release-blocker)

TestRun_DurationBound/5k_eps blocked the v0.4.1 release: its sustained-EPS lower floor (generated ≥ 0.85× eps·after) is not portable to a 2-core GitHub Actions runner, which cannot physically push 5k EPS through generate→pipeline→send (CI ratio ~0.34 → did not sustain EPS), yet passes on a fast dev box (~1.04). This is a test-portability defect — the IB-19 engine fix is correct and unchanged.

Change (founder-confirmed Option (c) — test-only)

  • Removed the 5k subtest's sustained-EPS lower-floor assertion (old engine_test.go:528-531) and its now-dead minGenRatio const. No capability-probe / measured-send-rate / skip-with-log machinery.
  • Retained the sustained-EPS floor only at the 1k subtest (10 req/s — trivially sustainable on 2 cores).
  • Preserved the IB-19 stop.after overrun guard (wall ≤ after+1.0s AND generated ≤ 1.15× eps·after) universally at BOTH 1k and 5k, tolerances unchanged.
  • Added an AC4 governance comment at both overrun blocks: these are the IB-19 regression guard and must not be hidden behind testing.Short() (both ci.yml:58 and release.yml:35 run this test under -race -short).

Out of scope (unchanged): internal/engine product code, IB-19, overrun tolerances, ci.yml/release.yml flags.

Evidence

  • Fail-before (overrun guard, be2d195 pre-IB-19 engine, current test overlaid): 1k wall=9.248s / ratio 3.63×, 5k wall=6.021s / ratio 1.32× → overrun assertions FAIL (5k fails via the upper bound even with the floor removed, proving the guard is preserved).
  • Pass-after (branch): go test -race ./internal/engine -run TestRun_DurationBound -v → 1k ratio 1.10, 5k ratio 1.03, both PASS.
  • Authoritative gate go test -race -short ./... (== ci.yml/release.yml): 28 packages ok, 0 FAIL, exit 0.
  • go build ./... ✓ · go vet ./... ✓ · gofmt -l . clean · golangci-lint run ./internal/engine/... → 0 issues.

(The non--short A1a/A1b throughput acceptance benchmarks in bench_a1_test.go fail on constrained/loaded hardware by design — they need ~100k EPS on a fast dedicated box and are -short-skipped in every CI/release gate. That file is untouched here.)

Diff: internal/engine/engine_test.go only (test-only).

🤖 Generated with Claude Code

The TestRun_DurationBound/5k_eps sustained-EPS lower floor (generated ≥
0.85× eps·after) is not portable to constrained CI: a shared 2-core
runner cannot physically push 5k EPS through generate→pipeline→send, so
it lands far below the floor (ratio ~0.34) and failed the v0.4.1 release
Test gate, though it passes on a fast dev box (~1.04).

Remove the 5k lower-floor assertion outright (founder Option (c)); retain
it only at the 1k subtest (10 req/s, trivially sustainable on 2 cores).
No capability probe / measured-send-rate / skip machinery.

The IB-19 stop.after overrun guard (wall ≤ after+1s AND generated ≤
1.15× eps·after) is preserved universally at BOTH 1k and 5k — it still
fails-before on be2d195 (1k 3.63×/9.25s, 5k 1.32×/6.02s) and passes-after
on main. A governance comment marks those assertions as the IB-19
regression guard that must not be hidden behind testing.Short(), so
ci.yml and release.yml keep catching the class pre-merge.

Test-only: no internal/engine product code changed; IB-19 not re-opened.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@miladaslaner
miladaslaner merged commit 4950dd6 into main Jul 2, 2026
4 checks passed
@miladaslaner
miladaslaner deleted the fix/IB-21-durationbound-5k-floor-portable branch July 2, 2026 15:42
miladaslaner added a commit that referenced this pull request Jul 22, 2026
…#90)

The TestRun_DurationBound/5k_eps sustained-EPS lower floor (generated ≥
0.85× eps·after) is not portable to constrained CI: a shared 2-core
runner cannot physically push 5k EPS through generate→pipeline→send, so
it lands far below the floor (ratio ~0.34) and failed the v0.4.1 release
Test gate, though it passes on a fast dev box (~1.04).

Remove the 5k lower-floor assertion outright (founder Option (c)); retain
it only at the 1k subtest (10 req/s, trivially sustainable on 2 cores).
No capability probe / measured-send-rate / skip machinery.

The IB-19 stop.after overrun guard (wall ≤ after+1s AND generated ≤
1.15× eps·after) is preserved universally at BOTH 1k and 5k — it still
fails-before on 9bf3c44 (1k 3.63×/9.25s, 5k 1.32×/6.02s) and passes-after
on main. A governance comment marks those assertions as the IB-19
regression guard that must not be hidden behind testing.Short(), so
ci.yml and release.yml keep catching the class pre-merge.

Test-only: no internal/engine product code changed; IB-19 not re-opened.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant