Severity: HIGH
Location: src/review/auto-apply.ts:145
Found via a full-system adversarial audit of the ORB review/scoring engine (2026-07-15) — 2 independent skeptic passes, both had to vote "confirmed" for this to survive.
Failure scenario
Day 1: repo acme/widgets resolves 10 maintainer-lane recommendation outcomes, 2 positive / 8 negative (mergePrecision=0.2 < RISK_MERGE_PRECISION 0.9). computeTuningRecommendations (auto-tune.ts:295-336) emits a warn rec with overridePayload {confidenceFloor:0.95}; runSelfTune shadow-soaks it (validatedUntil = now+24h). Over the next 24h the underlying cause is fixed and 90 more maintainer-lane outcomes resolve, all positive: cumulative positive=92, negative=8, decided=100, mergePrecision=0.92 — now ABOVE the 0.9 risk floor, so a fresh computeTuningRecommendations run on this data would NOT emit a tightening rec at all. Day 2's cron tick calls evaluateShadowPromotion (auto-apply.ts:145-163), whose signature only takes {override, liveFloor, liveScopeCap, decided, validatedUntilIso, nowIso} — it has no mergePrecision/weightedMergePrecision parameter and never re-derives one. It checks only (a) isStrictlyTightening vs the current live floor (still true, since nothing has raised the live floor yet), (b) decided>=10 (true, 100>=10), (c) soak elapsed (true). All three pass, so the STALE confidenceFloor:0.95 override is promoted to LIVE via writeLiveOverride (auto-apply.ts:359), permanently raising qualityGateMinScore even though the repo's own current, self-measured precision no longer justifies it.
Impact
The one-way tightening ratchet is decoupled from the truth it was supposedly measuring: it applies a 24-hour-old snapshot's verdict without checking whether that verdict still holds, so a single transient bad batch of outcomes produces a permanent, needlessly strict gate that later good behavior can never reverse (only a manual DELETE override clears it). This is exactly the "score affects config, but the correction never happens because the promotion is blind to updated evidence" failure mode the audit was asked to hunt for — contributors on that repo get held/blocked at a stricter bar than the system's own live data says is warranted.
Severity: HIGH
Location:
src/review/auto-apply.ts:145Found via a full-system adversarial audit of the ORB review/scoring engine (2026-07-15) — 2 independent skeptic passes, both had to vote "confirmed" for this to survive.
Failure scenario
Day 1: repo acme/widgets resolves 10 maintainer-lane recommendation outcomes, 2 positive / 8 negative (mergePrecision=0.2 < RISK_MERGE_PRECISION 0.9). computeTuningRecommendations (auto-tune.ts:295-336) emits a warn rec with overridePayload {confidenceFloor:0.95}; runSelfTune shadow-soaks it (validatedUntil = now+24h). Over the next 24h the underlying cause is fixed and 90 more maintainer-lane outcomes resolve, all positive: cumulative positive=92, negative=8, decided=100, mergePrecision=0.92 — now ABOVE the 0.9 risk floor, so a fresh computeTuningRecommendations run on this data would NOT emit a tightening rec at all. Day 2's cron tick calls evaluateShadowPromotion (auto-apply.ts:145-163), whose signature only takes {override, liveFloor, liveScopeCap, decided, validatedUntilIso, nowIso} — it has no mergePrecision/weightedMergePrecision parameter and never re-derives one. It checks only (a) isStrictlyTightening vs the current live floor (still true, since nothing has raised the live floor yet), (b) decided>=10 (true, 100>=10), (c) soak elapsed (true). All three pass, so the STALE confidenceFloor:0.95 override is promoted to LIVE via writeLiveOverride (auto-apply.ts:359), permanently raising qualityGateMinScore even though the repo's own current, self-measured precision no longer justifies it.
Impact
The one-way tightening ratchet is decoupled from the truth it was supposedly measuring: it applies a 24-hour-old snapshot's verdict without checking whether that verdict still holds, so a single transient bad batch of outcomes produces a permanent, needlessly strict gate that later good behavior can never reverse (only a manual DELETE override clears it). This is exactly the "score affects config, but the correction never happens because the promotion is blind to updated evidence" failure mode the audit was asked to hunt for — contributors on that repo get held/blocked at a stricter bar than the system's own live data says is warranted.