You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Epic #8211 track A. The single largest wrongness source identified by the #8170 retro analysis is STALE configuration: a threshold that was right when set, wrong after the repo's reality moved. The loosening evaluator (evaluateKnobLoosening, src/services/loosening-knobs.ts) already answers 'can we safely loosen?' — nothing answers the inverse operator question: 'is what is CURRENTLY live still the best-supported setting, in either direction?'
Requirements
⚠️ Required pattern. PURE module beside the registry, mirroring evaluateKnobLoosening's exact discipline: deterministic, no IO, same split primitive (splitBacktestCorpus with the knob's own seed/fraction), same Pareto floor (compareBacktestScores), same never-on-noise sample minimums from the registry entry. Anchor on evaluateKnobLoosening and the #8086 comparator as your ≥2 analogues.
evaluateKnobDrift(knob, cases, liveValue, nowMs) → a drift report or null: evaluates EVERY registry candidate plus the shipped value (not only loosenings — a TIGHTER alternative dominating live is exactly the stale-config signal), and reports the best strictly-dominating alternative with both split comparisons attached.
Distinguish direction in the result ("looser" | "tighter" | "shipped"), because the consumer's messaging differs: a looser winner duplicates the loosening loop's own proposal (informational); a tighter winner means live config is likely stale (actionable warning).
Null (never a guess) when the corpus misses the knob's sample minimums or nothing strictly dominates on the visible split while non-regressing held-out.
Problem
Epic #8211 track A. The single largest wrongness source identified by the #8170 retro analysis is STALE configuration: a threshold that was right when set, wrong after the repo's reality moved. The loosening evaluator (
evaluateKnobLoosening, src/services/loosening-knobs.ts) already answers 'can we safely loosen?' — nothing answers the inverse operator question: 'is what is CURRENTLY live still the best-supported setting, in either direction?'Requirements
evaluateKnobDrift(knob, cases, liveValue, nowMs)→ a drift report or null: evaluates EVERY registry candidate plus the shipped value (not only loosenings — a TIGHTER alternative dominating live is exactly the stale-config signal), and reports the best strictly-dominating alternative with both split comparisons attached."looser" | "tighter" | "shipped"), because the consumer's messaging differs: a looser winner duplicates the loosening loop's own proposal (informational); a tighter winner means live config is likely stale (actionable warning).Deliverables
Test Coverage Requirements
Mirror test/unit/loosening-knobs.test.ts's membership-probe seeding technique. Both sides of every ternary/nullish arm.
Links & Resources
#8211 (epic), src/services/loosening-knobs.ts, packages/loopover-engine/src/calibration/backtest-compare.ts
Boundaries
Pure evaluation only — no cron, no alert, no writes (the wire is a separate maintainer issue). No new dependencies.