Skip to content

Continuous monitoring and early warning for track-record scores - #2

Merged
prasants merged 1 commit into
mainfrom
feature/score-monitoring
Jun 12, 2026
Merged

Continuous monitoring and early warning for track-record scores#2
prasants merged 1 commit into
mainfrom
feature/score-monitoring

Conversation

@prasants

Copy link
Copy Markdown
Owner

Continuous monitoring and early warning

Stacked on feature/score-firewall (#1); review that first. This adds the
deterioration-detection layer the marketplace-operator workflow needs: a
point-in-time score says how good a record is today, monitoring says whether
it is deteriorating — early enough to act.

Dual-window scoring

  • The expanding window is the authoritative published score.
  • A trailing window is compared against the preceding equal-length
    window
    . Equal lengths make the comparison length-fair — a naive
    "recent vs. whole history" gap is contaminated by record length, since
    sufficiency and the Deflated Sharpe reward more data and make every short
    window look like decay.
  • Divergence is measured on annualised Sharpe, not the composite: the
    composite saturates at the short-record cap on sub-year windows, which
    would flatten the signal exactly where it must be sharp. TRAILING_DIVERGENCE
    is the pre-emptive alert; it fires before the slow expanding score reacts.

What's in it

  • score.monitoringCadence (daily/weekly/monthly/yearly), Monitor,
    ScoreSnapshot, ScoreHistory, Alert / AlertKind / Severity.
  • Alerts: trailing divergence, new critical flag (manipulation emerging
    over time), grade drop, score drop — each referencing its triggering
    snapshot.
  • Auditable history: an append-only, hash-chained ScoreHistory;
    is_chain_valid() proves the record was not reordered or back-edited, and
    each snapshot stays firewall-verifiable.
  • Thresholds are operator-tunable parameters; delivery (webhook, dashboard,
    polling) is a hosted adapter over the emitted data structures.

Quality

  • 25 new tests; full score package suite (59 tests) passing. ruff lint +
    format clean. Additive: no changes to the scoring engine or firewall.

Docs

  • docs/monitoring.md — specification, dual-window rationale, and the
    calibration/delivery boundaries.

A point-in-time score says how good a record is today; monitoring says
whether it is deteriorating, early enough to act. The scoring machinery is
run forward through time, emitting alerts when a record turns.

- Dual-window scoring: an expanding window is the authoritative published
  number; a trailing window compared against the preceding equal-length
  window is the early-warning signal. Equal lengths keep the comparison
  length-fair, and divergence is measured on annualised Sharpe (uncapped,
  sensitive on short windows) rather than the integrity-capped composite.
- Cadences: daily, weekly, monthly, yearly.
- Alerts: trailing divergence (pre-emptive), new critical flag, grade drop,
  score drop; each references the triggering snapshot.
- Append-only, hash-chained ScoreHistory proves the record was not reordered
  or back-edited; each snapshot stays firewall-verifiable.
- Thresholds are operator-tunable; delivery is a hosted adapter over the
  emitted data structures. 25 new tests; docs/monitoring.md spec.
@prasants
prasants changed the base branch from feature/score-firewall to main June 12, 2026 11:16
@prasants
prasants merged commit a02f632 into main Jun 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant