Skip to content

feat(miner): wire the Phase 7 calibration runner (calibration-run.ts) into calibration-cli.ts — it has zero real callers #8317

Description

@JSONbored

Context

packages/loopover-miner/lib/calibration-run.ts (#4248) is the "Phase 7 calibration runner" — its own header comment describes it as the module that finally connects two previously-built-but-unwired halves (the engine's computePhase7CalibrationLoop combine contract and the deterministic replay scorer). It exports runHistoricalReplayCalibrationCycle, recordCalibrationSnapshot, readCalibrationSnapshots, and latestCalibrationSnapshot, and is exercised thoroughly by test/unit/miner-calibration-run.test.ts and test/integration/miner-calibration-loop.test.ts.

None of these functions has a real caller anywhere in packages/loopover-miner/lib/** or bin/** outside their own test files — confirmed by grepping the whole package for runHistoricalReplayCalibrationCycle, recordCalibrationSnapshot, readCalibrationSnapshots, and MINER_CALIBRATION_SNAPSHOT_EVENT. loop-cli.ts (the miner's real attempt loop) has zero references to calibration. This means loopover-miner's Phase 7 calibration snapshot event is never actually written in any real deployment — the whole module is fully built and fully tested, but dead in production. (The closed issue #8185, which extended this module's snapshot payload with a backtest track record, itself assumed in its own body that "the MINER_CALIBRATION_SNAPSHOT_EVENT [is] written each cycle" and pointed at lib/loop-cli.ts as a resource — that assumption does not hold against the actual code.)

runHistoricalReplayCalibrationCycle already tolerates a missing replayRun (it degrades to historicalReplay: null, and the engine combine holds no_historical_replay_signal) — so a real caller does not need the still-unbuilt "replay executor" (replay-task-bridge.ts's own header comment names that executor as the genuinely unbuilt next step, out of scope here) to start producing real, meaningful snapshots today using just the pr_outcome signal.

The prOutcome input (PrOutcomeCalibrationInput = { mergeConfirmed, mergeFalse, closeConfirmed, closeFalse, hold?, observedAt? }, from @loopover/engine) is field-for-field identical to calibration.ts's existing CalibrationRow shape, which calibration-cli.ts's bare calibration command already computes per project via buildCalibrationReport. That report is sitting right next to the runner it should be feeding.

Requirements

⚠️ Required pattern. Mirror calibration-cli.ts's existing backtest-threshold/apply-min-rank/revert-min-rank subcommand shape exactly (parse args[0], open the event ledger, run, persist, print --json or text, return an exit code, close the ledger in finally) for a new calibration snapshot [--json] subcommand.

  • Add a calibration snapshot [--json] subcommand to runCalibrationCli (calibration-cli.ts).
  • For each CalibrationRow in the existing buildCalibrationReport(...).rows (already computed by the bare calibration command's own code path), build a PrOutcomeCalibrationInput by direct field mapping: { mergeConfirmed: row.mergeConfirmed, mergeFalse: row.mergeFalse, closeConfirmed: row.closeConfirmed, closeFalse: row.closeFalse, hold: row.hold }.
  • Call runHistoricalReplayCalibrationCycle({ prOutcome, repoFullName: row.project }, { eventLedger }) once per project row — do not aggregate across projects into one fleet-wide call; every other per-repo-scoped store/report in this package (the calibration report itself, recordCalibrationSnapshot's own repoFullName option, readCalibrationSnapshots's repoFullName filter) already keys on repoFullName, so per-project snapshots are the correct match for the existing read side.
  • Pass backtestTrackRecord into the snapshot too, not just prOutcome: runHistoricalReplayCalibrationCycle's snapshotPayloadFromResult call never populates meta.backtestTrackRecord, so the backtestTrackRecord field ams-calibration: aggregate backtest track record into the miner calibration report + snapshot event #8185 added to CalibrationSnapshotPayload is currently always null on every snapshot even after this issue's wiring lands. Extend runHistoricalReplayCalibrationCycle (or the new CLI call site — either is acceptable, but pick one and be consistent) to accept and forward a backtestTrackRecord meta value, and pass the bare calibration command's own already-computed computeAmsBacktestTrackRecord(readAmsThresholdBacktestRuns(eventLedger)) result into it (same value the bare command already prints).
  • Do not attempt to read .loopover-ams.yml's miner.calibration/calibration section for Phase7CalibrationConfig in this issue — no reader for that section exists yet anywhere in the package, and building one is out of scope here. Omit config entirely (the engine's computePhase7CalibrationLoop defaults to phase7LoopEnabled: false when config is omitted, so the resulting snapshot will legitimately show the loop disabled until a future issue adds config wiring — this is honest, not a bug, and must not be worked around by fabricating a config object).
  • calibration snapshot --json prints the array of per-project { repoFullName, snapshot } results (or an explicit "no decided predictions yet" message when the report has no rows, matching the bare command's own hasSignal handling); the text form prints one line per project.

Deliverables

  • calibration snapshot [--json] subcommand in calibration-cli.ts, following the backtest-threshold subcommand's exact shape.
  • backtestTrackRecord flows into every persisted snapshot's payload (no longer permanently null).
  • Per-project snapshot recording, using buildCalibrationReport's existing rows as the prOutcome source.

Test Coverage Requirements

packages/loopover-miner/** is not Codecov-gated, but npm run test:ci must stay green with full branch coverage in test/unit/miner-calibration-cli.test.ts and/or test/unit/miner-calibration-run.test.ts: cover a report with one project and one with multiple projects (one snapshot event per project, not one merged event), a report with zero rows (hasSignal: false, no snapshot attempted), the --json and text render paths, and a snapshot payload asserting backtestTrackRecord is populated (not null) when backtest runs exist and explicitly null when none do (mirroring normalizeBacktestTrackRecord's existing tolerant-degrade branch).

Expected Outcome

loopover-miner calibration snapshot actually persists calibration_snapshot ledger events from real prediction/outcome data, so readCalibrationSnapshots/latestCalibrationSnapshot (and any future consumer of the Phase 7 metric) have real history to read instead of a permanently-empty ledger type.

Links & Resources

Metadata

Metadata

Assignees

No one assigned

    Labels

    gittensor:featureGittensor-scored feature linked to a feature issue — scores a 0.25x multiplier.help wantedExtra attention is needed

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions