feat(miner): add calibration backtest-eligibility for ContributionProfile replay

[andriypolanski]

Summary

• Closes miner: calibration backtest-eligibility — replay a candidate ContributionProfile against the captured exclusion corpus #8545
• New pure core packages/loopover-miner/lib/ams-eligibility-backtest.ts exporting backtestEligibilityCandidate(events, currentProfiles, candidateProfiles): reads the four eligibility ruleIds from signal_rule_fired / signal_human_override events, reconstructs FilterCandidate from bounded metadata (miner: capture bounded candidate context (labels/assignees/owner) in eligibility-exclusion fired-event metadata #8544), classifies via filterCandidatesByProfiles (no filter reimplementation), and scores with the same engine backtest helpers + min-rank sample floors as ams-calibration: backtest a proposed min-rank skip threshold against the node's own selection history #8184.
• New CLI subcommand calibration backtest-eligibility --profile <path> [--json] in calibration-cli.ts, mirroring runBacktestThreshold: under-floored corpus prints an explicit line and exits 0 without persisting; success prints skippedNoContext and renderBacktestComparison for visible + held-out splits; invalid/missing --profile → exit 1; operational errors → exit 2.
• Persists each completed run as ams_eligibility_backtest_run via recordAmsEligibilityBacktestRun; computeAmsBacktestTrackRecord now aggregates threshold + eligibility runs (parity with ams-calibration: aggregate backtest track record into the miner calibration report + snapshot event #8185). Main calibration report and calibration snapshot include eligibility runs in track-record attachment.

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck (via npm run build:miner)
• npm run build:miner
• npx vitest run test/unit/miner-ams-eligibility-backtest.test.ts test/unit/miner-calibration-cli.test.ts — 47 passed
• CI-scoped Codecov simulation: npx vitest run test/unit/miner-ams-eligibility-backtest.test.ts test/unit/miner-calibration-cli.test.ts --coverage.all=false --coverage && npx diff-cover coverage/lcov.info --compare-branch=origin/main — 100% patch lines (147/147) and 100% patch branches (188/188) on changed miner files
• npm run test:coverage (full unsharded gate)
• npm run test:ci
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• Miner-only change (packages/loopover-miner/lib/**). Tests import .ts via variable specifiers (miner-ams-eligibility-backtest.test.ts, miner-calibration-cli.test.ts) so codecov/patch grades changed lines under --coverage.all=false. Full test:ci left to CI.

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

Not applicable — miner calibration CLI advisory backtest only.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[loopover-orb]

Tip

✅ LoopOver review result - approve/merge recommended

_{Review updated: 2026-07-24 21:49:58 UTC}

5 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

• safe to merge

Review summary
This PR adds a pure eligibility-exclusion backtest module and a `calibration backtest-eligibility` CLI subcommand, mirroring the existing #8184/#8185 threshold-backtest shape closely and reusing the same engine primitives, min-rank sample floors, and track-record aggregation. The wiring is consistent: corpus building skips metadata-less events into `skippedNoContext` rather than fabricating data, the classifier correctly reconstructs candidates and re-runs the real `filterCandidatesByProfiles`, and `computeAmsBacktestTrackRecord` cleanly extends to accept eligibility runs as an additional optional argument, preserving backward-compatible call sites. Test coverage is extensive (517 lines) and appears to exercise both split floors, malformed-row tolerance, and CLI exit-code paths (0/1/2) per the stated contract.

Nits — 5 non-blocking

• The 'debug leftovers' flag on calibration-cli.ts:330-343 is a false positive — those are the existing `console.log` output lines for the new CLI subcommand's human-readable report, not debug leftovers; still worth confirming no stray `console.log` was left in for local testing.
• packages/loopover-miner/lib/ams-eligibility-backtest.ts:163 and calibration-cli.ts:346 exceed the depth-4 nesting threshold — consider extracting the fired/override branch or the try/catch error-mapping into a small helper for readability.
• calibration-cli.ts is now 434 lines, over the 400-line soft threshold; the new `backtest-eligibility` handler plus its two new helper functions (`loadCandidateProfiles`, `readCurrentContributionProfiles`) could move into ams-eligibility-backtest.ts or a small cli-helpers module to keep the CLI file thin.
• `reconstructEligibilityFilterCandidate`'s regex `^([^/]+\/[^/#]+)#issue-(\d+)$` silently drops any targetKey that doesn't match this exact shape — worth a one-line comment noting this is intentional (fail-closed to 'reversed') rather than a parsing gap, since it's not obvious from the diff alone.
• Consider a short doc comment on `AMS_ELIGIBILITY_SPLIT_SEED` clarifying why a distinct seed from min-rank's is needed despite reusing its constants (namespace collision avoidance) — currently only implied.

Decision drivers

• ✅ Code review — No blockers (1 reviewer)
• ✅ Gate result — Passing (No configured blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8545
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 224 registered-repo PR(s), 142 merged, 22 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	andriypolanski; Gittensor profile; 224 PR(s), 22 issue(s).
Improvement	✅ Minor	risk: clean · value: minor · LLM: moderate

Linked issue satisfaction

Addressed
The PR adds the pure ams-eligibility-backtest.ts core reading the four eligibility ruleIds from signal_rule_fired/signal_human_override events, reconstructing FilterCandidate from metadata with skippedNoContext tracking, classifying via the imported (not reimplemented) filterCandidatesByProfiles, reusing ams-rank-corpus.ts's split/floor constants and returning null when under-floored, plus a backt

Review context

• Author: andriypolanski
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, Rust, Cuda, JavaScript, Kotlin, MDX, Scala
• Official Gittensor activity: 224 PR(s), 22 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Keep the PR focused and include validation evidence before maintainer review.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver approves — the gate is satisfied and CI is green.