feat(engine): build a labeled backtest corpus from rule-fired and human-override history

[cleanjunc]

Summary

Closes #8083.

Adds packages/loopover-engine/src/calibration/backtest-corpus.ts — the labeled backtest corpus builder the parent epic (#8082) needs: where computeRulePrecision (signal-tracking.ts) aggregates fired/override pairs into one precision number, buildBacktestCorpus(ruleId, fired, overrides) keeps each paired firing as an individual replayable BacktestCase ("this rule fired against this target, and a human later said it was right/wrong").

Everything follows the issue's spec exactly:

• Pure and self-contained — no IO, no DB, no env, no imports beyond signal-tracking.ts's existing event types.
• BacktestCase exactly as specified; metadata is the fired event's metadata, omitted entirely (never set to undefined) when absent — the same optional-property discipline RuleFiredEvent uses.
• Pairing rule (documented in the function's doc comment, computeRulePrecision-style): a firing pairs with the override sharing the function's ruleId and the same targetKey, choosing the override closest in time strictly after that firing; when none strictly follows, the most recent override by occurredAt. One case per fired event, never duplicates. Unmatched firings are excluded — mirrors computeRulePrecision's "only the decided ones count" discipline.
• ruleId filtering mirrors overrideMatchesRule's one-line filter with a comment noting the mirror; signal-tracking.ts itself is untouched (additive-only, per the issue's boundary).
• Barrel export added on its own line immediately after the existing signal-tracking.js export in packages/loopover-engine/src/index.ts, exactly where the issue specifies.

Tests land in both suites:

• packages/loopover-engine/test/backtest-corpus.test.ts (the issue's mandated file, mirroring signal-tracking.test.ts's fixture style) — all five specified cases plus the equal-instant boundary of "strictly follows".
• test/unit/backtest-corpus.test.ts — a root-side vitest twin importing the engine src directly (the same direct-src pattern test/contract/*-parity.test.ts and test/integration/miner-*.test.ts already use). The engine package's own node --test suite runs against dist/ outside root vitest's coverage instrumentation, so this twin is what makes the new file's coverage visible to the codecov/patch gate rather than reporting 0%.

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• Instead of the full unsharded test:coverage (30+ min on this machine), I ran the directly-affected suites: npm run build --workspace @loopover/engine (green) + the engine package's node --test on the new test file (9/9) + root vitest on the new twin with coverage (7/7), and measured patch coverage empirically by intersecting coverage/lcov.info DA/BRDA records with git diff -U0 against upstream/main: 100% of changed lines and branches in backtest-corpus.ts (both sides of every conditional: has-override vs no-override, single vs multiple overrides including the no-strictly-following fallback, matching vs non-matching ruleId, present vs absent metadata).
• actionlint/test:workers/build:mcp/test:mcp-pack/ui:*/npm audit: untouched surfaces — the diff is one new engine src file, one barrel-export line, and two test files; no workflows, workers, mcp, UI, or dependency changes (audit's current moderate findings exist on upstream/main itself).

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

The three unchecked Safety boxes are N/A: no auth/session/UI change of any kind — a pure, additive engine module plus tests.

UI Evidence

N/A — no UI change (pure engine module).

Notes

• The equal-instant boundary ("an override at exactly the firing's own instant does not count as strictly following — the fallback picks it as most-recent") is pinned by its own test in both suites, since it's the one spot where the two halves of the pairing rule meet.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.01%. Comparing base (f442184) to head (a44bca9).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #8094      +/-   ##
==========================================
- Coverage   92.01%   92.01%   -0.01%     
==========================================
  Files         753      754       +1     
  Lines       77157    77181      +24     
  Branches    23330    23335       +5     
==========================================
+ Hits        70998    71018      +20     
  Misses       5053     5053              
- Partials     1106     1110       +4     

Flag	Coverage Δ
shard-1	57.16% <0.00%> (+2.23%)	⬆️
shard-2	51.35% <0.00%> (-2.78%)	⬇️
shard-3	52.81% <100.00%> (-1.21%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...loopover-engine/src/calibration/backtest-corpus.ts	100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

[loopover-orb]

Caution

🛑 LoopOver review result - reject/close recommended

_{Review updated: 2026-07-22 22:09:06 UTC}

4 files · 1 AI reviewer · 1 blocker · CI green · clean

🛑 Suggested Action - Reject/Close

Review summary
Adds a pure buildBacktestCorpus function that pairs RuleFiredEvent/HumanOverrideEvent records into labeled BacktestCase objects, using a nearest-strictly-following-else-most-recent pairing rule, plus barrel export and dual test suites (engine node:test + root vitest). The pairing logic is correctly implemented and thoroughly tested, including the equal-instant boundary case, ruleId filtering, and metadata omission discipline matching RuleFiredEvent's pattern. Closes #8083 as claimed; additive-only, no existing file behavior changed.

Nits — 6 non-blocking

• backtest-corpus.ts:52-71 has an O(fired × overrides) nested filter/loop; fine at current event volumes but worth noting if this module's inputs grow large.
• backtest-corpus.ts:65 nests to depth 5 (candidates loop → deltaMs check) per the external note; consider extracting the 'find best candidate' logic into a small helper to flatten it, though it reads clearly as-is.
• The `paired as HumanOverrideEvent` cast at backtest-corpus.ts:75 relies on an invariant proven only by comment reasoning — a `// eslint-disable` or `assert`-style guard would make the non-null guarantee checkable by tooling rather than just prose.
• Consider replacing the manual min/max scan loops with a single reduce or two Array.prototype sort-and-pick calls for readability, though the current explicit loops are already correct and clear.
• The dual test-suite pattern (dist/ node:test + src vitest) is explained via a comment in test/unit/backtest-corpus.test.ts — worth confirming this is indeed the established convention other calibration modules follow, since it doubles test maintenance.
• Pull request duplicates other open work — Check for an existing pull request or issue covering this change and coordinate or consolidate before continuing.

Why this is blocked

• Linked issue overlaps another open PR: Other open pull requests reference the same linked issue set: feat(calibration): pure BacktestCase corpus builder from fired/override events #8093, feat(engine): pure BacktestCase corpus builder from fired/override event pairs #8095. — Review the related PRs before spending reviewer time on duplicate work.

📋 Copy for AI agents — paste into your coding agent

Fix the following blocker(s) from this PR review:

1. Linked issue overlaps another open PR: Other open pull requests reference the same linked issue set: #8093, #8095. — Review the related PRs before spending reviewer time on duplicate work.

Decision drivers

• ❌ Code review — 1 blocker (1 reviewer)
• ❌ Gate result — Blocking (Repo-configured hard blocker found.)

Context & advisory signals — never blocks the verdict

Signal	Result	Evidence
Linked issue	✅ Linked	#8083
Related work	⚠️ Same linked issue: #8093, #8095	Another open PR references the same linked issue.
Change scope	❌ 8/20	High review scope from cached public metadata (1 linked issue).
Validation posture	✅ 25/25	PR body includes validation/test evidence.
Contributor workload	✅ 10/10	Author activity: 46 registered-repo PR(s), 15 merged, 33 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	cleanjunc; Gittensor profile; 46 PR(s), 33 issue(s).
Improvement	✅ Minor	risk: low · value: minor · LLM: moderate

Linked issue satisfaction

Addressed
The PR adds backtest-corpus.ts with the exact BacktestCase type and buildBacktestCorpus function implementing the specified nearest-following/fallback-to-most-recent pairing logic, correctly omits metadata when absent, filters by ruleId without modifying signal-tracking.ts, adds the barrel export in the exact required location, and includes the mandated test file plus a duplicate root-side test su

Review context

• Author: cleanjunc
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository is configured for direct PR review.
• Public profile languages: Python, TypeScript, JavaScript, Scala
• Official Gittensor activity: 46 PR(s), 33 issue(s).
• Related work: Items reference the same linked issue calibration: pure BacktestCase corpus builder from RuleFiredEvent/HumanOverrideEvent pairs #8083. (PR #8095)
• Related work: Items reference the same linked issue calibration: pure BacktestCase corpus builder from RuleFiredEvent/HumanOverrideEvent pairs #8083. (PR #8093)
• Related work: Items reference the same linked issue calibration: pure BacktestCase corpus builder from RuleFiredEvent/HumanOverrideEvent pairs #8083. (PR #8095, PR #8093)

Contributor next steps

• Start here: Compare feat(calibration): pure BacktestCase corpus builder from fired/override events #8093, feat(engine): pure BacktestCase corpus builder from fired/override event pairs #8095.
• Then work through the remaining 2 steps in the Signals table above.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

• @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
• @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
• A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

• Re-run LoopOver review

[loopover-orb]

LoopOver is closing this pull request on the maintainer's behalf (Linked issue overlaps another open PR; duplicate of open PR #8093). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.