feat(miner): add structured reviewer-consensus calibration signal

[galuis116]

Summary

Adds a new pure engine module, packages/gittensory-engine/src/reviewer-consensus-calibration.ts, in the same opt-in calibration family as objective-anchor, pairwise-judge, gate-verdict, and finding-severity calibration. It gives the miner replay harness a structured, default-off signal for how well a review's verdicts agree across independent reviewers.

When a review runs more than one independent reviewer (multiple models, or the same model sampled multiple times), each reviewer casts a per-dimension verdict (pass / warn / fail). This module ingests, per replayed PR, the set of votes per gate dimension and scores the consensus:

• Per dimension, unrecognized/abstention votes are dropped, the rest are tallied, the plurality outcome is chosen (ties broken toward the more severe outcome so a genuine split never rounds a real fail/warn down to pass), and the agreement fraction is the plurality's share of the definite votes.
• The per-PR score is the vote-count-weighted mean of the per-dimension agreements, so a dimension reviewed by more reviewers carries more weight than one seen by a single reviewer.
• It composes with the objective-anchor and pairwise signals via computeReviewerConsensusCompositeCalibrationScore, renormalizing weights when a signal is absent.

It complements the pairwise judge (which measures the order-stability of a single judge) by measuring agreement across independent reviewers, and it is distinct from gate-verdict calibration (which scores the outcome) and finding-severity calibration (which scores per-tier confirmation). Exactly like the sibling signals, the type surface is deliberately narrow — repo/run ids and per-dimension votes only. No raw review text, secrets, trust scores, reward values, private rankings, or maintainer evidence is represented, and renderReviewerConsensusCalibrationAuditMarkdown Markdown-escapes and newline-collapses every caller-supplied id.

Public API (exported from the package barrel)

• resolveReviewerConsensusCalibrationConfig(manifest) — default-off opt-in from miner.calibration.shareStructuredReviewerConsensus (with a top-level calibration.* alias) plus an optional non-negative weight; malformed values fail closed with warnings.
• ingestReviewerConsensusCalibrationSignals(signals) — validates + normalizes opted-in signals, rejecting with a specific reason (not_opted_in / empty_dimensions / invalid_repo / invalid_run_id); aggregates repeated dimensions and normalizes dimension + vote aliases.
• computeReviewerConsensusCompositeCalibrationScore(input) — blends objective-anchor + pairwise + reviewer-consensus (numbers or the sibling score objects), renormalizing weights and falling back to objective-only when all weights are zero.
• renderReviewerConsensusCalibrationAuditMarkdown(result) — deterministic, public-safe Markdown.

This is deliberately default-off and safe to call at ingestion time. No app routes, deploy config, or existing modules change — only the new module, its barrel export, its README.md section, and its unit test.

_Resubmit of #3396 (auto-closed on one AI-reviewer blocker): the composite scorer's normalizeCompositeWeights substituted the default 45/35/20 blend on all-zero input weights, making the documented objective-only fallback unreachable so weights: {0,0,0} silently got the default blend. Fixed to preserve zero weights (the reviewer's suggested change), and the all-zero test now asserts the objective-only fallback (weights {1,0,0}, composite == the objective score) instead of a loose range.

No linked issue: it sits in the established calibration-signal family and is a self-contained, additive, default-off engine module whose rationale is fully described here (issue creation on this repo is collaborator-only).

Scope

• The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
• This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
• This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
• I linked an issue, or this is small enough that the summary explains why an issue is not needed.

Validation

• git diff --check
• npm run actionlint
• npm run typecheck
• npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
• npm run test:workers
• npm run build:mcp
• npm run test:mcp-pack
• npm run ui:openapi:check
• npm run ui:lint
• npm run ui:typecheck
• npm run ui:build
• npm audit --audit-level=moderate
• New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

• The full npm run test:ci aggregate ran green (including build:miner, which compiles the engine src/ — the new module and barrel export type-check clean — and typecheck) plus npm audit --audit-level=moderate (0 vulnerabilities). A dedicated node:test suite (packages/gittensory-engine/test/reviewer-consensus-calibration.test.ts, mirroring the sibling calibration tests) covers config resolution, ingestion/rejection reasons, dimension + vote normalization, plurality tie-breaking, agreement weighting, composite weight renormalization, and the Markdown renderer. The module lives under packages/ (outside the root src/** Codecov scope), so the 99% patch rule does not apply to it.

Safety

• No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
• Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
• Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
• API/OpenAPI/MCP behavior is updated and tested where needed.
• UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
• Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
• Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

The module's type surface is deliberately private-data-free (repo/run ids + per-dimension votes only), it is default-off (ingestion requires an explicit shareStructuredReviewerConsensus: true opt-in), and the renderer escapes and newline-collapses all caller-supplied ids. Pure and deterministic: no auth/CORS/session surface, no API/OpenAPI shape change, no UI.

Notes

• New file reviewer-consensus-calibration.ts + its barrel export + a README.md section + a node:test suite. No schema, migration, OpenAPI, wrangler, or generated-artifact impact.

[superagent-security]

Superagent didn't find any vulnerabilities or security issues in this PR.

[loopover-orb]

Warning

🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨

⏸️ Gittensory review result - manual review recommended

_{Review updated: 2026-07-05 07:01:51 UTC}

4 files · 1 AI reviewer · no blockers · readiness 80/100 · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This adds a self-contained reviewer-consensus calibration module, exports it through the engine barrel, and covers the main opt-in, ingestion, scoring, tie-break, weight renormalization, and markdown-rendering paths. The implementation is consistent with the sibling calibration pattern: it fails closed on opt-in, drops unknown votes/dimensions, normalizes absent component weights, and keeps rendered audit output escaped. I did not find a reachable correctness defect in the provided diff.

Nits — 5 non-blocking

• nit: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:198 treats an empty string weight as `0` because `Number(value.trim())` converts `""` to zero, which is surprising for a malformed config value that the README says should fall back to the default.
• nit: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:467 accepts prebuilt ingestion objects with only a shallow `{ accepted: array, rejected: array }` check, so plain JS callers can accidentally pass malformed accepted rows and get `NaN` scoring instead of re-ingestion or rejection.
• nit: packages/gittensory-engine/src/reviewer-consensus-calibration.ts:481 uses `input.objectiveAnchor.score` and `input.pairwise.pairwiseJudgeScore` without rounding/clamping object-form scores, while number-form scores are rounded and clamped.
• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:198: change the string branch to reject blank strings before `Number(...)`, or tell me why blank config should intentionally mean zero weight.
• packages/gittensory-engine/src/reviewer-consensus-calibration.ts:467: either validate accepted ingestion rows before averaging or keep the public path narrower by only accepting raw signal inputs unless there is a strong caller need for prebuilt ingestion.

Signal	Result	Evidence
Code review	✅ No blockers	1 reviewer
Linked issue	✅ No-issue rationale	PR body explains why no issue is linked.
Related work	✅ No active overlap found	No same-issue or scoped active PR overlap found.
Change scope	✅ 20/20	Low review scope from cached public metadata (no linked issue context).
Validation posture	❌ 5/25	Preflight is holding this PR: the review lane is unavailable, so it is not ready for automated review.
Contributor workload	✅ 10/10	Author activity: 1888 registered-repo PR(s), 1249 merged, 59 issue(s).
Contributor context	✅ Confirmed Gittensor contributor	galuis116; Gittensor profile; 1888 PR(s), 59 issue(s).
Gate result	✅ Passing	No configured blocker found.

Review context

• Author: galuis116
• Role context: outside_contributor
• Public audience mode: oss maintainer
• Lane context: Repository registration is not available in the local Gittensory cache.
• Public profile languages: not available
• Official Gittensor activity: 1888 PR(s), 59 issue(s).
• PR-specific overlap: none found.

Contributor next steps

• Await review-lane availability.
• Refresh registry data or choose a registered active repo.
• Link the issue being solved, or explicitly explain why this is a no-issue PR.

Signal definitions

• Related work = same linked issue, overlapping active PRs, or title/path similarity.
• Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
• Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
• Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
• Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.

_{🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed}

---

💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by Gittensory, a quiet PR intelligence layer for OSS maintainers.

• Re-run Gittensory review

[loopover-orb]

Gittensory approves — the gate is satisfied and CI is green.