Context
Sub-issue of #9216 (frozen-repo benchmark harness), epic #8534 . Requirement 5 of that issue is explicit: leaderboard artifacts are exactly the eval-interface's consumable score records — no parallel format. This issue emits them.
Requirements
Emit EvalScoreRecord as specified on Spec: validator-facing eval interface — what SN74 consumes from LoopOver as the objective eval provider #9215 — workUnit.kind === "benchmark_run", carrying benchmarkId and snapshotRef. No benchmark-specific record shape, no extra top-level fields.
Commitments block must be fully populated : snapshotChecksum (benchmark: frozen-repo snapshot builder — leak-proof repo state at commit T #9259 ), scoringRuleVersion, window bounds, splitSeed and heldOutFraction (benchmark: anti-overfit controls — seeded held-out, per-repo splits, submission caps, rotation #9263 ). A record whose commitments do not let a third party re-derive the number is not publishable — that is the entire point of the block.
Trust tier from the actual execution (benchmark: untrusted candidate-agent execution — sandbox, limits, attested when hardware allows #9264 ), never asserted optimistically: attested only with a real envelope, otherwise reproducible.
Held-out scores obey benchmark: anti-overfit controls — seeded held-out, per-repo splits, submission caps, rotation #9263 's publication cadence — the emitter is where that policy is enforced, so a leaderboard cannot accidentally leak held-out membership by publishing per-submission.
Digest-committed and immutable : recordDigest over canonical JSON; a corrected score is a new record with a later issuedAt, never a mutation.
Deliverables
Expected outcome
The leaderboard and the validator-consumable score feed are the same data, so a number on the board is a number a third party can verify.
References
Sub-issue of #9216 . Record shape: #9215 . Depends on #9259 , #9262 , #9263 , and the serving endpoint issue. Epic #8534 .
Context
Sub-issue of #9216 (frozen-repo benchmark harness), epic #8534. Requirement 5 of that issue is explicit: leaderboard artifacts are exactly the eval-interface's consumable score records — no parallel format. This issue emits them.
Requirements
EvalScoreRecordas specified on Spec: validator-facing eval interface — what SN74 consumes from LoopOver as the objective eval provider #9215 —workUnit.kind === "benchmark_run", carryingbenchmarkIdandsnapshotRef. No benchmark-specific record shape, no extra top-level fields.snapshotChecksum(benchmark: frozen-repo snapshot builder — leak-proof repo state at commit T #9259),scoringRuleVersion, window bounds,splitSeedandheldOutFraction(benchmark: anti-overfit controls — seeded held-out, per-repo splits, submission caps, rotation #9263). A record whose commitments do not let a third party re-derive the number is not publishable — that is the entire point of the block.attestedonly with a real envelope, otherwisereproducible.recordDigestover canonical JSON; a corrected score is a new record with a laterissuedAt, never a mutation.Deliverables
EvalScoreRecordsExpected outcome
The leaderboard and the validator-consumable score feed are the same data, so a number on the board is a number a third party can verify.
References
Sub-issue of #9216. Record shape: #9215. Depends on #9259, #9262, #9263, and the serving endpoint issue. Epic #8534.