Repository navigation
2.4.1
What's Changed
The judge's identity is now in its provenance pin.
Shipped as a patch. Both changes below alter a published value, so the 1.0.0 policy would call this a minor at least; it goes out as 2.4.1 as a deliberate exception, because the old pin was answering a provenance question wrongly and the sooner it stops the fewer runs are affected.
Fixed
-
Scorecard.judge_versionnow includes the judge's model:anthropic:claude-sonnet-4-6@v1rather thanjudge@v1.It was
key@judge_version. Swapping the judge's model while leaving the declaredjudge_versionalone produced a byte-identical pin, so any provenance check reading that field passed a comparison against a baseline scored by a different model:before the swap: judge@v1 after the swap: judge@v1 ← same pin, different judgeNow:
before the swap: anthropic:claude-sonnet-4-6@v1 after the swap: anthropic:claude-opus-4-1@v1The model is the thing most likely to change and the thing a declared version is most likely to miss.
Changed
- A single judge's key defaults to its provider (
anthropic) instead of the literaljudge. A panel already defaulted tokey or provider, so the one-judge case was the odd one out, andjudgenamed nothing thegradercolumn did not already say. It appears in thejudges.namecolumn.
Upgrading: every suite with a judge re-baselines once, because judge_version is part of what identifies a comparable run — the intended behaviour, just paid all at once. Queries filtering judges.name = 'judge' need the provider instead. Set key: on a judges: entry to pin a name of your own.