v0.3.1 — Cited evidence was half a sentence
Two things: a correctness fix worth upgrading for, and a warning that finally tells you what to
do about it.
Nothing covered by semantic versioning changed — no specjudge.api symbol, no schema field, no
exit code. Upgrading needs no action.
Cited evidence was half a sentence
Since 0.2.0 the judge has had to cite the fragment of your spec behind each rating. It turns out
it was being shown mutilated fragments.
Bullets were matched line by line, so any requirement that wrapped lost everything after the
wrap:
your spec: - **FR-002**: Report generation MUST run entirely on the user's device, since no
personal data may leave it.
the judge: "**FR-002**: Report generation MUST run entirely on the user's device, since no"
In the evaluation corpus, 140 of 160 fragments were affected. Most real specs wrap their
requirements, so most real projects have been judged on half-requirements and shown
half-sentences as evidence for two releases.
Fixed. Measured on a real 24B judge, accuracy went from 19/19 to 20/20 with one more dimension
answered — the tool was not only quoting less, it was judging worse.
A thin spec now tells you what is missing
Reported by @pete-lewis83, whose diagnosis was the useful part:
the problem isn't that the warning is missing, it's that the podium prints right underneath
it and wins.
Exactly right. A one-line caveat above a ranked table with medals and prices is not a fair
fight. So the caveat moved to where reading actually ends, and it stopped being vague:
This ranking rests on a thin definition. Before acting on it:
• the spec declares no numbered requirements (looked for FR-NNN)
→ state what the system must do, one numbered requirement per behaviour
• no acceptance criteria found (looked for SC-NNN or NFR-NNN)
→ say how you will know it works — the measurable bar each requirement meets
Each line names what it searched for. These are heuristics over Spec Kit conventions and they
will sometimes be wrong; a project that numbers its criteria differently deserves to be able to
dismiss the warning in a second rather than be accused of not having written them.
The HTML report had the same problem and gets the same fix.
Also
The judge is now asked for the fragment most relevant to a dimension rather than one that
proves the rating. Demanding proof made any whole-project dimension unanswerable — every
over-abstention measured across two judges was domain_specialization, which no single sentence
supports. That halved.
One thing that did not get fixed, stated plainly: the judge still does not notice when a
spec contradicts itself. Two prompt formulations were tried and produced identical results. It
is documented in #19 with what was measured and the two untried levers, and the evaluation
corpus keeps expecting an abstention there, so every run keeps reporting the gap until somebody
closes it.
Full changelog: https://github.com/JoaquinRuiz/SpecJudge/blob/main/CHANGELOG.md#031---2026-08-07