v0.6.2 — standards eval + honest corrections
A standards-anchored evaluation (ISO/IEC 25010:2023, ISO/IEC/IEEE 29119/12207/1012/29148, SWEBOK v4 — docs/evals/standards/) graded the product honestly: not a complete, professionally-tested product — an unusually honest v0 with real strengths and real, named gaps. Its anti-rubber-stamp critic found concrete defects; all were verified by hand and fixed:
- Hash-chain honesty (the key correction). The
verdict.loghash chain is tamper-EVIDENT (catches accidental corruption + any append outsidelog_verdict.sh→INTEGRITY-COMPROMISED) but NOT forgery-proof — it is an unkeyed SHA-256 over public scripts, so an adversary who reproduces the formula can forge a validDONE(shown in ~15 lines). Earlier "DONE can't be forged" wording overstated it and is corrected. Real forgery-proofing needs an out-of-band trust root the agent can't reach — the deferred R13 signed manifest + external CI (a local HMAC key would be theater: a shell agent can read any key the harness can). - pre-commit consistency (real bug): its inlined logic had drifted from
verdict_state.pyand wrongly blocked a legitimate red→greenDONE. Now identical latest-iteration rule. - All 6 terminal states now reachable:
STUCK-OSCILLATING/STUCK-INCONCLUSIVEwere defined but never emitted;run_gatesnow emits both (an all-skipped run is no longer a false green). Proven by new scenarios — self-eval suite 11 → 13 green. - Re-freeze in the same repo (mode-444 lock) fixed; R1 sole-writer CI lint added (was claimed, didn't exist); CONTRIBUTING de-staled.
Still open, now clearly named: no unit tests / no coverage number for the harness's own 15 scripts (biggest test-adequacy gap); verifier independence is architectural, not mechanically enforced; DONE needs gate evidence but not a logged verifier-CONFIRMED row; forgery-proofing awaits R13.
Full assessment: docs/evals/standards/COMPLETENESS-VERDICT.md.