SynthWorld 0.12.0
Changed
-
Breaking (ambiguity answer key):
same_name_and_date_of_birthis now
insufficient, notseparate(#77). The pair is two people in canonical truth, but
the public evidence — matching name and birth date, nothing distinguishing them —
cannot justify concluding it. The pack's first consumer abstained on exactly this pair
and independently gave the same reason. Membership truth is unchanged; the public and
memberships artifacts are byte-identical, and only the dispositions artifact was
re-cut (new digest inGOLDEN_REVIEW.md). A resolver scored against the old key that
answeredseparatehere was being rewarded for clairvoyance; one that abstains is now
scored correctly. -
EVALUATION_SCHEMA_VERSIONis0.2.0.TaskMetricgains optionalfamilyand
support_meaning, so every task's report carries two more keys - extraction,
entity resolution, relationship inference and risk included, even though none of
their metrics changed meaning. The wire shape is what moved, so the schema knob is
what moves; no per-task scoring version changes, because a scoring version here means
the metric definitions and those are untouched. A stored0.1.0report does not
load under the new model — the report'sschema_versionis a single-value literal —
so read archived reports with the library version that wrote them. -
Agentic metrics are grouped into five families and every denominator says what it
counts, so a report can be read by family and each ratio re-derived rather than
trusted. No metric value moves. The split carrying the most information is
observabilityagainst the rest: an agent can decide well and record badly, or the
reverse, and those need different fixes. Measured on the reference trace, wrecking
the recording drops observability to 0.25 while identity resolution, authorization
and delegation stay at 1.0; wrecking the decisions drops authorization to 0.40 while
observability stays at 1.0. -
AGENTIC_BENCHMARK.mdgains a per-metric glossary: what each measures, what 0.0 and
1.0 mean, and its denominator. Sixteen of the twenty metrics had no mention in any
top-level document, so the only way to learn what they measured was to read the
scorer or diff scores between policies. (They were cited in
agent-authority-contract/control-catalogue.yamland its design-intent notes, which
a first version of this entry overlooked while claiming a repository-wide count.)
What's Changed
- Correct the 0.11.0 release notes: nine of eleven, not eleven by @bluntmachetti in #70
- Give agentic metrics families, denominators, and a glossary (#71) by @bluntmachetti in #72
- Create the GitHub release from the tag, not by hand by @bluntmachetti in #74
- Derive ambiguity dispositions from evidence instead of hand-authoring them (#62) by @bluntmachetti in #75
- Stop disposition_of concluding things a resolver would not (#81, #85) by @bluntmachetti in #87
- Stop the third disposition class starving by @bluntmachetti in #88
- Stop rewarding clairvoyance on same_name_and_date_of_birth (#77) by @bluntmachetti in #89
- Tranche closeout: honest v2 framing, v1 result sizing, key custody by @bluntmachetti in #93
- Prepare SynthWorld 0.12.0 by @bluntmachetti in #94
Full Changelog: v0.11.0...v0.12.0