v0.31.0-beta — Three fixes, and five defects it declares rather than hides
Pre-releaseNine verticals, 470 questions. Three fixes, five instruments — and a changelog that names five defects this release does not fix.
Temporal recency could not rank anything
It scored 15/15 at V1, V8 and V9 — the only shape in the family on which no two systems could be told apart. It asked about the last three events in the chain, so gold was two adjacent links and the answer was a single transitive step over two sessions that named the asked events outright.
Guessability was ruled out before the construct was touched: the answer was first-named 6/15, middle 3/15, last 6/15, all at chance. The shape was sound and simply too easy on both halves.
It now asks about events spanning the chain.
| before | after | |
|---|---|---|
recency V9 |
15/15 | 6/15 |
| vertical V9 | 42/50 | 30/50 |
| headroom | 0.16 | 0.40 |
Gold grows from 2 links to count − 1 and stays minimal: on A<B<C<D<E asked over {A, C, E}, dropping any intermediate link removes a transitive step the answer needs.
The whole vertical was redrawn. A seeded generator redraws everything downstream of any change to the draw sequence, so all 50 questions differ — not the 15 that were targeted. Question ids are stable across that redraw, which makes an id-keyed join succeed while comparing different questions. Temporal sha a6c10b3d… → 31d26e60…; controls reset.
Prospective: firing semantics are now required, not optional
The consuming project ran the corpus with ProspectiveFiring and ValidTime=Current both dark and scored 49/50. Every shape named the thing, which hands a similarity retriever the words of the session it needs while the harness supplies "today" and the corpus supplies the due date — in-context arithmetic no memory feature is needed for.
The new due-window shape names nothing: several reminders whose only distinguishing property is when each falls due, and an answer that is a set whose membership changes with the as-of instant.
| before | after | |
|---|---|---|
| V8 interference cost | 0.00 | 0.28 — the family's first real one |
| V9 | 34/50 | 23/50 |
| headroom | 0.32 | 0.54 |
Prospective sha 6ddd3e9b… → a570b890…; also a full redraw, same id caveat.
Semantic gained a judge body — and the shared preamble did not change
Semantic shipped without a body deliberately, but that left it the only vertical with nothing to settle the preamble's abstained/missed line, which states the distinction as uncertainty versus denial and then illustrates abstained with "I have no record of that" — a denial.
Editing the preamble was tried, measured and reverted: it converted the family's canonical genuine-refusal cases to Missed across four verticals and dropped agreement 0.983 → 0.966. The rule that survives is a vertical body supersedes the shared preamble.
All 26 Semantic cases were enumerated before a line of the body was written, and 19 carry a declared route asserted independently of the outcome — so a template reaching the right label by the wrong reasoning fails rather than passing quietly. Semantic is now 26/26 in all three runs; family agreement 0.987 (0.991 / 0.987 / 0.996).
What this release admits about itself
Five defects ship declared and unfixed, in the changelog, with their directions stated. A release that carries wrong published numbers silently is the failure mode these corpora exist to catch.
V2 cannot judge closed-choice questions. Its reject line is an observed rate of 0.20; the chance floor is 0.50 at k=2. 71 of 470 questions enumerate their own alternatives, and 69 carry a V2 pass that records the reference model abstaining, not the corpus being non-inferable. A pure guesser is rejected with probability 0.989. Flattering direction.
The same floor sits under published arm counts. Chance-corrected: prospective V9 14/21 (0.67) → 0.33, temporal 24/35 (0.69) → 0.45, episodic 14/15 → 0.87. Headroom is a difference and largely cancels; the absolute counts do not.
Forgetting's coverage is diluted. Published mean_realised 0.670; over questions that have gold, 0.529, against a band floor of 0.50. Its 15 never-known probes are G=0 by design and realised_coverage returns a constant 1.0 for them, so 30% of the published mean measures nothing — and the echo calibration optimised that. Queued as a declared corpus revision.
Two shapes are out of band and were declared nowhere: episodic/list-order 0.275 (a reference retriever surfaces a quarter of its gold, so V9 there measures retrieval failure rather than reasoning) and forgetting/still-valid 0.467.
85.7% of the family by character is equalisation padding, uniform across all nine (82.4%–88.5%). Measured to be load-bearing, not waste — removing it puts 12 features over the 0.75 separability bar where 0 are now — so no redesign is proposed. Arithmetic additionally carries ledger voice (a bare common-noun subject carrying a value directly) at 77.5% of value-bearing sentences and 0.0% everywhere else.
Five instruments, none of which costs a model call
| tool | what it answers |
|---|---|
validate_v2_chance_floor.py |
which passes a coin-flipper would also have earned |
measure_signal_density.py |
what fraction of the text is content at all |
measure_padding_value.py |
whether the scaffolding earns its keep |
validate_coverage_population.py |
whether a published mean counts items it cannot measure |
make_padding_free_variant.py |
controlled diagnostic cells for a consumer's extractor |
Corpus verification
| vertical | sha256 | |
|---|---|---|
| arithmetic | 2feda94be7e8c026 |
unmoved |
| bitemporal | f5b384d7f0ff9c0f |
unmoved |
| conjunction | c62ef477359749c1 |
unmoved |
| episodic | 2c6000a6912e5ef0 |
unmoved |
| forgetting | ba759097b9bd1fd3 |
unmoved |
| prospective | a570b890a5b92c3b |
moved |
| semantic | de0b1c22521198de |
unmoved |
| temporal | 31d26e60fd21d00e |
moved |
| workingmemory | 7e04e4cb1717cbbf |
unmoved |
All nine: probes.status = run, probed_corpus_sha256 == shipped sha, reference deployment gpt-5.5. Judge prompt fingerprint b1d3f7216032ba27…, calibration 230 cases over 3 runs.
Independently verified read-only by the consuming project's pre-tag probe before this tag was cut.