v0.30.0-beta — Two new verticals, and one shape that cannot discriminate

Nine verticals, 470 questions. Two new ones, both built controls-first with a measured baseline before any judge template was written.

TypedMemEval-Conjunction — questions no single memory type can answer

A fact of type A must be resolved and an operation of type B applied to it. Retrieving either half is necessary and neither is sufficient, so a stack strong on one type and weak on the other scores like a stack weak on both — which is what a per-type score cannot show.

shape	join	n
value-then-count	Semantic current-value + Arithmetic count	20
alias-then-count	Semantic co-reference + Arithmetic count	15
order-then-value	Temporal order + Semantic current-value	15

V1 49/50   V2 50/50   V3 49/50   V8 50/50   V9 18/50   headroom 0.62

Read the shapes, never the mean:

shape	V9	headroom
alias-then-count	1/15	0.93
value-then-count	2/20	0.85
order-then-value	15/15	0.00

order-then-value is saturated under BM25 and cannot discriminate retrievers at all. The 0.62 mean is carried entirely by the other two — the mean-satisfiable-by-averaging defect one level up, at headroom rather than coverage, declared in the corpus rather than left inside an average.

The first vertical with genuinely mixed gold (35 arithmetic+semantic, 15 semantic+temporal), so a per-type denominator is computable. ADR-027 §10's instruction not to inherit the parts' certifications proved load-bearing: its own V7 caught two gold-only constructions the parent verticals' passes would have papered over.

TypedMemEval-Semantic — resolution, not recall

§2.1 refused plain-fact Semantic as saturated by construction. These three shapes share what plain recall lacks — retrieving the evidence is necessary and not sufficient: current-value (an attribute replaced k times), co-reference (a fact asked under a different designation), source-attribution (which conversation a belief came from).

V1 50/50   V9 34/50   headroom 0.32

It ships with no judge body, and that is the finding — 0.958 across three runs on the shared preamble alone. Bitemporal and Temporal each needed one because each genuinely collided with the preamble; Semantic does not collide.

The judge now reports the route it took

Checking only the final label cannot distinguish a template that discriminates properly from one that suppresses a label outright. question_asks is emitted in the judge's JSON at zero extra calls and asserted independently of the outcome, so reaching the right answer by the wrong route fails the build.

Alongside it: a per-vertical judge floor of 0.80 on both the recorded result and the live arm — a family-wide threshold is satisfiable by averaging, which is how Bitemporal once sat at 0.750 behind a green 0.946. Falsification-verified.

Family agreement 0.991 over 224 calibration cases, eight of nine verticals at 1.000.

Also

• Temporal gained a judge body — the second and last vertical falling through to StandardBody. Measured against a baseline taken before the body existed.
• bench typedmemeval --evidence-detail references|content — Full capture was unreachable because the command passed options: null unconditionally. Every layer beneath was correct; the caller never set it.
• tools/validate_factgrain_axis.py — the fact-grain difficulty axis cannot be validated against our own arms: the only measure that predicts V9 misses is derived from the same BM25 that V9 is. Committed as a runnable script so the negative result is reproducible.
• Per-item gold type labels in the sidecar — corpus_sha256 covers the whole corpus JSON, so putting them in the extension would have invalidated every probe record.

Verifiable without spending anything

Only semantic (de0b1c22521198de) and conjunction (c62ef477359749c1) are new bytes. Every other corpus is byte-identical to 0.29.0-beta, including bitemporal (f5b384d7f0ff9c0f). Empty rate 0.0 on every arm of every vertical, and probed_corpus_sha256 == corpus_sha256 throughout.

Full changelog: https://github.com/AgentEvalHQ/AgentEval/blob/v0.30.0-beta/CHANGELOG.md