v0.3.0 — Scale study, fair baselines, portability
The post-campaign studies are complete — run on a Slurm GPU cluster (RTX 5000/6000 Ada, H100) against the same frozen splits and a byte-identically rebuilt canonical store (tgms replay, new in this release).
The scale study (frozen CollegeMsg test, 94 tasks)
| model | TGMS EM | TGMS probes | best baseline | TGMS UCR |
|---|---|---|---|---|
| Qwen2.5-7B fp16 | 0.138 | 0.38 | 0.096 | 0.000 |
| Qwen2.5-14B fp16 | 0.340 | 0.77 | 0.191 | 0.000 |
| Qwen2.5-32B fp16 | 0.628 | 1.000 | 0.277 | 0.000 |
| Qwen2.5-72B AWQ | 0.511 | 0.31 | 0.138 | 0.000 |
- The operator-backed advantage grows with model capability (4.6× from 7B to 32B) while retrieval/text-to-query baselines stay flat.
- Belief-state probes saturate at 1.000 at 32B.
- The 72B-AWQ regression isolates quantization, not scale, as the planning bottleneck.
Fair-baseline study
Vector-RAG at its intended k=20 retrieval breadth (native 32k window, H100, 14B fp16) scores 0.021 vs ours 0.362 in the same run — worse than the k=1 configuration it was supposedly hobbled by, at 32.5k vs 6.5k tokens/task. Un-hobbling the baseline made it worse: the failure is representational, not budgetary.
Portability
Llama-3.1-8B 0.043, Phi-4-mini 0.015 with an untuned manual — plan competence needs a capability threshold; the unsupported-claim rate is 0.000 in every cell ever measured (4 scales, 3 families, 2 quantizations, 2 clusters). Safety is architecture-invariant.
New in the code
tgms replay+ vaulted canonical store (benchmarks/frozen-v1/) — byte-identical store reproduction (D-023).- Hard-bounded text-to-Cypher execution (killable child processes), bounded Kùzu buffer pool, HF-offline serving, Slurm serve+eval tooling.
- Decisions D-022–D-024; full details in docs/TECHNICAL_REPORT.md §8.2c.