measure(perf): #700 lever 3 measured and rejected, and the ingest cost table made honest

[jasonssdev]

Closes #700.

#700 ranks five levers against a measured time budget and forbids building any of them before its measurement exists. Working it meant discovering that two of the five were already dead, and that the biggest available speed-up had been measured and had failed.

Where the five levers actually stand

Lever	Status
1. Generate less (two-phase)	Measured, REJECTED — #728
2. Fewer, larger windows	Measured, REJECTED — #699, and contraindicated by #714
3. Smaller models for classification	Measured here, NOT ADOPTED
4. Concurrency	Untouched — still needs its own measurement
5. OLLAMA_KEEP_ALIVE	Already documented

Lever 1 nearly halved wall clock (165.7s → 87.0s) and lost the quality band on both axes — writing the body turns out to be the brake on enumeration. Lever 2 dropped subject recall 0.80 → 0.69 with 1 run in 8 erroring, and #714/#721 then moved _CHUNK_THRESHOLD the other way for transcripts, because a large window hits the 8192 generation ceiling and fails the ingest outright. So the inference-bound half has no cheap win left, and this PR records that where the next person will look instead of leaving it to be rediscovered.

Lever 3 — measured, not adopted

qwen2.5:3b (1.9 GB) against the default, 15 runs per arm, same machine and session, both harnesses driving live production code:

harness	model	quality	stability	latency
contradictions	qwen3:8b	FP 0.19, TP 1.00, acc 0.92	0.97	29.9s
contradictions	qwen2.5:3b	FP 0.33, TP 1.00, acc 0.86	0.96	13.2s
edge_typing	qwen3:8b	acc 0.36	0.99	29.3s
edge_typing	qwen2.5:3b	acc 0.39	0.81	11.6s

2.3–2.5× faster on both, and not adopted on either. The contradiction judge nearly doubles the antonym false-positive rate #558 existed to reduce — and since #598 findings persist and are ranked in status/next, a false positive now costs attention repeatedly instead of dying with the process. Edge typing is not less accurate, but 0.81 modal share means it changes its answer on one edge in five; since #624 routes asymmetric types through per-item consent, that makes the consent queue itself differ between two runs over an unchanged bundle.

Neither was added to RECOMMENDED_TASK_MODELS either — that map means "measured best on this task's harness", which a speed-and-size trade is not. The models: seam already accepts it for anyone who wants that trade on constrained hardware.

Fifteen runs is the floor, and that is the method finding

At n=5 this same pair measured 0.28 vs 0.36 — a tidy, publishable, meaningless result, and I had already written the conclusion from it before checking. Measuring qwen3:8b against itself, identical settings minutes apart:

sample	runs	antonym FP
runs-baseline-20260816T084135Z-qwen3-8b.json	5	0.44
runs-baseline-20260816T084923Z-qwen3-8b.json	15	0.19

A 0.25 spread inside one arm, larger than the 0.14 gap between the two models. Edge typing likewise reversed its ranking between n=3 and n=15. Both samples are committed.

Both harnesses were measuring under conditions the product never runs

They built OllamaClient(model=...) with no generation ceiling and no context window: unbounded generation against the 600s transport deadline, and whatever window each model's own Modelfile ships — the 32K/10 GB footprint #691 pinned away. Now pinned to production's defaults and recorded in every stored run, because they are part of an arm's identity: an arm recorded before this change is not comparable with one recorded after it.

The user-facing half

#700 says a table of what to expect "is worth more to a new user than any of the optimisations above". It ships in docs/cli.md and docs/faq.md — with counts derived from the pipeline, because #700's own table was wrong: it claimed a six-page note costs 1 call, when the union path asks twice below the threshold and then judges, and it predated the lower transcript threshold.

Document	Pages	Prose	Transcript
Short note	2	3 calls	4 calls
Minutes	5	3 calls	6 calls
Transcript	15	13 calls	14 calls
Long document	100	78 calls	79 calls

test_documented_ingest_call_counts_match_the_pipeline reads that table out of the document and replays every row through the real extract_concept_union against a counting stub. Mutation-verified in both directions: moving _MEETING_CHUNK_THRESHOLD fails it, and editing a documented cell fails it. The two conditional calls (re-ask, judge-skip) are documented as not pinned rather than implied to be, and boundary page counts were removed — on those the branch was decided by the fixture's filler length rather than by the constants.

Two stale doc claims fixed: contradiction was documented as having no harness at all (it gained one in #558, and it takes --model), and the #516 edge-typing figures were presented as reproducible when the fixture has since grown 17 → 23 edges and the client settings changed underneath them.

Verification

4852 passed, 1 skipped · ruff check clean · ruff format --check clean · mypy clean on 217 source files. Production code is untouched; no model default moved.

Review

Lineage review-e95634f4c3e92351, medium risk, reliability lens, APPROVED — 3 WARNINGs and 1 SUGGESTION, none blocking. pre-commit and pre-pr both return continue.

Three earlier review rounds were run and acted on rather than filed: they caught that the guard's boundary rows were decided by fixture filler rather than constants, that the first version re-implemented the fan-out instead of observing it, that stored runs omitted the comparability axis this change itself declares load-bearing, and that the doc attributed the #516 gap to the fixture alone when two things had changed. Each was fixed before the candidate that carries the receipt.

Residual findings and one pre-existing defect the review surfaced — every stored evals/edge_typing/ run since 2026-08-09 has all-zero confidences, across every model — are filed as a follow-up rather than fixed here.