v0.2.0: tracer scan, held-out parity certification, and an OOD safety gate

TRACER 0.2.0 turns "could we route this cheaper?" into a one-command answer, and tightens how the router certifies coverage.

Highlights

tracer scan — the day-one read

Point it at your existing LLM traces and get a fast, conservative verdict before training anything: how much of your traffic a near-free model can already answer at your target agreement, measured on a held-out slice it never saw (exact Clopper-Pearson bounds, not in-sample), with an estimated saving. Ships a self-contained HTML report with an interactive 3D map of your embedding space (hover-to-inspect cells, a Verdict/Label colour toggle, PCA/UMAP/t-SNE layouts).

pip install tracer-llm
tracer scan your_traces.jsonl --html scan.html

Embeddings are local by default (sentence-transformers), or bring your own: a precomputed .npy (--embeddings) or your own HTTP endpoint (--embed-url).

Held-out parity certification

The parity gate now certifies coverage on an exact held-out lower bound instead of an in-sample point estimate, so reported coverage holds up on real traffic at your target. Coverage is monotonic in the target you set, and a hybrid select-then-verify step recovers coverage at strict targets.

OOD safety gate

At inference the router now defers inputs that fall far from the training distribution (kNN distance) to the teacher, instead of returning a confident answer on off-distribution or adversarial input.

Lean by default

tracer fit now runs a fast linear + MLP zoo by default; add --trees for the heavier tree models on hard, high-class-count tasks, and --skip to drop named candidates.

Install / upgrade

pip install -U tracer-llm

Full notes in CHANGELOG.md.