feat(crf): decode, match and report — the generic half of evaluation

[jayhesselberth]

The last leech-side piece of the CRF chain. leech.crf.evaluate takes a
reference set, a grouping and a corpus; what a panel is — which classes exist,
which share a flowcell — stays with whatever defines the panel. Same seam as the
manifest: vocabulary on one side, signal ML on the other.

Three rules it holds rather than leaving to callers

Each produces a plausible-looking wrong number rather than an error, which is
why they belong in the library:

rule	what goes wrong without it
emitted_references applies target[state_len:] once	full-length matching forces state_len leading deletions into every alignment — inflating distances and compressing the margin, since an aligner puts them where they help most, discounting wrong references more than the right one
balanced_recall takes a grouping and raises on an empty one	with classes crossed with batch, a pooled table measures batch; and an empty grouping yields balanced_recall: null, which serializes fine and ships
recall averaged per class, not per read	a pooled accuracy over unbalanced classes is decided by the deepest class

The empty-grouping refusal is not hypothetical — escapepod-models'
crf_metrics.py documents having to guard exactly that downstream, because
registry.ship only checks that metrics carry a source.

The edit distance

lev_vs_refs scores one decode against the whole reference set at once — the
shape of every evaluation loop. The scalar form is R DP tables per read:
affordable on a 16k sample, not on a run.

Its vectorisation recovers the serial insertion term exactly as
j + cummin(tmp[k] - k). That identity is the only thing making the algebra
trustworthy, so it is asserted against the scalar implementation on 50 random
queries × 24 references rather than commented.

_lev_py keeps its own name deliberately: lev is edlib where edlib imports,
so a test written against lev would compare edlib with itself precisely on the
machines that have it. Ragged references fall back to the scalar path rather
than being padded to a common width, which would change the distances instead of
computing them.

Validated on production data

16 references, emitted length 44 from 48
decoded 4000 reads in 53s, mean len 42.0
LDX1-8:  8 classes, 1800 reads
LDX9-16: 8 classes, 2200 reads

Eight classes per flowcell is the pilot's code↔flowcell confound — exactly why
pooling would be wrong, and why the API makes the grouping explicit. (The
balanced recall from that run is the 2-epoch smoke model's, not a quality
claim; the shipped model reaches ~0.97 after 32. What this establishes is the
mechanics.)

Testing

• 25 tests; full suite 1494 passed, 44 skipped
• ruff format --check, ruff check, ty check, zensical build clean

Where this leaves the goal

With escapepod-models#83 (its crf/ deleted, 161 tests passing against leech
0.8.0), the CRF chain is out of that repo bar the pieces that belong to it: the
panels, the corpus extractor, the bundle, the DAG, and the bonito oracle.

🤖 Generated with Claude Code