v0.20.0

Performance

• The CPU boundary CNN loads with its input shape pinned and its
  convolution padding hoisted. Profiling demux --method cnn on the CPU
  put 38% of all CPU time in tract's padded_valid_x_loop, the per-element
  im2col path it falls back to when pads != 0 — the defect
  escapepod-classify already works around for the charging network. The
  loader also still optimized the graph with length symbolic, a choice
  #187 made obsolete when prep started padding every read to one shape.
  Measured on 119,255 RNA004 reads at 32 threads: 600 → 401 CPU-seconds from
  the input fact alone, 380 with the hoist; sorted per-read classifications
  identical. The rewrite now lives in escapepod_demux::onnx_rewrite and the
  CTC-CRF encoder loader applies it too. Pinning batch 1 means the CPU batch
  entry point runs reads one at a time, which is what CPU batching measured
  against anyway.

• Per-read work on the LLR path is bounded. The fingerprint is computed
  from at most the last MAX_FINGERPRINT_WINDOW (30,000) samples of the
  adapter window, and the LLR detector is handed at most LLR_DECODE_BOUND
  (200,000) samples of a read, in the fused pipeline and in demux detect
  and demux train. Neither touches a real read: adapter windows on a
  MinKNOW RNA004 run sat at 4,070 (median) / 5,120 (p90) samples, and 98% of
  its reads are under the decode bound. What they bound is the read-length
  tail: 2.7% of that run's reads (over 100k samples, up to 11.3M) held 62%
  of all samples, LLR reported "adapter" windows up to 4.6M samples on
  stalled pores, and the pipeline spent its last 6 of 11 seconds on one
  rayon worker fingerprinting one such chunk while the pool idled. With the
  reads the caps target filtered out, wall fell 11.4 → 5.1 s and CPU 61 →
  31 s on node-local input. --method cnn was already bounded and is
  unaffected. ADAPTed's own LLR sees only max_obs_trace samples; adopting
  that would be the parity-improving choice but renormalizes most real reads
  and needs a validation run first.

Fixed

• DtwSvmModel::validate now checks everything the predictor indexes by
  class or pair: thresholds, prob_a/prob_b, the dual_coef row count,
  classes and label_mapper coverage, and n_classes >= 2. A short
  thresholds used to accept every call for the classes past its end; the
  others panicked inside a rayon worker on the first read. A WarpDemuX
  model's threshold_type must be kernel or ratio — anything else
  silently selected the ratio rule. GbmModel::validate refuses a tree with
  a back-edge, which recursed without end in GbmPredictor::new.
• demux train no longer skips an unreadable POD5 (or batch) and exits 0
  with a consensus built from whatever else was there — the #293 failure
  the other stages were cured of in #294.
• demux classify --window with --model is an error rather than silently
  ignored: a DTW-SVM model carries its own window, and the WarpDemuX and
  GBM heads use none. The documented example used it that way.
• demux classify's SVM and GBM status lines go through tracing to
  stderr like every other stage's, so -q silences them and a piped stdout
  stays clean.
• A reads-table row that fails to decode is warned about instead of
  vanishing from the output of every demux stage.
• compute_consensus_fingerprint breaks a tie between equally common
  fingerprint lengths deterministically (shortest wins); it used to fall to
  HashMap iteration order, the last non-determinism in demux train.
• The AVX2 and AVX-512 logsumexp kernels guard the -inf - -inf = NaN of
  an all-unreached lattice cell themselves. chain_head reduces through
  them over the first rows of every --ref-scores scan, and its result was
  correct only because ln8's max(x, MIN_POSITIVE) clamp happened to
  return its second operand on an unordered compare — the accident
  chain_tail's guard was written to stop depending on. Bit-identical for
  every reachable lane; pinned by chain_head_absorbs_unreached.
• The GPU CRF encoder remembers a failed --ref-scores panel upload instead
  of discarding the error and retrying it every batch, and the fused
  pipeline reports the fallback at the end of the run (the host scan it
  falls to is +57% wall, #297).
• GPU DTW-SVM classification returns an error when its producer thread dies
  or returns fewer results than queries; every caller zipped the result with
  its queries, so the missing reads silently dropped out of the output.

Removed

• linfa and linfa-svm from escapepod-demux's train feature. They
  were compiled for every --features train build and never imported; the
  SVM fit has been a labels-only stub since #152.

---

Which artifact

Artifact	Linkage	--gpu
…-x86_64-unknown-linux-musl.tar.gz	static (musl)	no
…-aarch64-unknown-linux-musl.tar.gz	static (musl)	no
…-x86_64-unknown-linux-gnu-gpu.tar.gz	dynamic, glibc ≥ 2.28	yes
…-x86_64-apple-darwin.tar.gz	dynamic	no
…-aarch64-apple-darwin.tar.gz	dynamic	no

The musl builds are the portable default and the right thing for
an unattended installer to fetch. The GPU paths cannot be static —
they dlopen their runtimes — so they ship in the single dynamically
linked …-linux-gnu-gpu artifact instead, built against glibc 2.28
(RHEL/Rocky/Alma 8+, Ubuntu 20.04+).

What …-linux-gnu-gpu needs at run time

Only when --gpu is actually requested — otherwise it behaves
exactly like the musl one. It expects a CUDA 12 runtime and
cuDNN 9, with an NVIDIA driver ≥ 535 (the DTW kernels target the
CUDA 12.2 driver API), and a CUDA-enabled libonnxruntime matching
the ort 2.0.0-rc.13 it links.

Rather than assembling that by hand, see
GPU acceleration,
which covers the pixi environment that supplies it and how to
confirm the CUDA execution provider actually loaded.

v0.19.0

Added

• A waveform bundle declares where its per-base sequence comes from
  (#312). waveform_model.preprocessing.reference_source is accepted, and a
  value this runtime does not assemble is refused at load, naming both what the
  bundle asked for and what escpod does.

  The bundle already carried motif_reference: "fasta" and a focus_rule
  saying to "find CCAGGC in the REFERENCE". Both are true, and both are about
  locating the anchor in reference coordinates, which the FASTA is fine for.
  Neither says where the bases come from — and every corpus built so far
  takes those from each read's MD tag, via pysam's
  get_reference_sequence(). Reading "the REFERENCE" the obvious way and
  slicing the FASTA is what this runtime did until #306, and it validated
  cleanly: 87 of 256 corpus chunks bit-identical against 256 of 256, with
  nothing raised on the other 169.

  It is not a difference that can be resolved upstream. The N in all 47
  records of the shipped panel is the 3'-terminal base of the 5' adapter's RNA
  tail CUGGN, ordered degenerate on purpose; over 1,055,660 reads no
  letter covers even 55% of that position, so substituting the modal base
  would be silently wrong for 45% of reads — strictly worse than an N, which
  is at least visibly unknown. The ambiguity is a permanent property of the
  input, so the runtime has to be right about its source permanently, and a
  declaration it can refuse is the only thing that makes that checkable.

  Absent means md, so every bundle already published stays readable —
  WaveformPreprocessing is deny_unknown_fields, which is also why the key
  has to be accepted here before escapepod-models can emit it
  (rnabioco/escapepod-models#109), the same sequencing
  barcode_crf_fdx4_rna004's signal.anchor used.

  A fixture now carries the ambiguity: trna_reference_ambiguous.fa is the
  fixture panel with the code put back where the real one has it. The existing
  reference has the ambiguity removed — 0 codes against the panel's 47 —
  so every fixture read took the resolved path and no golden here could ever
  have seen the difference, the same blind spot #306's own feature_set had.
  The new test pins that the ambiguity changes nothing end to end, and, in the
  same pass, that putting the same N into the MD-derived sequence does
  move the chunks — so the equality is evidence about the source rather than
  two paths that never differ.

• escpod classify runs a waveform_model charging bundle (#306).
  A third bundle variant, beside gbm and feature_model: it reads a signal
  window rather than a column vector — normalised current plus its k-mer
  residual, the sequence k-mer context scattered along the signal axis, and 12
  per-base dwell/level rows — and emits a single BCE logit. On the feature
  network's own training rows and test reads it is worth +0.0050 AUROC
  (n=1,387,667; 3 seeds x 2 geometries, 6/6 positive, paired sd 0.00008), and
  0.988 recall against 0.960 at the fnn's own FPR.

  The chunk assembly moved down into escapepod-signal
  (escapepod_signal::chunk) rather than being written a third time here. It
  was already implemented twice — leech's Python dataset and leech-core's Rust
  pipeline — and the failure mode is on the record: escapepod-classify
  reproduced a superseded feature definition for two months and its counted
  golden missed it, because all 19 fixture reads took the other branch. The
  module is generic in the way that matters: the anchor is a base index, the
  window is (left, right) samples, and the channels are a list the caller
  supplies, so two models that read the same twelve rows in a different order
  are two Vecs rather than two code paths.

  Nothing about those rows is hard-coded here. The bundle ships
  waveform_model.channels.{signal,features}.order — asked of the corpus
  builder at build time rather than transcribed — and the runtime resolves it,
  refusing a name it cannot compute or a length that disagrees with the count
  beside it. This is the one rule no shape check can catch: permute the rows
  and the tensor still has exactly the dimensions the graph wants, every read
  still scores, and the answers are wrong.

  Two further rules are cross-checked rather than assumed, because each fails
  silently. preprocessing.motif/motif_offset must agree with the anchor
  block — this variant anchors at motif +2, one base earlier than the
  feature-grid variants' +3, and inheriting the other offset places every
  window off-anchor and validates cleanly. And the graph's single logit is the
  logit of whichever class the bundle names: leech assigned its class
  integers at merge time and gave charged 0, so P(charged) is
  1 - sigmoid(logit) here, and reading it the obvious way inverts every call
  without erroring.

  The refinement refusal at load is narrowed rather than removed: the column
  variants still cannot reproduce a banded-DP pass, and are still refused for
  it; the windowed variant reproduces it from its declared parameters.

  The reference each read is scored against comes from its own MD tag, not
  from the FASTA. That is how the training corpus builds it (pysam's
  get_reference_sequence()), and the two disagree wherever the FASTA carries
  an ambiguity code — every reference in the shipped tRNA panel holds exactly
  one N, where the alignment recorded a concrete base. It is not a one-base
  difference: levels are looked up per 9-mer, so one unknown base makes nine
  consecutive k-mers unknown and leaves a run of zero levels the corpus does not
  have, which the banded DP then walks a different path through for the rest of
  the read. Slicing the FASTA instead cost 169 of 256 chunks their
  bit-exactness, with boundaries moving a sample throughout the read and the
  feature window 96 bases downstream wrong — while erroring on nothing. A read
  without an MD tag is refused (no_md_tag) rather than fallen back on, for
  that reason. The assembly is now bit-identical to the corpus on 256/256
  chunks, end to end within 3.3e-6 — the graph's own residual.

  It runs through tract, statically linked, like every other ONNX graph
  escpod runs — so it is in the default build and works from a stock release
  binary. That is true only from charging_tcn_rna004@v0.1.1 onward, and the
  reason is worth recording, because it is the second time this model family
  has hit it.

  The first export, @v0.1.0, could not go through tract at all: not for want
  of an op, but because tract 0.23's shape inference cannot close it. Measured
  five ways, and the failures fall into two independent causes, either of
  which alone is enough:

  • dynamo writes a value_info entry for all 667 intermediates with the batch
    axis as the symbol batch. A consumer that pins the batch — which every
    ONNX loader here does — cannot unify that, and tract dies at the first
    convolution with Sym(batch) vs Val(1). Every other graph escpod loads
    carries zero value_info; the legacy TorchScript exporter never wrote any.
  • adaptive_avg_pool1d(390 -> 11), which dynamo open-codes into a rank-8
    GatherND because the output size does not divide the input.

  It is not nn.MultiheadAttention, which this changelog, the module doc
  and escapepod-models#96 all named until somebody read the graph. The
  offending GatherND consumes relu_17, the last block of signal_tcn; its
  (11, 37) bool mask is a bin mask and its (11,) divisor is
  [36, 36, 37, 36, 37, …], that pool's bin widths. cross_attn exports as
  plain Mul/MatMul/Softmax/MatMul/Gemm, with no mask and no gather.
  #306's original suspect was right and its retraction was not.

  Neither onnx-simplifier (which folds away every Shape node) nor
  onnxruntime's own optimiser makes it loadable, so it could not be papered over
  at load time the way fnn::hoist_conv_padding papers over padded
  convolutions. This ran through onnxruntime via ort for exactly as long as
  that was true — which, ort being load-dynamic, meant a libonnxruntime.so
  on ORT_DYLIB_PATH and no way to run such a bundle from a static-musl release
  at all.

  The fix belonged in the export, and is the one this family already needed once:
  escapepod-models retracted "tract cannot run Resize" in July after finding
  the same shape-inference cause, an export fix costing one line and no retrain,
  and a 6x tract speedup for free. @v0.1.1 is that re-export (leech 0.10.0) —
  the pool written as one MatMul against a constant segment-mean matrix,
  value_info stripped, same weights, no retrain, evaluation bit-identical,
  479 -> 319 nodes and GatherND 2 -> 0. With it go the ort dependency, the
  classify-waveform feature, and the per-rayon-worker session pool that ort
  needed because Session::run takes &mut self. Re-measured from this side
  with escapepod-demux/examples/tract_dynamo_probe.rs, kept so the claim can
  be re-run against a later tract or a later export (its counts are tract's own
  after parsing, hence larger than the ONNX node counts):

  v0.1.0   669 nodes   analysis fails at node_GatherND_329 / node_index
  v0.1.1   471 nodes   optimized to 655, runs, output [1, 1]
  

  So an unloadable graph is now a bundle problem with a named fix and a
  build-time gate on it (escapepod-models#96 and #97), rather than a runtime
  gap: @v0.1.0 fails at load with tract's own analysis error and the file
  named. The lesson generalises, and is why that gate exists: onnxruntime loaded
  the broken graph perfectly, so the export's own torch round-trip was green
  throughout. "It exports and agrees with torch" is a weaker claim than "a
  runtime can load it".

  The swap is not free and the cost is worth stating: on the same harness and
  the same 256 chunks, tract is **6.27 ms/chunk again...

v0.18.1

Fixed

• signal classify no longer aborts when a tRNA body happens to contain the
  junction motif (#298). The CCA|adapter junction was located by requiring
  CCAGGC to occur exactly once per reference record, and that check ran
  before the common-arm check that actually identifies the junction. But the
  motif is just CCA plus the adapter's opening bases, and a 6-mer collides over
  ~75 nt of tRNA body in ~1.5% of records -- 4 of 282 in hg38, 51 of 3315 in
  danRer11. One such record aborted the entire run for that sample, and
  dropping the offenders is not neutral: 48 of danRer11's 51 are Glu, so it
  would remove most of one amino-acid family from a charging analysis.
  Candidates are now filtered by the arm first and uniqueness required of what
  survives, which resolves every record of both references to exactly one
  junction at the correct position. Two genuinely broken cases -- no arm-backed
  match, and several -- now report distinctly instead of being conflated with
  this benign one.

---

Which artifact

Artifact	Linkage	--gpu
…-x86_64-unknown-linux-musl.tar.gz	static (musl)	no
…-aarch64-unknown-linux-musl.tar.gz	static (musl)	no
…-x86_64-unknown-linux-gnu-gpu.tar.gz	dynamic, glibc ≥ 2.28	yes
…-x86_64-apple-darwin.tar.gz	dynamic	no
…-aarch64-apple-darwin.tar.gz	dynamic	no

The musl builds are the portable default and the right thing for
an unattended installer to fetch. The GPU paths cannot be static —
they dlopen their runtimes — so they ship in the single dynamically
linked …-linux-gnu-gpu artifact instead, built against glibc 2.28
(RHEL/Rocky/Alma 8+, Ubuntu 20.04+).

What …-linux-gnu-gpu needs at run time

Only when --gpu is actually requested — otherwise it behaves
exactly like the musl one. It expects a CUDA 12 runtime and
cuDNN 9, with an NVIDIA driver ≥ 535 (the DTW kernels target the
CUDA 12.2 driver API), and a CUDA-enabled libonnxruntime matching
the ort 2.0.0-rc.13 it links.

Rather than assembling that by hand, see
GPU acceleration,
which covers the pixi environment that supplies it and how to
confirm the CUDA execution provider actually loaded.

v0.18.0

Fixed

• The reads table is written in batches again, instead of one batch per file
  (#297). filter, merge, subset and split build the reads table with
  build_reads_table{,_remapped} rather than Writer, and those wrote the
  whole table as a single Arrow record batch however large it was — so
  read_batch_size on FilterOptions/MergeOptions/SubsetOptions was
  declared, defaulted, and never read. escpod filter asking for 10,000 rows
  per batch wrote 40,000 reads as one.

  Measured, for scale: MinKNOW writes ~10,000 reads per batch (1,575,748 in
  158) and the pod5 Python package exactly 1,000 (17,919,658 in 17,920).
  Neither writes a whole file as one batch.

  It is not cosmetic for anything that reads escpod's output back. demux
  shards its reader threads by batch index and only emits a block at a batch
  boundary, so a single-batch file is read by exactly one thread whatever
  ESCAPEPOD_DEMUX_FILLERS says, and nothing reaches the GPU until the entire
  file has been decoded. On a 100k-read escpod-written file that was 11.2 s of
  a 19 s stage with the card idle, and it made every tuning knob look flat.
  Fixing it took that stage to 11.3 s with the first block arriving in 0.3 s
  and GPU utilisation going from 22-30% to ~60%.

  Every default is now 1,000, and the five CLI sites that overrode it (merge at
  100,000, the rest at 10,000) inherit it, so the geometry has one definition.
  Note this never affected MinKNOW input, which was already many-batched:
  measured on a real 1.3M-read run, GPU utilisation is ~94% and the first block
  arrives in 0.4 s both before and after.

Build / Tooling

• The noodles crates are bumped as a set (#302): noodles-bam 0.94 -> 0.95,
  noodles-sam 0.89 -> 0.90, noodles-csi 0.60 -> 0.61. They share types across
  their own version boundary, so Dependabot's one-PR-per-crate bumps could not
  compile individually (E0308 in escapepod-signal, which uses bam and sam
  together). No source changes were needed.

Performance

• The CRF encoder's onnxruntime sessions no longer spin (#296). Each
  encoder worker built its CUDA session with an intra-op pool --threads wide
  and spinning enabled, so --threads 32 with two workers put 64 spinning
  intra-op threads on a 32-core allocation — for a graph that runs on the
  device and gives that pool almost nothing to compute. This is the same defect
  #240 fixed for the adapter-CNN session, which never reached this one.
  Measured on an A30 (demux basecall, 40k reads, interleaved): process CPU
  858% -> 733%, perf samples in libonnxruntime.so 15.6% -> below the 1%
  report floor, output byte-identical.

• demux --annotate interns its barcode labels (#296), writing the sidecar
  from a dictionary plus per-read codes rather than a string per read.

• The CRF encoder pool uses every visible GPU instead of reserving one for
  adapter detection (#297). Detection is ~5% of device time since #187, so
  holding a whole card for it left that card 12% busy on two GPUs and 27% on
  four while the lone encoder device pinned at 93%. Measured on a real 1.3M-read
  run, interleaved, 2 reps:

  GPUs   reserve device 0            encode everywhere        vs 1 GPU
    1    162.5 s  (gpu0 88%)         same policy              --
    2    136.6 s  (12% / 93%)         80.8 s  (88% / 88%)     1.19x -> 2.01x
    4     56.7 s  (27% / 75%)         55.3 s  (72% / 58-66%)  2.86x -> 2.94x
  

  Two GPUs go from 1.19x to 2.01x. Four are a wash — the pool was already
  wide enough there.

• --ref-scores runs its reference scan on the GPU instead of on the host
  (#297). The flag that gates production demuxing was the one configuration
  with no GPU decode path: try_run_and_decode_with_refs always copied the
  whole score tensor back and ran the CPU lattice decode, where the plain path
  decodes on the device and never copies at all. Over the isolated CRF stage on
  100k reads that was 19.7 s at 656% CPU against 12.4 s at 114% without the
  flag — +57% wall for five and a half extra cores, with the card idle.

  The constrained scan is now a CUDA kernel (the one #241 left unwritten): one
  block per read, a grid-stride loop over chain cells, both alpha buffers
  double-buffered in shared memory, and logZ_full reduced on the device so
  only n_refs floats per read come back rather than a strided gather over a
  157 MB alpha buffer. It runs between the two decode passes by necessity, not
  by choice — it needs the raw scores that pass 1 overwrites in place.

  End to end on one A30 over 100k reads, arms interleaved in one allocation:
  38.7 s → 31.5 s wall (1.23x), and 5.9 cores → 1.3 cores. Barcode calls are
  identical for all 100,654 reads; crf_logp, crf_margin and mean_logpost
  agree to 2e-4, which is the output's own print precision.

  Note this does not close #297. GPU utilisation is unchanged at ~22%, so
  removing 4.5 cores of host work from the critical path bought only 20% wall.
  That is evidence for the issue's "overlap-bound, not throughput-bound"
  framing: the pipeline is waiting on something structural, not on compute.

  A panel that does not fit the kernel's shared memory, or whose fan-in exceeds
  the fixed accumulator, falls back to the CPU scan — slower, never wrong.

---

Which artifact

Artifact	Linkage	--gpu
…-x86_64-unknown-linux-musl.tar.gz	static (musl)	no
…-aarch64-unknown-linux-musl.tar.gz	static (musl)	no
…-x86_64-unknown-linux-gnu-gpu.tar.gz	dynamic, glibc ≥ 2.28	yes
…-x86_64-apple-darwin.tar.gz	dynamic	no
…-aarch64-apple-darwin.tar.gz	dynamic	no

The musl builds are the portable default and the right thing for
an unattended installer to fetch. The GPU paths cannot be static —
they dlopen their runtimes — so they ship in the single dynamically
linked …-linux-gnu-gpu artifact instead, built against glibc 2.28
(RHEL/Rocky/Alma 8+, Ubuntu 20.04+).

What …-linux-gnu-gpu needs at run time

Only when --gpu is actually requested — otherwise it behaves
exactly like the musl one. It expects a CUDA 12 runtime and
cuDNN 9, with an NVIDIA driver ≥ 535 (the DTW kernels target the
CUDA 12.2 driver API), and a CUDA-enabled libonnxruntime matching
the ort 2.0.0-rc.13 it links.

Rather than assembling that by hand, see
GPU acceleration,
which covers the pixi environment that supplies it and how to
confirm the CUDA execution provider actually loaded.

v0.17.2

Fixed

• Every command that takes POD5 input accepts a directory, and refuses a
  path that does not exist (#293). escpod demux basecall <run>/pod5 logged
  one WARN, wrote a header-only CSV and exited 0 — a result nothing
  downstream can tell from a run where no read passed. In the report that
  found this, the empty table became a per-barcode share over zero rows and
  reported 0.00x enrichment, which is a meaningful value in that analysis:
  a failed run rendered as a clean negative result, caught only by noticing it
  finished in 85 s.

  demux fingerprint had the same failure one stage earlier (and without even
  the warning), and demux detect, demux split, the fused demux pipeline
  and subset died on a directory with a bare No such device (os error 19)
  from the mmap. None of the six called resolve_pod5_inputs, which
  merge/view/index/annotate/repack/filter/resquiggle/signal classify have always used; they all do now, so a directory expands to the
  *.pod5 under it, a missing path is Path does not exist:, and an empty
  directory is No POD5 files found in directory:. No -r flag: escpod's
  directory expansion is recursive everywhere already, and one command
  needing an opt-in would be the odd one out.

  subset gains multi-input support as a consequence, via the subset_files
  that demux split already used: a group whose reads span several files of a
  run comes out as one output rather than needing a merge afterwards.

  A POD5 that fails to open or decode after that validation is now fatal in
  demux fingerprint and demux basecall rather than skipped. Those paths
  swallowed a truncated or corrupt file and wrote a short table with a zero
  exit; demux detect has always propagated the same three errors, and the
  point of #293 is that the stages of one pipeline should not disagree about
  what counts as a failure.

---

Which artifact

Artifact	Linkage	--gpu
…-x86_64-unknown-linux-musl.tar.gz	static (musl)	no
…-aarch64-unknown-linux-musl.tar.gz	static (musl)	no
…-x86_64-unknown-linux-gnu-gpu.tar.gz	dynamic, glibc ≥ 2.28	yes
…-x86_64-apple-darwin.tar.gz	dynamic	no
…-aarch64-apple-darwin.tar.gz	dynamic	no

The musl builds are the portable default and the right thing for
an unattended installer to fetch. The GPU paths cannot be static —
they dlopen their runtimes — so they ship in the single dynamically
linked …-linux-gnu-gpu artifact instead, built against glibc 2.28
(RHEL/Rocky/Alma 8+, Ubuntu 20.04+).

What …-linux-gnu-gpu needs at run time

Only when --gpu is actually requested — otherwise it behaves
exactly like the musl one. It expects a CUDA 12 runtime and
cuDNN 9, with an NVIDIA driver ≥ 535 (the DTW kernels target the
CUDA 12.2 driver API), and a CUDA-enabled libonnxruntime matching
the ort 2.0.0-rc.13 it links.

Rather than assembling that by hand, see
GPU acceleration,
which covers the pixi environment that supplies it and how to
confirm the CUDA execution provider actually loaded.

v0.17.1

Recovers the v0.17.0 release. 0.17.0's GPU artifact failed to build, which
skipped the Create Release job — so 0.17.0 reached PyPI but has no GitHub
Release and no binaries. PyPI does not allow re-uploading a version, so
v0.17.0 is superseded rather than corrected: 0.17.1 is the first 0.17.x
with downloadable binaries, and carries the same changes as 0.17.0 plus
the two fixes below. The v0.17.0 tag was deliberately not moved, so it
still points at exactly what PyPI was built from.

Build / Tooling

• PyPI can no longer publish a version whose binaries failed to build.
  This is the second half of the v0.17.0 failure, and the part that made it
  unrecoverable. publish-pypi needed only [wheels, sdist], so it was
  independent of the CLI builds by design — the stated intent being that a
  wheel failure should never block the GitHub Release. Run in the other
  direction that same independence meant the GPU binary could fail, the
  release job be skipped, and PyPI publish regardless. 0.17.0 is therefore
  installable with pip but has no GitHub Release and no binaries, and since
  PyPI refuses re-uploads, that half cannot be withdrawn or corrected — only
  superseded.

  The rule now is that the irreversible step goes last: publish-pypi
  waits on release, which waits on build, build-gpu, wheels and
  sdist. A GitHub Release can be deleted and re-created and the workflow
  re-run; a PyPI upload cannot, so it happens only once everything that could
  still fail already has not. Under this graph v0.17.0 would have published
  nothing at all, which is the correct outcome.

  Making release wait on the wheels fixes a second, quieter bug. The wheels
  and sdist are attached to the GitHub Release as assets, but the job
  downloads artifacts with no pattern — so the asset list was decided by a
  race, and a wheel job slower than the binaries would have been silently
  omitted. It never happened, which is exactly why it was worth closing.
  SHA256SUMS.txt now covers the wheels as well, which it never did.

  skip-existing is set on the publish step. With PyPI last, re-running a run
  that failed after publishing would otherwise die on a duplicate upload
  with the GitHub Release still broken — the re-run needs to be able to reach
  the thing that failed.

Fixed

• The v0.17.0 GPU release artifact failed to build, and no GitHub Release
  was published. ort was declared without default-features = false, so
  its default download-binaries came along and dragged in ureq with
  native-tls, and therefore openssl-sys. Under load-dynamic — which is
  how this crate uses ort, and the reason nothing CUDA is needed at build
  time — that machinery never downloads or links anything, so the whole TLS
  stack was dead weight in the graph. It became fatal in #277's new artifact
  job, which builds in a manylinux_2_28 container carrying no
  openssl-devel: openssl-sys's build script failed, and since release
  needs build-gpu, the GitHub Release for v0.17.0 was skipped entirely.
  (The wheels were unaffected and 0.17.0 is on PyPI.)

  ort now takes default-features = false and restates the four defaults
  actually used (std, ndarray, tracing, api-27) alongside cuda and
  load-dynamic. api-27 is kept explicitly rather than dropped, so the
  required onnxruntime API version is unchanged. This removes 19 packages
  from Cargo.lock — openssl*, native-tls, schannel,
  security-framework, and the rest of that subtree — with no additions and
  no version changes, and no code change of any kind.

  CI could not have caught this, which is the more interesting half. The
  gpu feature is built on every PR, but on ubuntu-latest, which has
  libssl-dev — so openssl-sys compiled happily there and the break
  surfaced only in the release container, on a tag, after PyPI had already
  published. The feature-builds job now asserts openssl-sys is unreachable
  under both gpu and models-download, over --target all -e all so a
  target-specific or build-dependency edge counts too. Every TLS user in this
  workspace is meant to be rustls: the musl artifacts are statically linked
  and the GPU artifact builds in manylinux, and neither has an OpenSSL to
  find.

---

Which artifact

Artifact	Linkage	--gpu
…-x86_64-unknown-linux-musl.tar.gz	static (musl)	no
…-aarch64-unknown-linux-musl.tar.gz	static (musl)	no
…-x86_64-unknown-linux-gnu-gpu.tar.gz	dynamic, glibc ≥ 2.28	yes
…-x86_64-apple-darwin.tar.gz	dynamic	no
…-aarch64-apple-darwin.tar.gz	dynamic	no

The musl builds are the portable default and the right thing for
an unattended installer to fetch. The GPU paths cannot be static —
they dlopen their runtimes — so they ship in the single dynamically
linked …-linux-gnu-gpu artifact instead, built against glibc 2.28
(RHEL/Rocky/Alma 8+, Ubuntu 20.04+).

What …-linux-gnu-gpu needs at run time

Only when --gpu is actually requested — otherwise it behaves
exactly like the musl one. It expects a CUDA 12 runtime and
cuDNN 9, with an NVIDIA driver ≥ 535 (the DTW kernels target the
CUDA 12.2 driver API), and a CUDA-enabled libonnxruntime matching
the ort 2.0.0-rc.13 it links.

Rather than assembling that by hand, see
GPU acceleration,
which covers the pixi environment that supplies it and how to
confirm the CUDA execution provider actually loaded.

v0.16.1

Added

• sequence_bases_with_context: the k-mer context window as bases (#274).
  #272 moved the signal-level k-mer encoding upstream and leech now calls it
  (rnabioco/leech#222), but one overlap survived, and it was the same shape of
  problem: leech kept its own copy of the context windowing, because
  sequence_ints_with_context returns ints and a training corpus serialises
  the context — sequence_with_kmer_context is a string in the chunk format,
  which data merge/load_chunks read back as one. So the downstream caller
  needed bases, and deriving them from the ints by hand would have been a third
  copy of the window rather than the end of the second.

  The new form is the same cut, padded with UNKNOWN_BASE_CHAR (N), and it
  composes exactly: sequence_to_int of the bases is
  sequence_ints_with_context, padding included, because
  base_to_int(UNKNOWN_BASE_CHAR) is UNKNOWN_BASE. That equivalence is what
  makes this a refactor rather than a new rule, and a test sweeps it over five
  contexts and every offset from the start of the sequence to past its end, so
  a divergence between the two forms fails rather than ships. The windowing
  arithmetic itself is now one private helper (context_range) that both
  public forms call.

  Worth doing because this is the step where KmerContext's halves are not
  interchangeable — swap them and every k-mer is read from a window displaced
  by before - after bases, silently, and encode_signal_kmer cannot detect
  it because it only ever sees their sum. A second copy of precisely that rule
  is the one this crate least wants to keep.

v0.16.0

Added

• escapepod-signal owns the signal-level k-mer encoding
  (seq_encoding, #271). mapping (#262) already produced a base→signal
  map; the primitive that consumes one — scattering the one-hot k-mer context
  along the signal axis, the 36-channel sequence input of a leech
  seq_encoding="signal_kmer" model — lived downstream in leech, inside a
  cdylib Python extension module that Rust cannot link. Since that tensor is
  computed in the dataset it is not in leech's exported ONNX graph
  (rnabioco/leech#220), so a Rust runtime has to build it before it can call
  the model at all, and "call leech-core" is not an option. The choice was to
  transcribe the rule or not to run those models — which is how
  KmerTable::extract_levels ended up with two centring conventions and how
  escapepod-classify reproduced a superseded feature definition for two
  months.

  The new module is encode_signal_kmer (plus an _into form for a hot loop
  that would otherwise allocate per chunk), sequence_ints_with_context for
  cutting the context window a chunk needs, and the A/C/G/T=U alphabet
  (base_to_int, sequence_to_int) that both take —
  resquiggle::kmer_table now shares that one definition rather than carrying
  its own copy. KmerContext names the (before, after) pair, since
  transposing it displaces every k-mer window by before - after bases and
  still returns a correctly shaped tensor, and it is where channels() (36 for
  the usual (4, 4)) is computed rather than in each caller.

  Parity with leech's NumPy reference is pinned bit-exactly over 35 cases
  (tests/signal_kmer_parity.rs, regenerate with
  tests/fixtures/gen_signal_kmer_golden.py) — the encoding is exactly zeros
  and ones, so there is no tolerance to argue about. The golden is generated
  from the NumPy path deliberately: leech's own compiled extension
  disagrees with its own fallback on a span whose start is negative, because it
  clamps after an as usize cast, so the start lands on signal_len, the
  span comes out empty and the base disappears. Measured against
  leech_core 0.8.0 on a 3-base window with a map of [-8, 10, 20, 30]: 60
  hot samples from the extension against 90 from NumPy, and for
  [-30, -20, 40, 60] a span covering the entire window vanishes to 0. This
  crate keeps the surviving tail, which is both the readable definition and
  what a reference-anchored map — whose entries legitimately go negative once
  the aligned region is cropped — needs.

• POD5 V6 files are readable (upstream 0.3.46). V6's only change is that
  the reads-table channel column is retyped from uint16 to uint32 — same
  name, same position, so nothing about the container moves. But because it
  retypes an existing column rather than appending new ones the way V4 and V5
  did, it is not a change a narrow reader can ignore: pinned to uint16, every
  V6 file fails outright rather than degrading. channel is now resolved to
  whichever width the file carries and widened to u32, on the per-row path,
  the bulk columnar path, and the row extractor alike. V0–V5 files are
  unaffected.

  ReadData.channel, ReadColumns.channel, and the Python ReadData.channel
  / Writer.add_read parameter are u32 accordingly, matching upstream's own
  C++ ReadData. to_dict/to_pandas/to_polars hand back a uint32 column
  where they used to give uint16.

Changed

• Written files stay V5; an unrepresentable channel now fails the write.
  Emitting V6 today would make every file escpod produces unreadable by every
  installable reader: the newest pod5 on PyPI is 0.3.44, and it rejects a
  uint32 channel with Schema field 'channel' is incorrect type: 'uint32'
  (verified against an escpod-written V6 file). The trade would be a hard break
  with the deployed ecosystem in exchange for channel numbers no flow cell
  produces — PromethION tops out at 3000. So the emitted column stays uint16
  and files stay stamped 0.3.44, while reading stays lossless at both widths.

  The one input that would lose data — a channel above u16::MAX, which can
  only come from a genuine V6 file — is refused with an error naming V6 rather
  than silently written as channel % 65536. escpod inspect summary's
  channel statistics widen to match.

  This flips once ONT publishes v6-capable wheels: narrow_channel in
  escapepod-pod5::schema::reads is the single site, and
  emitted_channel_width_matches_the_stamped_version pins the schema width and
  POD5_VERSION together so they cannot drift apart.

v0.15.0

Build / Tooling

• The POD5 compat job stops rebuilding its dependency graph every run.
  0.14.0 moved it off --release onto an optimised-but-not-LTO profile, which
  did not work: Swatinem/rust-cache derives its key from Cargo.lock and the
  toolchain, not from the cargo profile, and GitHub refuses to overwrite an
  existing cache key. The key therefore still held the old release artefacts,
  so every run restored artefacts it could not use, rebuilt everything, and
  then declined to save because the key already existed. Measured on main:
  363 s of a 400 s job was cargo build, the compat test itself was 1 s,
  and the cache post-step wrote nothing. Warm cost went 316 s → 400 s — the
  change cost more than it saved.

  Two fixes, both needed. The cache key now carries an explicit suffix that is
  bumped whenever the profile changes, so a profile switch can actually be
  saved. And the job builds the dev profile rather than an optimised one,
  because the suite round-trips a five-read fixture and the binary's
  throughput is irrelevant to it; third-party crates still compile at
  opt-level = 2 through [profile.dev.package."*"], so only escapepod's own
  crates drop to -O0, and those have to rebuild on any source change anyway.
  The now-unused ci-bin profile is removed.

Added

• A cache of open, indexed readers, so the read-id index is built once per
  file instead of once per reader (#258). Reader caches its index in a
  OnceLock on the instance, so a consumer that opens a reader per batch
  throws the index away and rebuilds it on the next batch. That is not a small
  constant: on a 145 GB POD5 on a network filesystem it was minutes of
  uninterruptible sleep in folio_wait_bit_common per batch at ~0.6% of one
  core — the 10–80x data-preparation regression in rnabioco/leech#176. #251
  fixed the other half of it (the scan variants are gone and lookups index
  unconditionally), but "one reader per file per process" was left to every
  consumer, and each consumer that did not write it silently got the slow path.
  leech wrote it in Rust, and then wrote the same idea again, independently, in
  Python.

  Both shapes ship, because they answer different questions. cached_reader()
  is the process-global convenience, and it is the one that makes consumers
  actually stop hand-rolling this. ReaderCache is the owned type underneath
  it, for a library that needs the lifetime bounded or a process where one
  stage must not share readers with another; global_reader_cache() reaches
  the global's len() / clear().

  The value is in the ordering and the failure semantics rather than in the
  static, so those are the parts worth stating:

  • The file is opened outside the lock, which guards only the map. A slow
    open on one path never blocks a lookup on another, and the lock is never
    held across I/O, so this cannot deadlock. Two threads racing the same path
    cost one redundant open and both get the winner's Arc — publication goes
    through entry, not insert, so a race can never leave two live readers
    (and two indexes) for one file.
  • The index is warmed before the entry is published, so N workers hitting
    their first batch together find it built instead of piling up inside one
    lazy init. The warm-up respects autoindex_max(): above that read count it
    is skipped, because warming is a guess that random access is coming and a
    huge file that is only iterated should not pay for an index nobody asked
    for. Skipping only defers the build to the first lookup that demands one,
    and because the reader is now shared that build still happens once per file
    rather than once per batch — the cache keeps its whole value above the
    threshold, it just stops guessing.
  • A failed index build is logged, not propagated. Reader::open failing
    is an error, because there is no reader to hand back. An un-indexable
    POD5 is still a perfectly good reader for iteration, metadata, and signal
    access, and failing an open for a caller that may never do a lookup is
    worse than the slowdown; a caller that does demand a lookup sees the same
    error then, from the call that needs it. (One correction to the issue's
    framing: after #251 such a file is not "readable, just slowly" — the error
    surfaces from reads_by_ids rather than degrading to a scan. The reader
    stays usable; lookups by read id do not.)

  Keys are canonicalized, falling back to the path as given if that fails, so
  reads.pod5, ./reads.pod5, and a symlink to it are one entry rather than
  three readers with three indexes. The reader is opened on the canonical path
  too, so .p5s sidecar resolution does not depend on which spelling happened
  to arrive first. What stays resident is the index and not the file —
  ~24 bytes/read, so a few tens of MB even for a multi-million-read POD5 — and
  entries are never evicted, with clear() as the escape hatch for a process
  that walks an unbounded set of files.

• Reader::read_index_if_built() — the non-committing half of
  read_index(): it never loads a sidecar and never scans, so it is the only
  way to ask whether a reader is warm without making it warm. Without it the
  warm-before-publish ordering above is unobservable, and a test that "checked"
  it by calling read_index() would only be asserting its own side effect.

• escapepod_signal::mapping: the two Oxford Nanopore coordinate
  conventions that produce a resquiggle's input. refine_signal_map has
  always taken a sequence→signal map; nothing in the workspace produced
  one. So every consumer wrote its own eight lines off the mv/ns/ts tags
  and its own CIGAR walk — three copies in this repo alone (the charging
  classifier's anchoring, the resquiggle command, a test helper), plus the
  ones downstream. Each is a shifted map away from answering a different
  question than the caller thinks, with no error to show for it, which is the
  same argument that moved the k-mer level primitives here.

  • seq_to_signal_from_moves(moves, stride, trim_offset, num_samples) —
    Remora's query_to_signal = np.nonzero(mv)[0] * stride, returned in
    trimmed-signal coordinates with num_samples - trim_offset as the
    closing boundary, because that is the frame the move table is in and the
    frame refine_signal_map is handed. A caller indexing the untrimmed POD5
    array adds trim_offset back; the charging anchoring now does that
    explicitly instead of folding + ts into the map's construction, where
    the frame was invisible.
  • ref_to_signal(query_to_signal, cigar) — reference→signal by the Remora
    knot convention: trailing non-match ops stripped, knots at the start and
    end - 1 of each match block (not end, which stretches every gap by a
    position), exact 1:1 integer lookup inside a block, and linear
    interpolation only across indel gaps.

  The CIGAR arrives as a local CigarOp { kind, len } rather than the
  (op, len) integer pair the convention is usually written with: the crate
  takes no alignment-library dependency for this, and a bare pair of integers
  is exactly what a caller transposes without the compiler noticing.

  ref_to_signal is integer arithmetic throughout except the one ratio each
  gap position needs — deliberately not the ref → float query → float signal
  chain that a pair of np.interp calls performs. Both interpolations there
  evaluate slope * (x - x0) + y0 with a pre-rounded slope, and the result is
  floored, so a one-ulp difference in the intermediate query coordinate
  becomes a one-sample difference in the answer: with the map [0, 7, 8] and
  a CIGAR of 1M 6D 1M, the float chain puts reference position 5 at sample 4
  instead of 5. It is rare — a 200 000-case sweep of realistic random CIGARs
  found no difference at all, and it takes a long deletion spanned by short
  dwells — which is precisely what makes it expensive to find once two
  consumers have each written their own version. It is pinned by a test here
  rather than rediscovered downstream.

Changed

• features::span_stats gains a median, a range, a fill policy and an
  out-of-range policy, and takes a SpanConfig instead of a bare
  Normalization (#260). The reduction was already the right one — one pass
  over the covered region with f64 prefix sums, O(1) per span, spans supplied
  by the caller — but three of its choices were baked in, and a consumer that
  disagreed with any of them could not use the function at all. leech therefore
  carried its own copy, then a second copy, and the two disagreed on exactly
  one of those choices (rnabioco/leech#200): the Python fast path skipped a
  span with a negative start and left zeros, the Rust pipeline computed over
  the truncated span. Same read, different features, depending on which path
  reached it. The payoff is not line count; it is that the numbers stop
  depending on which code ran. Precedent: #204, where the rule that decides
  what a model sees was moved to the crate that owns the reduction rather than
  re-derived in each caller.

  The three gaps, all now named fields on SpanConfig rather than assumptions.
  SpanStatsOut grows optional median and range buffers, built through
  SpanStatsOut::new(..).with_median(..).with_range(..) — optional because
  neither can come from the prefix sums (each needs its own pass over the span,
  and the median a select or a sort on top), so a caller wanting only
  dwell/mean/sd does not pay for them. SpanFill { Nan, Zero, Value(f32) }
  chooses what an unresolved span gets: Nan stays the default and stays the
  honest answer — an unresolved base has no observation, and a substituted
  value is indistinguishable from a real one — but that argument does not
  survive contact with a neural network, where one NaN poisons the forwa...

v0.14.0

Build / Tooling

• CI audit: duplicate compile passes removed, cache churn stopped. Measured
  per-job on a warm run (PR #252) and a cold one (a Dependabot lockfile bump),
  then cut by what the numbers showed rather than by what looked redundant.

  • check is gone; clippy covers it. cargo clippy --workspace --all-targets runs the same rustc front-end over the same unit graph as
    cargo check --workspace --all-targets and then adds lints, so a green
    clippy already implied a green check. The workspace was being compiled
    twice per PR for no additional signal. The same pair ran in release.yml's
    macOS gate, where it cost 10x — GitHub bills macOS minutes at ten times
    Linux.

  • Doctests moved into the test job. nextest does not run doctests, but
    cargo test --doc needs the same toolchain, profile and unit graph that
    nextest has just built. As its own job it rebuilt the whole dependency
    graph from scratch — 406 s cold, against roughly 2 s of actual doctest
    execution — and held one of the largest caches in the repo. Sharing the
    target directory makes it nearly free.

  • The POD5 compat suite stops building a shipping artefact. It was
    cargo build --release, i.e. fat LTO and codegen-units = 1, to produce a
    binary that round-trips small fixtures. That made it the longest job in the
    workflow (316 s warm, 480 s cold) and therefore the critical path of every
    PR. It now builds the new ci-bin profile — opt-level = 3 kept,
    whole-program optimisation and single-threaded codegen dropped — and points
    the suite at it through the ESCPOD_BIN override the harness already had.

  • Coverage runs the instrumented suite once instead of twice. The job
    ran cargo llvm-cov nextest in full for lcov and then again in full for
    HTML. It now runs once with --no-report and renders lcov, the threshold
    summary and HTML from that single set of profiles.

  • Two feature builds dropped as duplicates. The features job exists
    because "the default jobs never compile the opt-in features" — but
    cnn-detect and crf-decode are both in escapepod-cli's default cli
    feature, so workspace feature unification already compiled escapepod-demux
    with them in the ordinary clippy job. The genuinely opt-in ones (gpu,
    cnn-gpu, crf-gpu, models-download) are untouched.

  • PR runs no longer write to the Actions cache. The repo held 11 GB
    across 34 entries against GitHub's 10 GB per-repository ceiling, so it sat
    in permanent LRU eviction — which is why nominally warm jobs still showed
    cold timings. Every Swatinem/rust-cache step now carries
    save-if: github.ref == 'refs/heads/main', so PRs restore from main's
    cache but never add to it, and cache entries stop multiplying per branch.

  • Prose-only changes no longer start a Rust build. paths-ignore on
    docs/**, **/*.md, LICENSE* and .gitignore. A commit touching both
    prose and code still runs everything — paths-ignore suppresses a run only
    when every changed path matches — and with no branch protection on main
    there are no required checks for a skipped run to block.

Fixed

• A targeted lookup no longer scans the whole reads table when there is no
  .p5s sidecar (#251). reads_by_ids, find_signal_rows_by_ids and
  find_signal_rows_with_calibration_by_ids chose between an indexed path and
  a full scan with a has_index() helper that answered "is there a sidecar on
  disk?" — a different question from "can I use the index?", and one with a
  different answer whenever a sidecar is absent, which is the common case. So
  every call scanned the file, and none of them built the index that would have
  made the next call a seek, even though read_index() was already there,
  self-caching, and cheaper than the scan being chosen instead.

  The fallback was never the cheap option. The index is a scan, projected to
  the read_id column, so building it moves strictly fewer bytes than one
  execution of the path it declined — against 22 columns for reads_by_ids,
  2 and 4 for the signal lookups — and it is then cached for every later call.
  The early exit that justified the scan ("stops once all targets are found")
  does not fire in the access pattern that matters: targets arrive in BAM
  order, unrelated to POD5 storage order, so the last one sits near EOF and the
  scan runs to the end of the file. Per call. Downstream this cost
  rnabioco/leech roughly 10–80x on data preparation, silently; on a 145 GB
  merged POD5 one call for 1000 ids had not returned after 13.5 minutes.

  All three entry points now go through read_index(). The scan variants are
  gone rather than kept behind a size threshold: what decides whether a scan
  could win is where the targets land, not how many there are, so a threshold
  on target_ids.len() cannot detect the one case it would be for.

Changed

• autoindex_max() moved from the Python bindings into escapepod-pod5,
  where the decision it encodes actually lives (#251). It was reachable only
  from Python — two call sites there warmed the index on context-manager entry
  to route around the scan described above — so no Rust caller could reach the
  policy, and the workaround had to be written again in every consumer.
  ESCAPEPOD_AUTOINDEX_MAX and the 5,000,000-read default are unchanged.

  Its meaning is now narrower and honest: it gates speculative indexing only.
  Entering a Python reader as a context manager still checks it, because that
  is a guess that random access is coming and a large file that is only
  iterated should not pay for an index nobody asked for. A caller that has
  actually asked for random access always gets an index, whatever the file
  size — above the threshold the build is reported at warn naming
  escpod index, not traded for a scan. It was never a memory guard in any
  case: loading a .p5s sidecar has always built the same in-memory entry
  table with no cap at all, so the same file with a sidecar already holds what
  the threshold claimed to prevent.

• Dependency bumps (lockfile only, no behavior change): the Arrow ecosystem
  arrow + parquet 59.1 → 59.2 and the tract-* stack 0.23.4 → 0.23.5,
  plus bit-vec 0.9 → 0.11 and bitflags 0.19.8 → 0.19.9 (#250).

Added

• escapepod-pod5 can log. The format crate — the layer every other one
  sits on — had no tracing dependency at all, which is why a per-call rescan
  of a 145 GB file was indistinguishable from slow I/O and cost a day of
  profiling to find. Building a read index now says that it is happening, why
  (no sidecar), and what it cost (reads, batches, elapsed).

• A reads_by_ids group in io_hot_paths, covering the three arms that
  matter: index loaded from a sidecar, index built on the first call, and the
  warm steady state. Its doc states the limit of what it can prove — at fixture
  scale a scan and an indexed seek cost the same, so the guard against taking
  the wrong path is the invariant test that asserts the index is built exactly
  once for two lookups, not the benchmark.