v0.16.0
Added
-
escapepod-signalowns the signal-level k-mer encoding
(seq_encoding, #271).mapping(#262) already produced a base→signal
map; the primitive that consumes one — scattering the one-hot k-mer context
along the signal axis, the 36-channelsequenceinput of a leech
seq_encoding="signal_kmer"model — lived downstream in leech, inside a
cdylibPython extension module that Rust cannot link. Since that tensor is
computed in the dataset it is not in leech's exported ONNX graph
(rnabioco/leech#220), so a Rust runtime has to build it before it can call
the model at all, and "call leech-core" is not an option. The choice was to
transcribe the rule or not to run those models — which is how
KmerTable::extract_levelsended up with two centring conventions and how
escapepod-classifyreproduced a superseded feature definition for two
months.The new module is
encode_signal_kmer(plus an_intoform for a hot loop
that would otherwise allocate per chunk),sequence_ints_with_contextfor
cutting the context window a chunk needs, and theA/C/G/T=Ualphabet
(base_to_int,sequence_to_int) that both take —
resquiggle::kmer_tablenow shares that one definition rather than carrying
its own copy.KmerContextnames the(before, after)pair, since
transposing it displaces every k-mer window bybefore - afterbases and
still returns a correctly shaped tensor, and it is wherechannels()(36 for
the usual(4, 4)) is computed rather than in each caller.Parity with leech's NumPy reference is pinned bit-exactly over 35 cases
(tests/signal_kmer_parity.rs, regenerate with
tests/fixtures/gen_signal_kmer_golden.py) — the encoding is exactly zeros
and ones, so there is no tolerance to argue about. The golden is generated
from the NumPy path deliberately: leech's own compiled extension
disagrees with its own fallback on a span whose start is negative, because it
clamps after anas usizecast, so the start lands onsignal_len, the
span comes out empty and the base disappears. Measured against
leech_core0.8.0 on a 3-base window with a map of[-8, 10, 20, 30]: 60
hot samples from the extension against 90 from NumPy, and for
[-30, -20, 40, 60]a span covering the entire window vanishes to 0. This
crate keeps the surviving tail, which is both the readable definition and
what a reference-anchored map — whose entries legitimately go negative once
the aligned region is cropped — needs. -
POD5 V6 files are readable (upstream 0.3.46). V6's only change is that
the reads-tablechannelcolumn is retyped fromuint16touint32— same
name, same position, so nothing about the container moves. But because it
retypes an existing column rather than appending new ones the way V4 and V5
did, it is not a change a narrow reader can ignore: pinned touint16, every
V6 file fails outright rather than degrading.channelis now resolved to
whichever width the file carries and widened tou32, on the per-row path,
the bulk columnar path, and the row extractor alike. V0–V5 files are
unaffected.ReadData.channel,ReadColumns.channel, and the PythonReadData.channel
/Writer.add_readparameter areu32accordingly, matching upstream's own
C++ReadData.to_dict/to_pandas/to_polarshand back auint32column
where they used to giveuint16.
Changed
-
Written files stay V5; an unrepresentable channel now fails the write.
Emitting V6 today would make every file escpod produces unreadable by every
installable reader: the newestpod5on PyPI is 0.3.44, and it rejects a
uint32channelwithSchema field 'channel' is incorrect type: 'uint32'
(verified against an escpod-written V6 file). The trade would be a hard break
with the deployed ecosystem in exchange for channel numbers no flow cell
produces — PromethION tops out at 3000. So the emitted column staysuint16
and files stay stamped0.3.44, while reading stays lossless at both widths.The one input that would lose data — a channel above
u16::MAX, which can
only come from a genuine V6 file — is refused with an error naming V6 rather
than silently written aschannel % 65536.escpod inspect summary's
channel statistics widen to match.This flips once ONT publishes v6-capable wheels:
narrow_channelin
escapepod-pod5::schema::readsis the single site, and
emitted_channel_width_matches_the_stamped_versionpins the schema width and
POD5_VERSIONtogether so they cannot drift apart.