Skip to content

v0.16.0

Choose a tag to compare

@github-actions github-actions released this 25 Aug 22:52
· 32 commits to main since this release
29cea89

Added

  • escapepod-signal owns the signal-level k-mer encoding
    (seq_encoding, #271).
    mapping (#262) already produced a base→signal
    map; the primitive that consumes one — scattering the one-hot k-mer context
    along the signal axis, the 36-channel sequence input of a leech
    seq_encoding="signal_kmer" model — lived downstream in leech, inside a
    cdylib Python extension module that Rust cannot link. Since that tensor is
    computed in the dataset it is not in leech's exported ONNX graph
    (rnabioco/leech#220), so a Rust runtime has to build it before it can call
    the model at all, and "call leech-core" is not an option. The choice was to
    transcribe the rule or not to run those models — which is how
    KmerTable::extract_levels ended up with two centring conventions and how
    escapepod-classify reproduced a superseded feature definition for two
    months.

    The new module is encode_signal_kmer (plus an _into form for a hot loop
    that would otherwise allocate per chunk), sequence_ints_with_context for
    cutting the context window a chunk needs, and the A/C/G/T=U alphabet
    (base_to_int, sequence_to_int) that both take —
    resquiggle::kmer_table now shares that one definition rather than carrying
    its own copy. KmerContext names the (before, after) pair, since
    transposing it displaces every k-mer window by before - after bases and
    still returns a correctly shaped tensor, and it is where channels() (36 for
    the usual (4, 4)) is computed rather than in each caller.

    Parity with leech's NumPy reference is pinned bit-exactly over 35 cases
    (tests/signal_kmer_parity.rs, regenerate with
    tests/fixtures/gen_signal_kmer_golden.py) — the encoding is exactly zeros
    and ones, so there is no tolerance to argue about. The golden is generated
    from the NumPy path deliberately: leech's own compiled extension
    disagrees with its own fallback on a span whose start is negative, because it
    clamps after an as usize cast, so the start lands on signal_len, the
    span comes out empty and the base disappears. Measured against
    leech_core 0.8.0 on a 3-base window with a map of [-8, 10, 20, 30]: 60
    hot samples from the extension against 90 from NumPy, and for
    [-30, -20, 40, 60] a span covering the entire window vanishes to 0. This
    crate keeps the surviving tail, which is both the readable definition and
    what a reference-anchored map — whose entries legitimately go negative once
    the aligned region is cropped — needs.

  • POD5 V6 files are readable (upstream 0.3.46). V6's only change is that
    the reads-table channel column is retyped from uint16 to uint32 — same
    name, same position, so nothing about the container moves. But because it
    retypes an existing column rather than appending new ones the way V4 and V5
    did, it is not a change a narrow reader can ignore: pinned to uint16, every
    V6 file fails outright rather than degrading. channel is now resolved to
    whichever width the file carries and widened to u32, on the per-row path,
    the bulk columnar path, and the row extractor alike. V0–V5 files are
    unaffected.

    ReadData.channel, ReadColumns.channel, and the Python ReadData.channel
    / Writer.add_read parameter are u32 accordingly, matching upstream's own
    C++ ReadData. to_dict/to_pandas/to_polars hand back a uint32 column
    where they used to give uint16.

Changed

  • Written files stay V5; an unrepresentable channel now fails the write.
    Emitting V6 today would make every file escpod produces unreadable by every
    installable reader: the newest pod5 on PyPI is 0.3.44, and it rejects a
    uint32 channel with Schema field 'channel' is incorrect type: 'uint32'
    (verified against an escpod-written V6 file). The trade would be a hard break
    with the deployed ecosystem in exchange for channel numbers no flow cell
    produces — PromethION tops out at 3000. So the emitted column stays uint16
    and files stay stamped 0.3.44, while reading stays lossless at both widths.

    The one input that would lose data — a channel above u16::MAX, which can
    only come from a genuine V6 file — is refused with an error naming V6 rather
    than silently written as channel % 65536. escpod inspect summary's
    channel statistics widen to match.

    This flips once ONT publishes v6-capable wheels: narrow_channel in
    escapepod-pod5::schema::reads is the single site, and
    emitted_channel_width_matches_the_stamped_version pins the schema width and
    POD5_VERSION together so they cannot drift apart.