sleap-io v0.9.0

sleap-io v0.9.0 Release Notes

Summary

sleap-io v0.9.0 is a feature release that grows the annotation model in two new directions and makes every representation freely interconvertible. It adds a re-identification subsystem — a global Identity catalog plus per-detection appearance Embeddings that attach to every detection modality and round-trip through .slp — and frame-spanning Event annotations, the library's first annotation with a temporal extent (behavior bouts, stimulus epochs, review flags) with an ethogram catalog and inclusive frame intervals. Alongside these, unified modality interconversion gives pose, centroid, bounding box, segmentation mask, and ROI a single shared verb set (.to_centroid() / .to_bbox() / .to_roi() / .to_mask(), plus Centroid.to_pose()) with batch LabeledFrame.convert() / Labels.convert() entry points. It rounds out with name-based skeleton symmetry inference, a sio.download() primitive and sio download CLI, machine-readable --json CLI inspection, large-project save hardening (past HDF5's 64 KB per-attribute limit), an O(N) merge speedup, and pynwb 4 compatibility.

The SLP format advances 2.4 → 2.6 — 2.5 adds the identity/embedding groups and 2.6 adds events. Both are additive and read-on-group-presence, so older readers ignore them and new files still load everywhere.

Highlights:

• Re-identification subsystem (#513, #514, #515, #527, #535, #536) — new Identity (a named, cross-file ground-truth animal identity) and Embedding (a per-detection appearance vector) attach to every detection via identity / identity_score / identity_embedding, collect into Labels.identities, color renders (render --color-by identity), and persist to .slp (format 2.5). Appearance vectors stay off disk by default (save_embedding_vectors=False, mirroring embed).
• Frame-spanning Event annotations (#540) — EventType / UserEvent / PredictedEvent represent anything with a temporal extent over an inclusive [start_frame, end_frame] interval, with optional subject/target participants (Track or Identity) and framewise or scalar prediction confidence. Query with labels.get_events(...) / labels.events_at(video, frame_idx); persist to format 2.6.
• Unified modality interconversion (#531) — pose ⇄ centroid ⇄ bbox ⇄ mask ⇄ ROI share one verb set and batch convert() entry points, preserving track/identity/score metadata and the User/Predicted variant throughout.
• Infer left/right symmetries from node names (#534) — Skeleton.infer_symmetries_by_name() suggests symmetric pairs from names like eye_L/ eye_R or left_paw/right_paw (opt-in, non-mutating).
• sio.download() + sio download CLI (#528) — fetch a remote file straight to disk (http(s), cloud buckets, Google Drive) without loading it — a protocol-agnostic curl/wget replacement with streaming, atomic writes, and skip-if-exists.
• Machine-readable CLI inspection (#538) — sio show --json and sio filenames --json emit structured JSON for scripting; default human-readable output is unchanged.
• Reliable saving for very large projects (#517, #521, #523, #524) — .slp files no longer fail to save (or silently drop metadata) when provenance, merge history, or per-video source metadata exceeds HDF5's 64 KB per-attribute limit.
• Faster merges (#537) — appending merges are now O(N) instead of O(N²) (a reported ~218s → ~0.75s on a 9k-frame merge into a ~95k-frame project).
• pynwb 4 compatibility (#532) — NWB export works under pynwb 4.0 (still supports <4).

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx sleap-io@0.9.0 show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

---

Breaking Changes

v0.9.0 is overwhelmingly additive. Two small, easily-migrated changes:

Identity.color removed — fold color into metadata (#535)

As part of finalizing the re-ID data model, the Identity type's dedicated color field was removed in favor of its general string metadata map. Identity.metadata is now typed dict[str, str] (string values, required for .slp persistence).

# Before (0.8.0)
ident = sio.Identity(name="mouse_A", color="#e6194b")

# After (0.9.0) — color lives in metadata
ident = sio.Identity(name="mouse_A", metadata={"color": "#e6194b"})

Only affects code that set Identity(color=...). Identity was introduced in 0.7.0 for 3D multi-view binding and was not yet persisted, so most pipelines are unaffected.

Labels.merge() bounds merge_history to 1000 records by default (#521)

To keep provenance from growing without bound (and overflowing HDF5's 64 KB attribute limit), Labels.merge() now caps provenance["merge_history"] at the most recent 1000 records by default (oldest trimmed first). Only observable if you merge more than 1000 times and inspect the full history.

# Keep the complete, unbounded merge history (previous behavior)
base.merge(other, max_merge_history=None)

DEFAULT_MERGE_HISTORY_LIMIT = 1000 is exposed on sleap_io.model.labels.

---

New Features

Re-identification subsystem: Identity + Embedding (#513, #514, #515, #527, #535, #536)

v0.9.0 adds a re-identification (re-ID) subsystem for tracking known animals across videos, sessions, and experiments. Two model concepts:

• Identity — a named, cross-file ground-truth animal identity with arbitrary string metadata. Matched by name (default) or object identity, like Track.
• Embedding — a 1-D per-detection appearance / re-ID feature vector.

Every detection modality (Instance, PredictedInstance, Instance3D, PredictedInstance3D, bounding boxes, centroids, masks, ROIs, label images) gains identity, identity_score, and identity_embedding slots; InstanceGroup.identity binds a triangulated multi-view group. Labels.identities is a catalog auto-collected from the detections.

import numpy as np
import sleap_io as sio

mouse_a = sio.Identity(name="mouse_A", metadata={"strain": "C57BL/6"})

inst = sio.Instance.from_numpy(
    np.array([[10.0, 20.0], [30.0, 40.0]]),
    skeleton=skeleton,
    identity=mouse_a,
    identity_score=0.98,
    identity_embedding=sio.Embedding(np.random.rand(128).astype("float32")),
)

# Identities match by name across files
assert mouse_a.matches(sio.Identity(name="mouse_A"))

Persistence (SLP format 2.5). Identity links are written to a new /identity group and appearance vectors to a new /embeddings group. To keep large vectors off disk, save_slp / save_file gain save_embedding_vectors which defaults to False (mirroring the embed=False default) — identity links always persist, vectors only when you ask (#536):

sio.save_slp(labels, "out.slp", save_embedding_vectors=True)   # include vectors

Tooling. sio merge --identity {name,identity} controls how the identity catalog is deduplicated on merge; sio render --color-by identity colors renders by global identity (one palette color per entry in Labels.identities); and sio show reports identity and embedding counts (--json includes n_instances_with_identity_embedding). See docs/model/embedding.md.

Frame-spanning Event annotations (#540)

sleap-io gains its first annotation with a temporal extent. Unlike per-frame annotations that live on a single LabeledFrame, an Event spans an inclusive [start_frame, end_frame] interval and lives on Labels.events, alongside a controlled-vocabulary catalog Labels.event_types. Use it for behavior bouts, stimulus epochs, physiological events, or review flags.

Four new classes are exported: EventType (the ethogram catalog entry), the abstract Event, and its concrete UserEvent (ground truth) and PredictedEvent (adds an optional framewise scores trace and a scalar score). Events carry optional subject/target participants (each a Track or Identity), plus name, source, and string metadata.

import numpy as np
import sleap_io as sio
from sleap_io.model.event import UserEvent, PredictedEvent

video = sio.Video(filename="session.mp4")
labels = sio.Labels(videos=[video])

# Human-annotated behavior bout over frames 100–150 (inclusive)
labels.events.append(UserEvent(type="attack", video=video, start_frame=100, end_frame=150))

# Model-predicted event with a framewise confidence trace + scalar score
labels.events.append(
    PredictedEvent(
        type="rear", video=video, start_frame=10, end_frame=12,
        scores=np.array([0.8, 0.9, 0.7]), score=0.85,
    )
)

# What is happening at frame 120? (span-covering, not per-frame)
labels.events_at(video, 120)                    # -> [UserEvent(type="attack", ...)]
labels.get_events(type="rear", predicted=True)  # -> [PredictedEvent(type="rear", ...)]

labels.save("out.slp")   # persists via SLP format 2.6

Labels.get_events(video, subject, type, frame_idx, predicted) filters the collection (frame_idx matches any event whose span covers the frame); Labels.events_at(video, frame_idx) is the "what's happening now?" convenience wrapper. Events persist to a new /event_types + /events SLP group pair (format 2.6). See docs/model/events.md.

Scope note: events persist only in .slp. Converting to NWB, Label Studio, JABS, or a DataFrame drops them — sio convert prints an explicit warning when it does.

Unified interconversion between detection modalities (#531)

Pose, centroid, bounding box, segmentation mask, and ROI now share one verb vocabulary, so you can freely move between detection representations while preserving metadata.

import sleap_io as sio

inst = labels[0].instances[0]

# Anchor a crop centroid on the thorax, fall back to center-of-mass
centroid = inst.to_centroid(method="anchor", node="thorax", fallback="center_of_mass")
box  = inst.to_bbox(padding=4, rotated=True)               # oriented, padded bbox
mask = inst.to_mask(height=H, width=W, node_radius=6, edge_radius=3)

# Batch across a frame or the whole dataset
labels.convert(to="bbox", source="mask", padding=4, inplace=True)

# Round-trip a centroid back into a single-node pose
inst2 = centroid.to_pose()

Each of Instance, Centroid, BoundingBox, SegmentationMask, and ROI gains to_centroid(), to_bbox(), to_roi(), and to_mask() (with Centroid.to_pose() / from_pose() closing the loop), and LabeledFrame.convert(to, source, ...) / Labels.convert(...) reach every cell of the matrix in one call. Conversions preserve the User/Predicted variant (carrying score on predicted paths) and propagate track, tracking_score, identity / identity_score / identity_embedding, and an instance= backref. Degenerate inputs return an empty target object (check the new is_empty property) unless you pass error_on_empty=True.

As part of this, SegmentationMask.to_polygon() now returns a PredictedROI (with score) for predicted masks instead of downcasting to UserROI. The old Centroid.to_instance / from_instance names are deprecated in favor of to_pose / from_pose.

Infer left/right symmetries from node names (#534)

Skeleton.infer_symmetries_by_name() suggests symmetric node pairs from names, so flip augmentation and QC work even for skeletons imported without symmetry metadata.

import sleap_io as sio

skel = sio.Skeleton(["nose", "eye_L", "eye_R", "ear_L", "ear_R"])
skel.infer_symmetries_by_name()                 # [(1, 2), (3, 4)]

# Non-mutating: review, then apply explicitly
skel.add_symmetries(skel.infer_symmetries_by_name())

# Name-only helper (no Skeleton needed); custom tokens supported
sio.infer_symmetry_pairs_by_name(["front_left_paw", "front_right_paw", "tail"])  # [(0, 1)]

Recognizes left/right and l/r tokens (configurable via token_pairs) as whole name segments (eye_L/eye_R, left_paw/right_paw, L1/R1), pairing only unambiguous 1:1 stems. It is opt-in and non-mutating — you apply the suggestions with add_symmetries(). Contributed by @tom21100227.

sio.download() and sio download CLI (#528)

v0.8.0 added remote reading; v0.9.0 adds a primitive to fetch a remote file straight to disk without loading it — useful for large videos or formats you load separately.

import sleap_io as sio

sio.download("https://example.com/labels.slp")       # -> ./labels.slp
sio.download("s3://bucket/run/video.mp4", "data/")    # -> data/video.mp4
sio.download(url, headers={"Authorization": "Bearer <token>"})

# Fetch-then-load for formats not loadable directly over a URL
labels = sio.load_nwb(sio.download("https://example.com/labels.nwb"))

sio download https://example.com/labels.slp
sio download s3://bucket/run/video.mp4 data/
sio download https://example.com/a.slp out.slp -H 'Authorization: Bearer <token>' -f

Supports the same schemes as the loaders — http(s), cloud buckets (s3/gs/gcs/az/ abfs, needs the [cloud] extra), and Google Drive share links — with streaming to disk, atomic writes, and idempotent skip-if-exists (overwrite=True to force). No new dependencies.

Machine-readable CLI inspection: --json (#538)

sio show --json prints a structured document (path/name/size/format, a stats block, and full skeletons, videos, tracks, identities, event_types, events, and provenance sections); for a standalone video it prints a video-shaped payload. sio filenames --json prints a per-video filename + provenance listing (inspection mode only).

sio show labels.slp --json
sio show labels.slp --json --lf 0        # add per-instance point detail for one frame
sio show labels.slp --frames             # per-frame listing (works with or without --json)
sio filenames labels.slp --json

The default human-readable output is unchanged when --json is omitted.

---

Improvements

Reliable saving for very large projects (#517, #521, #523, #524)

Hardens .slp saving against HDF5's hard 64 KB per-attribute limit (issue #516), which could make large projects fail to save or silently lose metadata:

• #517 — provenance is written to a dedicated /provenance_json dataset instead of the metadata/json attribute. read_provenance falls back to the legacy attribute, so old files still load unchanged.
• #521 — Labels.merge() bounds merge_history (see Breaking Changes).
• #523 — oversized per-video source_video metadata spills from its attribute into a dataset (with a warning) when it exceeds 64 KB.
• #524 — a new opt-in save_slp(..., preserve_unknown=True) carries unrecognized top-level HDF5 datasets/groups across a load/save cycle for forward compatibility with newer sleap-io versions.

labels = sio.load_slp("from_newer_version.slp")
sio.save_slp(labels, "out.slp", preserve_unknown=True)

These are read-backward-compatible layout changes — no SLP format-version bump.

Faster Labels.merge() (#537)

Appending merges no longer invalidate and rebuild the frame-lookup index on every frame; the warm index is now updated in place, turning large merges from O(N²) into O(N) (a reported ~218s → ~0.75s for a 9,000-frame merge into a ~95k-frame project). Merged output is byte-identical. Contributed by @yixi0527.

Clearer embed errors (#530)

Embedding an out-of-range frame now raises an IndexError that names the offending video (index, filename, and frame count) instead of reporting only the frame index — actionable in multi-video projects. Contributed by @gitttt-1234.

---

Fixes

• #532 — NWB export is compatible with pynwb 4.0, which now requires num_samples on rate-based external ImageSeries. The writer sets it from the video frame count and passes it only when the installed pynwb accepts it, so pynwb < 4 is still supported.

---

Documentation

• New docs/model/events.md (frame-spanning events) and docs/model/embedding.md (re-ID identities and embeddings).
• docs/formats/slp.md — format-version history extended through 2.6.
• docs/model/{poses,centroids,boxes,rois,segmentation}.md — modality interconversion verbs and examples.
• docs/cli.md — download, show --json / --frames, filenames --json, and merge --identity.
• docs/model/3d.md, docs/model/labels.md, docs/merging.md, docs/remote.md, docs/examples.md updated.

---

Known Issues

• Events persist only in .slp. Converting to NWB, Label Studio, JABS, or a DataFrame drops events and event types; sio convert warns when this happens.

---

Changelog

Reflects net user-facing changes; intra-cycle refactors that net to no change are folded into their final form.

• #513: feat(model): global re-ID Identity with cross-file name matching (@talmo)
• #514: feat(model,io,cli): persist per-instance Identity, with merge and CLI support (@talmo)
• #515: feat(model,io): re-ID Embedding data model + SLP persistence (@talmo)
• #517: fix(io): store provenance in a /provenance_json dataset to dodge HDF5's 64 KB attribute limit (@talmo)
• #521: fix(model): bound merge_history growth in Labels.merge() (default cap 1000) (@talmo)
• #523: fix(io): spill oversized source_video metadata to a dataset (@talmo)
• #524: feat(io): opt-in preserve_unknown carry-over of unknown HDF5 datasets on save (@talmo)
• #527: feat(io,model): persist identity + re-ID embeddings across all detection modalities (@talmo)
• #528: feat(io): add sio.download() + sio download CLI for fetching remote files (@talmo)
• #530: fix(io): name the video in out-of-range embed errors (@gitttt-1234)
• #531: feat(model): unified interconversion between detection modalities (pose ⇄ centroid ⇄ bbox ⇄ mask/ROI) (@talmo)
• #532: fix(io): set ImageSeries num_samples for pynwb 4 compatibility (@talmo)
• #534: feat(skeleton): infer left/right symmetries from node names (@tom21100227)
• #535: refactor(model,io): finalize the re-ID identity/embedding data model (SLP format 2.5) (@talmo)
• #536: feat(io): default save_embedding_vectors to False (off by default, like embed) (@talmo)
• #537: perf(model): keep the frame index warm during merge for O(N) appending merges (@yixi0527)
• #538: feat(cli): add --json output to show and filenames for machine-readable inspection (@talmo)
• #540: feat: frame-spanning Event annotations (data model + Labels + SLP format 2.6) (@talmo)

Full Changelog: v0.8.0...v0.9.0