sleap-io v0.9.1

sleap-io v0.9.1 Release Notes

Summary

sleap-io v0.9.1 is a focused patch release pairing one new modeling concept with a major I/O performance fix. It adds first-class Category — class membership as a third grouping axis alongside Track and Identity — and eliminates a tens-of-minutes silent stall when embedding image-sequence frames into a .pkg.slp (a real 15,000-frame project drops from ~32 minutes to ~4 seconds). Two small, mechanical breaking changes ride along: .category on boxes/centroids/ROIs/masks is now a Category object rather than a bare string, and the SLP embed progress_callback gains a third phase argument.

The SLP format advances 2.6 → 2.7 — 2.7 adds the /categories group (mirroring identity). It is additive and read-on-group-presence, so older readers ignore it and pre-2.7 files load unchanged (identity-only files stay byte-identical).

Highlights:

• First-class Category (#542) — a named catalog entry (name + string metadata + matches()) mirroring Identity, attached per detection via category / category_score / category_embedding, collected into Labels.categories, colored by render --color-by category, and persisted to .slp (format 2.7). Motivated by the move toward object detection, where the object class (COCO "category") is a first-class citizen.
• Embedding writes are no longer slow or silent (#543) — dropping gzip on already-compressed frame bytes turns a tens-of-minutes post-progress-bar stall into seconds; a real 15,000-frame image sequence went ~32 min → ~4 s end-to-end.
• ImageVideo byte-copy fast path (#543) — embedding a PNG/JPEG image sequence now copies the source bytes verbatim: lossless (no re-encode artifacts), faster (no decode/encode), and smaller (~3.9 GB → ~2.0 GB on that project).
• Write-phase progress (#543) — a second Writing frames progress bar (and a "write" phase for progress_callback) so the write phase no longer looks hung.

Installation / Upgrade

# One-off CLI usage (no installation needed)
uvx sleap-io@0.9.1 show labels.slp

# Install as CLI tool (new install)
uv tool install "sleap-io[all]"

# Upgrade existing CLI tool installation
uv tool upgrade sleap-io

# Add to project (new dependency)
uv add "sleap-io[all]"

# Upgrade existing project dependency
uv lock --upgrade-package sleap-io && uv sync

See installation docs for more options.

---

Breaking Changes

v0.9.1 is a patch release with two small, mechanical breaking changes.

.category is now a Category, not a str (#542)

The pre-existing free-form category: str field on BoundingBox, Centroid, ROI, and SegmentationMask is promoted to category: Category | None. Construction is unchanged — a str → Category converter keeps existing code working — but code that reads .category as a string must now use .category.name (and treat None as unset).

# Construction is unchanged — a bare class-label string still works
bbox = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10, category="mouse")

# Before (0.9.0): a plain string
bbox.category            # "mouse"

# After (0.9.1): a first-class Category (empty string -> None)
bbox.category            # Category(name="mouse")
bbox.category.name       # "mouse"

The on-disk SLP string format is unchanged — the bbox/centroid/roi_categories/mask_categories datasets still store the plain name string, so files round-trip identically. LabelImage.Info.category also remains a plain str (it belongs to the separate panoptic-mask subsystem and is unaffected).

SLP embed progress_callback gains a phase argument (#543)

The embed progress_callback is now invoked in two phases and takes a third argument:

# Before:  progress_callback(current, total) -> bool
# After:   progress_callback(current, total, phase) -> bool   # phase: "embed" | "write"

"embed" fires while frames are loaded/encoded/byte-copied into memory; "write" fires while bytes are flushed to the HDF5 file (the newly-instrumented write phase). Returning a truthy value from either phase still cancels the export.

Any caller passing a 2-argument callback to write_labels / save_slp / save_file will break and must accept the third phase argument. The SLEAP GUI is the main such caller; it is version-fenced to an older sleap-io, so this is safe until that pin is bumped.

---

New Features

First-class Category: class membership alongside Track and Identity (#542)

Category completes a trio of grouping axes for detections. Where Track links the same individual within a video and Identity links the same individual across videos, Category groups individuals of the same class — the natural home for a classifier or object-detector's class label.

Concept	Scope	Example
Track	same individual within a video (tracking)	track_0
Identity	same individual across videos (unique ID)	C57BL6.cohort2.12
Category	individuals of the same class (classified / detected)	female_fly, fur_shaved

Category is a direct mirror of Identity: a named catalog entry with arbitrary string metadata, matched by name (default) or object identity. Every detection modality (Instance, PredictedInstance, BoundingBox, Centroid, ROI, SegmentationMask) gains category, category_score, and category_embedding slots; InstanceGroup gains a bare category. Labels.categories is a catalog auto-collected from the detections, exactly like Labels.identities.

import numpy as np
import sleap_io as sio

# The promoted free-form class-label string still works, and is now a Category
bbox = sio.UserBoundingBox(x1=0, y1=0, x2=10, y2=10, category="mouse")
bbox.category                    # Category(name="mouse")

# Full first-class use: catalog entry + per-detection score + embedding
fly = sio.Category(name="female_fly", metadata={"sex": "F"})
inst = sio.Instance.from_numpy(
    np.array([[10.0, 20.0], [30.0, 40.0]]),
    skeleton=skeleton,
    category=fly,
    category_score=0.94,
    category_embedding=sio.Embedding(np.random.rand(128).astype("float32")),
)

labels = sio.Labels(
    labeled_frames=[sio.LabeledFrame(video=video, frame_idx=0, instances=[inst])]
)
labels.categories                # [Category(name="female_fly")]

# Categories match by name, like Identity and Track
assert fly.matches(sio.Category(name="female_fly"))

sio.save_slp(labels, "out.slp", save_embedding_vectors=True)   # format 2.7

Persistence (SLP format 2.7). Category links are written to a new self-contained /categories group (a name catalog, an EAV metadata table, and a links dataset carrying category_idx / category_score), mirroring /identity. Category embeddings are stored as parallel category_* datasets alongside the identity vectors in /embeddings, so the two embedding kinds are independent and need not share dimensionality. As with identity/re-ID embeddings, appearance vectors stay off disk unless you pass save_embedding_vectors=True (default False, mirroring embed).

Tooling. sio merge --category {name,identity} controls how the category catalog is deduplicated on merge; sio render --color-by category colors renders by class; and sio show reports category counts (--json includes n_categories, a categories[] listing, and n_instances_with_category_embedding).

sio render preds.slp -o out.mp4 --color-by category
sio merge a.slp b.slp -o merged.slp --category name
sio show preds.slp --json          # includes n_categories, categories[], ...

Because the class label was already a string field, every reader and writer that consumed it — the COCO, Ultralytics, and JABS readers/writers, the GeoJSON writer, roi.__geo_interface__, and the get_*(category=...) filters — now reads .category.name, so those formats round-trip unchanged. See docs/model/category.md.

---

Improvements

Embedding writes are no longer slow or silent (#543)

Embedding video frames into a .pkg.slp could spend tens of minutes silently writing to disk after the Embedding frames progress bar had already finished. The cause was gzip on encoded frames: PNG/JPEG bytes are already entropy-coded, so gzip only compressed the fixed-length zero padding while forcing chunked storage. guess_chunk then picked a tall (~100–235 row) chunk, and the row-by-row writes forced a repeated read → decompress → modify → recompress → rewrite of every chunk, compounded by the 1 MB default chunk-cache thrashing.

Encoded frames are now stored uncompressed and contiguous (only the raw-array hdf5 format still uses gzip). Measured on 15,000 frames (0.56 GB of encoded bytes):

strategy	time	file
gzip + auto-chunk (old)	>100 s	0.56 GB
uncompressed contiguous (new)	1.8 s	0.60 GB

The tradeoff is that fixed-length zero padding is no longer compressed away, so files can be modestly larger when per-frame sizes vary widely; fixed_length=False (vlen) remains available for maximum compactness.

ImageVideo byte-copy fast path (#543)

When embedding an image sequence of PNG/JPEG files, the source bytes are now copied verbatim — no decode/re-encode. The embedded dataset's @format follows the source (e.g. jpg) with @channel_order RGB. This is lossless (byte-identical to the source, no added JPEG artifacts), faster (the "Embedding frames" phase speeds up too, since it skips decode+encode), and smaller (it stores the compact source bytes instead of a larger re-encoded PNG). On the 15,000-frame project above, the package shrank from ~3.9 GB (re-encoded PNG) to ~2.0 GB (byte-copied JPEG). MediaVideo (mp4) sources still decode and re-encode to PNG.

sio embed mars_top.slp -o mars_top.pkg.slp
# Embedding frames: 100%|██████████| 15000/15000 [00:01<00:00, 12071 it/s]
# Writing frames:   100%|██████████| 15000/15000 [00:02<00:00,  5578 it/s]

Progress on the write phase (#543)

The HDF5 write loop now reports progress: a second Writing frames tqdm bar on the CLI (shown after Embedding frames) and a "write" phase for progress_callback — so a long write no longer looks like a hang.

---

Fixes

⚠️ Spilled source_video metadata made embedded packages unreadable (#543)

Fixed: For projects whose source_video metadata (e.g. the filename list of a large image sequence) exceeds HDF5's 64 KB attribute limit, that metadata is spilled to a source_video/json dataset (since #516, which only taught the label loader to read it). HDF5Video.__attrs_post_init__ still read the source_video/@json attribute directly, so it raised KeyError: can't locate attribute: 'json', left Video.backend as None, and made the embedded frames unreadable — breaking sio split and frame access on any such package. The backend now reads the dataset-or-attribute, mirroring the loader.

---

Documentation

• New docs/model/category.md (class membership: Category, per-detection slots, catalog, and persistence).
• docs/formats/slp.md — format-version history extended through 2.7 and the embed storage-layout change (uncompressed contiguous encoded frames, ImageVideo byte-copy).
• docs/cli.md — merge --category, render --color-by category, and category reporting in show / --json.

---

Changelog

• #542: feat(model,io,cli): first-class Category (class membership) mirroring Identity (@talmo)
• #543: perf(io): fix slow/silent embed write; ImageVideo byte-copy; write-phase progress (@talmo)

Full Changelog: v0.9.0...v0.9.1