Skip to content

SLEAP v1.6.4

Latest
Latest

Choose a tag to compare

View all tags
@gitttt-1234 gitttt-1234 released this 28 Jul 07:13
· 5 commits to develop since this release
v1.6.4
cda4a22
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

SLEAP v1.6.4

SLEAP v1.6.4 is a release packed with GUI improvements — Label QC label-error detection, a negative-frame training workflow, Merge Instance, motion-trail rendering controls, and per-instance visibility tools — alongside 25 bug fixes covering frame selection, skeleton persistence, drag-and-drop, Wayland/Windows dialog issues, and inference-pipeline crashes, plus updated dependencies (sleap-io 0.9.2 with re-identification, Category, and Event annotations; sleap-nn 0.3.1 with instance segmentation, sliding-window tiling, and pretrained backbones).

Quick install/upgrade:

uv tool install --python 3.13 "sleap[nn]==1.6.4" --torch-backend auto

See the v1.6.0 release notes for full details on the latest major release.

New Features

Add optional "Mean Node Score" column to Instances panel (#2696)

Adds an optional "Mean Node Score" column to the Instances panel, showing each predicted instance's mean point confidence over its visible nodes, mirroring sleap-nn's filter_min_mean_node_score filter. The column is hidden by default and toggled from View > Show Mean Node Score, giving users GUI visibility into a value previously only accessible via the CLI.

Improve loss plot y-axis scaling and add batch subsample dropdown (#2699)

Replaces a hard-coded floor in the training loss plot's log-scale y-axis calculation with log-space-aware padding so the loss curve fills the plot instead of being squeezed into a thin sliver, and adds a "Batch Subsample" dropdown (1/10/100) that thins out batch-loss scatter points for faster rendering on long training runs without discarding the underlying data.

Add "Include unlabeled frames" option to Render Video Clip dialog (#2701)

Adds an "Include unlabeled frames in range" checkbox to the Render Video Clip dialog, wiring the GUI through to sleap-io's existing full-range rendering semantics so exported clips can include every frame in a range (with overlays only where instances exist) instead of just labeled frames — previously only available via sio render --all-frames.

Add Accept All Predictions bulk action (#2702)

Adds an "Accept All Predictions..." action under the Labels menu that converts every unused predicted instance across the whole project into a user instance in one step, complementing the existing per-frame "Add Instances from All Predictions" command.

Expose peak threshold in inference dialog (#2704)

Exposes a Peak Threshold field (with a "Default" checkbox) in the GUI inference dialog, letting users tune the minimum confidence for node detections without dropping to the CLI's --peak_threshold flag.

Show current epoch runtime in training monitor (#2705)

The training monitor now shows per-epoch runtime alongside total runtime in the plot title, making it easy to spot a stalled epoch without manual arithmetic.

Add Previous User Labeled Frame navigation (Ctrl+Shift+U) (#2706)

Adds a "Previous User Labeled Frame" command that mirrors the existing "Next User Labeled Frame" (Ctrl+U), letting users navigate backward through frames containing user-created instances.

Add frame range filter for labeling suggestions (#2707)

Labeling suggestions can now be restricted to an explicit From/To frame range when the target is "Current Video," applying to every suggestion method except frame-chunk (which already has its own range control).

Add Actual Size (1:1) view option (#2708)

Adds a "View > Actual Size (1:1)" toggle that displays the video at native resolution instead of SLEAP's default zoom-to-fit, addressing complaints that low-resolution videos looked overly pixelated when upsampled to fit the window.

Add keyboard shortcut for Propagate Track Labels toggle (#2714)

Adds a default P keyboard shortcut for the "Propagate Track Labels" toggle, so users can flip that setting during proofreading without reaching for the Tracks menu each time.

Add Replace Videos button to Videos dock (#2715)

Adds a "Replace Videos" button to the Videos dock, alongside "Add Videos" and "Remove Video," so replacing a project's video no longer requires a trip to the File menu.

Negative frames — mark empty frames as background training examples (#2716)

Adds a full GUI workflow for marking a frame as a "negative frame" (confirmed-empty, no animals), including a toggle command, seekbar/canvas visual indicators, two new training-config options (Use Negative Frames, Negative Loss Weight), and a Label QC safety check. This closes a GUI gap for a capability sleap-io and sleap-nn already supported but that had been requested since 2022 (#640).

Label QC label-error detectors + GUI (#2770)

Adds a full Label QC label-error detection suite — five new detectors (left/right flip, chimera instances, duplicate/split instances, chain-order errors, missing nodes) — plus a revamped GUI with per-detector toggles, threshold sliders, filter-by-issue-type, and reviewed-state tracking. Also includes a spine-relative flip-detection fix that cut false-positive flip flags from 7.6% to 2.0% on real-world data.

Per-instance visibility & view-only toggles in the Instances dock (#2772)

Adds per-instance visibility and "view only" checkboxes to the Instances dock, letting users hide/show individual instances on the canvas or isolate a single instance for inspection without touching the saved project data.

Merge Instance — combine two partial instances into one (#2773)

Adds a "Merge Instance" action that combines two partially-labeled user instances in the same frame (e.g., one with anterior keypoints, one with posterior) into a single instance via shift/ctrl-select in the Instances dock, taking the union of their labeled nodes.

Per-instance & Label QC node-visibility controls (#2784)

Bundles three related Label GUI improvements on a shared per-instance node-visibility model: a Shift+V shortcut for "Show Non-Visible Nodes," a per-instance "Invisible Nodes" checkbox in the Instances dock, and a Label QC "Display" mode selector to reduce clutter when reviewing crowded frames.

Guide crop size / input scaling for top-down models (#2793)

Adds inline guidance to the training config dialog to steer users away from common top-down model mistakes — an info button explaining that input scaling should usually stay at 1.0 for centered-instance models, plus warnings when the effective crop size would fall below 100px or below the largest labeled instance. Also fixes a latent bug where validation warnings were rendered below the fold and effectively invisible.

Expose motion-trail options in the render/export dialog (#2819)

Adds a "Motion Trails" section to the Render Video Clip export dialog, exposing sleap-io's trail rendering options (length, node, width, opacity, fade, color) directly in the GUI so users no longer need the sio render --trails CLI to produce trailed videos.

Reframe live trail overlay onto sleap-io vocabulary (#2827)

Reframes the live in-player trail overlay to match sleap-io's trail semantics: trails now render for untracked/single-instance projects (previously required a Track and showed nothing), a new Trail Node menu lets users pick which node the trail follows, and fade is now a true alpha gradient. Replaces the removed "Trail Shade" preference with a "Trail Opacity" control.

CLI Updates

Show build commit hash in sleap doctor and startup banner (#2767)

sleap doctor and the startup banner now show the exact git commit a SLEAP install was built from (for editable and git-URL installs), and a new sleap doctor --commit flag resolves the commit for PyPI/conda releases via the GitHub API — useful for diagnosing installs that came from GitHub rather than a tagged release.

Bug Fixes

Fix Frame Selection and Frame Count in GUI (Shift + Drag and Shift + Double Click) (#2078)

Fixed Shift+Drag and Shift+Double-Click frame selection on the seekbar producing a frame count that didn't match the video's actual frame count, caused by an incorrect slider-width calculation and an extra +1 in the frame-count display.

Fix #2684: persist skeleton on save when no instances are labeled (#2686)

Fixed a skeleton edited only via the New Node/New Edge buttons being silently dropped on save when the project had no labeled instances yet, because the GUI's in-progress skeleton was never attached to labels.skeletons until an instance existed.

Fix blank progress dialog during Generate Suggestions (#2695)

Fixed "Generate Suggestions" blocking the main GUI thread and rendering its progress dialog as a black rectangle (Linux/Wayland) or triggering the spinning beach ball (macOS); suggestion generation now runs on a background thread with a responsive progress dialog.

Fix Copy Prior Frame copying stale prediction instead of user correction (#2698)

Fixed "Copy Prior Frame" landing on a stale, uncorrected prediction instead of the user's correction when both existed in the prior frame; the copy source now prefers user instances over unused predictions.

Fix parent widget on LearningDialog to fix window stacking on Windows (#2703)

Fixed training/inference dialogs appearing behind the main SLEAP window on Windows by giving LearningDialog a parent widget so Qt can manage window stacking correctly.

Clarify Delete Predictions beyond Frame Limit dialog text (#2711)

Clarified the "Delete Predictions beyond Frame Limit" dialog, whose title and field labels previously implied instances inside the range would be deleted — the opposite of the actual behavior.

Skip predicted instances in crop size auto-computation (#2717)

Crop size auto-computation in the training dialog now ignores PredictedInstance objects and accounts for augmentation padding, fixing cases where a handful of outlier predictions (or missing rotation/scale padding) skewed the auto-computed crop size away from what sleap-nn actually uses during training.

Fix single-node skeleton new-instance IndexError (#2719)

Fixed a crash (silently swallowed as a no-op) when creating a new instance on a single-node skeleton after at least one instance already existed, caused by an IndexError in the node-pair alignment helper that assumes at least two nodes.

Handle list-valued Video.filename (ImageVideo) in Label QC detector (#2739)

Fixed Label QC's "Run Analysis" crashing with TypeError: unhashable type: 'list' on projects using the ImageVideo backend (image-sequence imports from CVAT/COCO), where Video.filename is a list rather than a string.

Keep videos table populated when a video frame fails to read (#2743)

Fixed a single video that intermittently fails to read its first frame (e.g., a flaky network drive on Windows) blanking the entire Videos table instead of just that row.

Label QC only analyzes user-labeled instances (#2746)

Fixed Label QC flagging unreviewed model predictions on suggested frames instead of analyzing only manually-confirmed ground truth; also fixes a related bug where clicking a flagged row could jump to the wrong instance.

Fix misleading error when running inference on empty project (#2750)

Fixed a misleading "You have unsaved changes" error shown when running inference/training on a brand-new, never-saved project; the pre-flight check now distinguishes "no videos," "never saved," and "unsaved changes" and shows the message that matches the project's actual state.

Refresh Label QC dock on project load (#2754)

Fixed the Label QC dock reporting "Need at least 2 instances to run QC analysis" immediately after opening a project that clearly has labeled instances, because the dock's cached Labels reference wasn't refreshed on every project load.

Force invisible_as_nan in Label QC geometry (#2759)

Fixed Label QC's geometry computation relying on sleap-io's default handling of invisible nodes, which could leak placeholder display coordinates into edge/angle/distance statistics and produce spurious "unusual joint angle"/"unusual edge length" flags.

Accept drag-and-drop file opens on Linux/macOS (#2761)

Fixed dragging a .slp file or video onto the main SLEAP window doing nothing on Linux and macOS (it only worked on Windows), because the drag-accept check looked for a Windows-only MIME type instead of the cross-platform text/uri-list.

Spread occluded nodes via force-directed layout when converting predictions (#2771)

Fixed occluded/undetected nodes rendering at a garbage default location when "show non-visible nodes" was enabled after converting predictions to user instances; hidden nodes are now spread across the skeleton body via a force-directed layout while staying hidden.

Set parent widget on seekbar tooltip to fix Wayland popup warnings (#2780)

Fixed Wayland flooding stderr with "Failed to create popup" warnings on every seekbar hover, by giving the per-frame tooltip an explicit parent widget so Qt can establish the required transient-parent relationship.

Warn when picked inference config has no trained model (#2788)

Fixed the inference dialog's model-config dropdown silently staying empty with the Run button greyed out when a picked training config had no trained checkpoint; the GUI now warns explicitly and also recognizes checkpoints stored in a custom ckpt_dir.

Render videos table row when a video backend fails to open (#2799)

Fixed a crash (AttributeError: 'NoneType' object has no attribute 'filename') in the Videos table when a video's backend failed to open (e.g., a momentary lock or slow network drive), which could abort an active-learning session right after training.

Decode inference subprocess output as UTF-8 (#2801)

Fixed the GUI inference runner decoding subprocess output with the OS locale encoding (cp1252 on Windows), which crashed with UnicodeDecodeError on certain Unicode output and masked the real underlying inference error.

Use points_array compatibility shim in DeleteAreaPredictions (#2822)

Fixed a crash in "Delete Predictions from Area" (AttributeError: 'PredictedInstance' object has no attribute 'points_array') caused by a call site that hadn't been migrated to the compatibility shim after sleap-io removed the points_array attribute.

Use current sleap-io API in plot_instance edge drawing (#2823)

Fixed sleap.util.plot_instance() raising AttributeError when drawing skeleton edges, due to two more leftover call sites from the sleap-io API migration; this public function isn't wired into any current GUI/CLI path but is used by external scripts.

Properly quote Hydra overrides in exported train-script.sh (#2829)

Fixed Hydra override values containing an = (such as auto-generated run names) still raising OverrideParseException in exported train-script.sh, because a prior fix (#2612) wrapped values in shell quotes that bash strips before Hydra ever sees them. Overrides are now quoted for Hydra itself and separately shell-escaped so both layers see a valid token.

Treat trainer_accelerator as machine-specific when loading a saved profile (#2830)

Fixed a saved training profile's trainer_accelerator value (e.g., mps from a Mac) carrying over verbatim when reloaded on a different machine (e.g., Linux + CUDA or CPU-only), by treating it as machine-specific like the existing trainer_devices/num_workers handling.

Default training config file dialog to show JSON and YAML together (#2832)

Fixed the training-config file picker defaulting to a "JSON only" filter that hid .yaml/.yml files until manually switched; a combined "Config files" filter is now the default.

Other

Serialize gh-pages pushes across docs and PR-preview workflows (#2690)

Fixed the docs deployment pipeline racing itself — the "Deploy MkDocs" and "Docs PR Preview" workflows both pushed to gh-pages under mismatched concurrency groups, causing rejected pushes on most develop merges for about two months.

Use official uv Windows install, bump version pins, fix broken nn.sleap.ai links (#2693)

Documentation fix: switched the Windows uv install command to the official form that explicitly bypasses PowerShell's execution policy (the old command could silently close PowerShell on restricted systems), bumped pinned version examples to current releases, and repaired 11 broken nn.sleap.ai links.

Align license section with README (#2700)

Brought the docs.sleap.ai landing page's license section in line with the README and the actual Clear BSD LICENSE file, removing outdated "research/academic use only" language.

Add workflow_dispatch to manually redeploy versioned docs (#2712)

Added a manual trigger to the docs deployment workflow so a published release's docs (e.g., a stale latest alias) can be redeployed from develop without waiting for a new release or push.

Remove 24 orphaned " copy" image duplicates (#2713)

Removed 24 unreferenced " copy" image duplicates (~12 MB) from the docs assets, trimming the size of every future gh-pages build.

Rename Debug Mode menu to Experimental Features (#2725)

Renamed the Help menu's "Debug Mode" toggle to "Experimental Features" (and its backing state key) as groundwork for gating upcoming experimental features behind the same switch; a pure rename with no behavior change.

Streamline installation page with one-liner commands and OS auto-detect (#2730)

Restructured the installation docs page around a TL;DR block with one-liner install/upgrade/develop commands, collapsed redundant sections, and added OS auto-detection to Windows/macOS/Linux tabs site-wide.

Split TL;DR install commands into per-command copy blocks (#2731)

Split the installation TL;DR's combined install/upgrade/develop code block into three separate blocks so each command's copy button copies only that command instead of all three at once.

Dependency Updates

sleap-io 0.7.1 → 0.9.2

  • ⚠️ Breaking: merge track matching now defaults to identity, not name — Labels.merge()/Labels.match() (and sio merge) no longer collapse tracks just because they share a name; pass track="name" to restore the old name-collapsing behavior.
  • ⚠️ Breaking: COCO polygon segmentation now rasterizes to masks by default — sio.load_coco() returns a UserSegmentationMask per annotation instead of vector UserROIs; pass segmentation_format="roi" to keep the old representation.
  • ⚠️ Breaking: Identity.color removed — color now lives in Identity.metadata (a dict[str, str]) instead of a dedicated field.
  • ⚠️ Breaking: .category is now a Category object, not a plain string — construction (category="mouse") still works via an implicit converter, but reading it back requires .category.name.
  • Re-identification subsystem — new Identity (cross-file ground-truth identity) and Embedding (per-detection appearance vector) types attach to every detection modality, collect into Labels.identities, and persist to .slp; appearance vectors stay off-disk unless save_embedding_vectors=True.
  • First-class Category — a third grouping axis alongside Track and Identity for class/object-detector labels, with its own catalog, scores, embeddings, and render --color-by category.
  • Frame-spanning Event annotations — sleap-io's first annotation with a temporal extent for behavior bouts, stimulus epochs, or review flags; persists only to .slp.
  • Unified modality interconversion — pose, centroid, bounding box, segmentation mask, and ROI share one verb set (.to_centroid(), .to_bbox(), .to_roi(), .to_mask()) plus batch convert() entry points.
  • Remote loading — .slp and video loadable directly from http(s), s3, gs/gcs, az/abfs, and Google Drive share links; new sio.download()/sio download fetches a remote file to disk.
  • DeepLabCut project import — sio.load_dlc_project() imports a full DLC project (skeleton edges, source videos, train/test splits).
  • COCO instance segmentation — the COCO reader imports polygon/RLE segmentation as SegmentationMask annotations, with optional category→track mapping.
  • Virtual on-read video cropping — Video.crop() provides a cropped view without re-encoding, round-trips through .slp, and bakes to real files via sio apply-crops.
  • Major I/O scalability fix (multi-camera sessions) — RecordingSession 3D point data moved out of the monolithic sessions_json string into a columnar /session_data HDF5 group; a real 108k-frame, 3-camera project's sessions_json shrank from 524 MB to single digits of MB, fixing files that were unreadable in browser-based (h5wasm/JS) tooling.
  • Embedding-write performance fix — a 15,000-frame project's .pkg.slp embedding time dropped from ~32 min to ~4 s by dropping gzip on already-compressed frame bytes; image-sequence embedding also gained a lossless byte-copy fast path.
  • Faster merges — appending merges are now O(N) instead of O(N²) (~218 s → ~0.75 s on a 9k-frame merge).
  • Reliability for very large projects — .slp saving is hardened against HDF5's 64 KB per-attribute limit for provenance, merge history (capped at 1000 records by default), and per-video source metadata.
  • Data-preservation fixes — analysis HDF5 export no longer drops instances in untracked multi-animal projects; merge preserves the is_negative background-frame flag; DLC/COCO readers retain empty frames; foreign Video/filename lookups resolve via a new Labels.match_video().
  • Other additions — motion trail overlays for rendering, name-based skeleton symmetry inference, machine-readable sio show --json, gzip-compressed mask/ROI datasets, and pynwb 4 compatibility.

See the sleap-io v0.9.2 release notes for full details (and the linked v0.7.1/v0.8.0/v0.9.0/v0.9.1 notes for the intermediate steps).

sleap-nn 0.2.0 → 0.3.1

  • ⚠️ Breaking: retrain required for whole-frame segmentation models — semantic_segmentation and bottomup_segmentation models trained before v0.3.1 have GT masks misaligned with the image grid; affected models must be retrained.
  • ⚠️ Breaking: mask-evaluation default metrics changed — run_evaluation(match_method="mask") now excludes GT masks linked to a PredictedInstance by default, raising default recall/F1 on affected files; pass user_labels_only=False to restore old behavior.
  • ⚠️ Breaking: sleap-nn predict now means pose inference — in v0.2.0, predict ran exported ONNX/TensorRT models and track ran pose inference; that's now reversed. track still works as a legacy command; exported-model inference moves to sleap-nn predict --runtime onnx|tensorrt / Predictor.from_export_dir(...).
  • ⚠️ Breaking: prediction .slp output is non-embedded by default — predictions reference the original source videos instead of writing a self-contained .pkg.slp; pass --embed true to restore embedding.
  • ⚠️ Breaking: training seed now defaults to 42 (was unset) — changes train/val split RNG versus v0.2.0; set seed: null for the old fully-random behavior.
  • ⚠️ Breaking: frame-caching failures during training now hard-fail instead of warning and continuing.
  • ⚠️ Breaking: centroid target-source resolution changed — centroid_source is now resolved once per dataset rather than per frame, which can change the effective training set for mixed-annotation datasets.
  • New unified sleap-nn predict command + Predictor Python API — a single entry point from model dir(s) + data to sio.Labels, with streaming, raw-tensor access, and in-memory frame prediction.
  • Centroid-only models are first-class — train/predict/evaluate a lone centroid head end-to-end, and (0.3.1) train directly from UserCentroid annotations, including centroid-only frames (active-learning workflows).
  • Instance segmentation (experimental in 0.3.0, hardened in 0.3.1) — bottom-up, top-down, and SAM-prompted backends; 0.3.1 adds a whole-frame semantic_segmentation model type and a clDice mask metric.
  • Sliding-window tiling (0.3.1) — cuts 4K++ frames with small objects into overlapping tiles for native-resolution inference.
  • Pretrained HuggingFace backbones (0.3.1) — any AutoBackbone (ConvNeXtV2, ResNet, Swinv2, DINOv2/v3, ...) usable as an encoder via sleap-nn[backbones].
  • Kalman tracking — --use_kalman joins optical-flow shift tracking.
  • Restored CPU/GPU overlap in predict (0.3.1) — a prior refactor had silently made frame decoding synchronous with GPU inference; overlap is restored.
  • GPU (CUDA 13/cu130) is now the default backend, with remote-URL --data_path and repeatable --output_format.
  • ⚠️ sleap-io dependency bumped to >=0.9.2,<0.10.0 — audited through sleap-io's re-ID/Category/Event annotations and large-project save hardening; sleap-io's breaking changes there don't affect sleap-nn code paths.

See the sleap-nn v0.3.0 and v0.3.1 release notes for full details.

Full Changelog: v1.6.3...v1.6.4

Assets 2
Loading