Repository navigation
Changelog
Added
-
Evaluation
- Edit the settings of an evaluation and rerun it, in the evaluation panel or the API.
- Delete an evaluation in the evaluation panel or the API. Annotations stay unchanged, and filters that use the evaluation are removed.
-
Filtering
- Filter by metadata in the query editor, e.g.
metadata["score"] >= 0.8. - Python SDK: Filter images and videos by metadata, including nested fields, e.g.
ImageSampleField.metadata["odometry.speed"] < 15. - Filter captions by tag and metadata in the captions tab.
- Reset all filters with one click with the new "Clear all filters" button.
- Filter by metadata in the query editor, e.g.
-
Annotations
- Copy an annotation to another annotation source in the details view, or create a new source while you copy.
- Set your own keyboard shortcut for AI-assisted labeling in the settings.
-
Other
- Get started quickly with a short onboarding tour on image datasets.
- See why a button is disabled in its tooltip.
- Show sharp pixel edges when you zoom into images with the new image interpolation setting.
- Python SDK: Embed video frames on a remote embedding server registered with
register_default_embedder. - Python SDK: Report anonymous usage data on how data is loaded (format and source). Set
LIGHTLY_STUDIO_ANALYTICS_ENABLED=falseto opt out.
Changed
-
Tags
- BREAKING: A tag now holds only samples of its own collection, and tags no longer have a
kindfield. PostgreSQL migrates automatically. DuckDB users must re-index their data before they create tags. - Adding many samples to a tag is faster.
- BREAKING: A tag now holds only samples of its own collection, and tags no longer have a
-
Embeddings
- BREAKING: Python SDK: Image search in the GUI now needs an embedder that implements
ImageBytesEmbedder. Implement it in your custom embedder to keep image search. - Python SDK: New video datasets embed frames with Perception Encoder (
PE-Core-T16-384) instead of MobileCLIP. Existing datasets are not changed. - Python SDK: Crops and frames are embedded automatically with a previously used compatible remote embedder.
- BREAKING: Python SDK: Image search in the GUI now needs an embedder that implements
-
Embedding plot
- Set the text axes with the new "Define axes" button. The applied texts show at the plot edges, and "Reset axes" returns to PaCMAP.
- Text axes are now balanced and comparable. Lines at x = 0 and y = 0 split the plot into four parts, and axes with weak text matches now look small.
- Points that are filtered out or have no category are easier to see on dark backgrounds.
-
Evaluation
- The evaluation panel has clearer settings and confusion matrix filters. Sorts are grouped by metric, show the highest counts first, and mark missing results.
- Python SDK:
dataset.evaluate()and evaluation recompute are faster on large datasets.
-
Other
- Python SDK:
start_gui()opens the browser. Passopen_browser=False, or uselightly-studio gui --no-browser, to turn this off.
- Python SDK:
Fixed
-
Annotations
- Annotation cards stay aligned, source colors stay stable, and confidence stays visible when you hide annotations or switch to editing.
- Deleting annotations now also deletes their captions.
-
Evaluation
- Evaluation results are kept when a recompute fails.
-
Other
- The grids return to the exact scroll position when you go back from the details view.
- Python SDK:
start_gui()stops cleanly on Ctrl+C, without a traceback. - The GUI footer no longer shows a dev commit hash for released versions.
What's Changed
- Bump lightly-studio-serve version to 0.1.2 by @github-actions[bot] in #2738
- Add: Configurable ai assisted labeling shortcut by @IgorSusmelj in #2730
- Do not skip frames without all components matched by @horatiualmasan in #2739
- fix(point clouds): select inner cuboids and fill on hover/select by @LeonardoRosaa in #2733
- Instruct AI to use pytest instead of make test by @michal-lightly in #2740
- fix: show annotation source name instead of raw ID in point cloud annotation details by @LeonardoRosaa in #2736
- Initialize analytics before GUI startup and handle tracker failures by @LeonardoRosaa in #2635
- Project embeddings onto two axis directions by @mrpositron in #2686
- Hide group item from menu for mcap sequences type by @ikondrat in #2742
- Read MCAP messages from a keyframe through a frame. by @horatiualmasan in #2744
- Require ImageBytesEmbedder for image search by @michal-lightly in #2743
- cache for floci uploads by @ikondrat in #2750
- feat: introduce export schema caching by @ikondrat in #2751
- Decode later frames of a group of pictures without restarting at the keyframe. by @horatiualmasan in #2745
- Cache one video decoder per camera channel. by @horatiualmasan in #2747
- Serve the camera frame at the tick's log time. by @horatiualmasan in #2748
- Show the tick's camera frame in the projection strip. by @horatiualmasan in #2749
- Accept projection axes on the embedding plot and its regions by @mrpositron in #2687
- Detect release commits with annotated tags in version info by @michal-lightly in #2753
- Update 3d scene by @horatiualmasan in #2765
- Clean up point cloud details page by @horatiualmasan in #2764
- Do not open a new tab when clicking a image in the camera strip by @horatiualmasan in #2767
- Add file name to mcap sequence grid by @horatiualmasan in #2768
- Add text-axis inputs for the embedding plot by @mrpositron in #2688
- Resolve embedder by type instead of a getter function by @michal-lightly in #2754
- drop floci caching by @ikondrat in #2773
- Post the release announcement to #releases by @iunir in #2772
- Lay out the embedding plot along two text axes by @mrpositron in #2689
- Fix issue with point cloud scene by @ikondrat in #2774
- Return why EmbedderRegistry has no embedder by @michal-lightly in #2769
- Add Storybook sampling goal prototype by @lukas-lightly in #2775
- Add first-time onboarding tour with driver.js by @LeonardoRosaa in #2755
- Embed video frames on a remote embedder by @gabrielfruet in #2682
- Fix grid scroll position on back navigation by @JonasWurst in #2776
- Add the docs release workflow on GitHub-hosted runners by @iunir in #2777
- Add image sample test helper with metadata by @lukas-lightly in #2783
- feat: Introduce component to render buffer by @ikondrat in #2787
- Add: Implement backend for image interpolation setting by @IgorSusmelj in #2782
- Add dialect-aware JSON scalar type extraction by @lukas-lightly in #2789
- Fix the stale OS Login key cleanup in the docs release by @iunir in #2786
- Support cuboid_3d in create and update annotation APIs by @LeonardoRosaa in #2778
- Reject tagging samples outside the tag's collection, insert tag links in one query by @gabrielfruet in #2756
- Migrate tag links to the collection of their sample by @gabrielfruet in #2757
- Remove the tag kind, now implied by the tag's collection by @gabrielfruet in #2758
- Add empty state guidance in the details view by @LeonardoRosaa in #2780
- Fix: Embedding plot colors barely visible on low contrast screens by @IgorSusmelj in #2801
- feat(point-cloud): create cuboid annotations in the scene by @LeonardoRosaa in #2790
- Introduce prefetch logic by @ikondrat in #2797
- Add a "Clear all filters" button to the filter sidebar by @michal-lightly in #2803
- Replace direct signup links in docs with product page by @michal-lightly in #2812
- Embed video frames with Perception Encoder by default by @gabrielfruet in #2804
- Add string and boolean metadata filters by @lukas-lightly in #2788
- fix: stop reloading each annotation during evaluation by @lorinczszabolcs in #2483
- feat(point cloud): rotate and translate a selected cuboid by @LeonardoRosaa in #2811
- fix issue with jump to another tick during buddering by @ikondrat in #2814
- Add: Count captions in the samples list by @IgorSusmelj in #2816
- Add: Show the caption count in the captions footer by @IgorSusmelj in #2817
- Show the axis lines in the text axes plot layout by @mrpositron in #2802
- Auto-open browser when starting GUI via start_gui by @LeonardoRosaa in #2813
- Embed new crops and frames in the parent collection's space by @gabrielfruet in #2799
- Add: Duplicate an annotation into an existing data source by @IgorSusmelj in #2845
- Add: Numeric metadata filter expressions by @lukas-lightly in #2847
- Add: Create a data source while duplicating an annotation by @IgorSusmelj in #2846
- Stop start_gui() without a traceback on Ctrl+C by @mrpositron in #2850
- Fix: Align annotation cards between viewing and editing by @IgorSusmelj in #2855
- Fix: Simplify evaluation setup and run summaries by @IgorSusmelj in #2866
- Disable preloading for Grid by @ikondrat in #2854
- Add: Assisted labeling provider interface by @iunir in #2822
- Add a changelog skill that loads when CHANGELOG.md is edited by @gabrielfruet in #2821
- Igor explain evaluation sorting and show highest counts first by @IgorSusmelj in #2867
- Move text axes into a popover editor on the embedding plot by @mrpositron in #2848
- Fix issue with jumping during buffering by @ikondrat in #2819
- Add: Metadata filter documentation and video support by @lukas-lightly in #2851
- Show the applied text axes at the plot edges by @mrpositron in #2852
- Add: fal-client as an optional dependency by @iunir in #2825
- Fix: Preserve evaluation results when rerunning by @IgorSusmelj in #2871
- Report ingest source and format in a samples_ingested event by @gabrielfruet in #2868
- Add: Update settings and rerun an existing evaluation by @IgorSusmelj in #2873
- Add: Edit an evaluation from its saved settings by @IgorSusmelj in #2874
- Add: Delete an evaluation and clear its active filters by @IgorSusmelj in #2875
- Document text axes on the embedding plot by @mrpositron in #2870
- Add: Delete an evaluation from the panel and clear its filters by @IgorSusmelj in #2876
- Add: Batched annotation delete resolver by @JonasWurst in #2856
- Small refactor: Remove duplicate Reset axes button by @mrpositron in #2877
- Add: Delete captions of deleted annotation samples by @JonasWurst in #2857
- Update mcap example by @horatiualmasan in #2878
- Fix: Make request timing tests independent of real time by @iunir in #2879
- Embed a sample from MCAP image frames by @horatiualmasan in #2881
- Add: fal.ai SAM 3 assisted labeling provider by @iunir in #2826
- Resize a selected cuboid by dragging its faces by @LeonardoRosaa in #2815
- Fix: Use the annotation box locator in the change label e2e test by @iunir in #2880
- Read the camera keyframes of MCAP sequences by @horatiualmasan in #2883
- Add: Assisted labeling status endpoint by @iunir in #2864
- Add .experimental folder for agentic workflow artifacts by @michal-lightly in #2889
- fix sequence dataset redirect by @horatiualmasan in #2890
- Add: Conversion of assisted labeling prompts and masks by @iunir in #2828
- Remove point cloud feature flag by @horatiualmasan in #2888
- Add: Tag and metadata filters to the captions tab by @IgorSusmelj in #2843
- Embed MCAP sequences when they are indexed by @horatiualmasan in #2885
- Add: Smart select preview endpoint by @iunir in #2829
- Add: Metadata comparison nodes to GUI query expressions by @lukas-lightly in #2893
- Refactor: Delete single annotations through the batched resolver by @JonasWurst in #2858
- Add: Execute metadata filters from GUI query expressions by @lukas-lightly in #2894
- Filter MCAP sequences in the API by @horatiualmasan in #2895
- Get the sample IDs of matching MCAP sequences by @horatiualmasan in #2896
- Fix: Keep annotation colors and confidence stable when hiding by @IgorSusmelj in #2844
- Fix: Cleanup tag filtering in captions view by @IgorSusmelj in #2898
- Unify workspace toolbars into a shared Toolbar component by @LeonardoRosaa in #2891
- Clarify plugin registration modes in docs by @mrpositron in #2903
- Select MCAP sequences on click in the point cloud grid by @horatiualmasan in #2900
- Update plugins docs for SAM3, zero-shot classification and OpenRouter captioning by @mrpositron in #2905
- Filter MCAP sequences by tag in the point cloud grid by @horatiualmasan in #2901
- Add disabledReason prop to Button component by @LeonardoRosaa in #2872
- Add: Metadata comparisons to the GUI query grammar by @lukas-lightly in #2899
- Add: Find all instances preview endpoint by @iunir in #2830
- Search MCAP sequences by text by @horatiualmasan in #2906
- Show MCAP sequences in the embedding plot API by @horatiualmasan in #2907
- Add: Preview and save of assisted labeling results by @iunir in #2832
- Remove: Onboarding tour feature flag by @LeonardoRosaa in #2886
- Show MCAP sequences in the embedding plot by @horatiualmasan in #2909
- Create selections of MCAP sequences in the API by @horatiualmasan in #2910
- Create selections of MCAP sequences in the GUI by @horatiualmasan in #2911
- Add: Shared pop-up and status for assisted labeling tools by @iunir in #2833
- Disable class balancing strategy for mcap dataset by @horatiualmasan in #2913
- Fix: Make confusion matrix filters easier to read by @IgorSusmelj in #2914
- Add: Bulk delete annotations endpoint by @JonasWurst in #2860
- feat(point cloud): add delete button to annotation list items by @LeonardoRosaa in #2820
- Bump lightly-studio version to 1.2.0 by @github-actions[bot] in #2919
Full Changelog: v1.1.3...v1.2.0
Released by @JonasWurst