Repository navigation
v1.1.3
Changelog
Added
- Add
quickstart-enterpriseCLI command to seed a remote enterprise instance with a demo dataset. - Python SDK: Index
.mcaprecordings into a dataset withMcapDataset.load_or_create(...)anddataset.add_mcaps_from_path(...). - Python SDK: Attach 3D cuboids and frame tags from annotation MCAPs with
dataset.add_labels_from_folder(...). A later call skips sequences that already have that annotation source. - Python SDK: Embed search queries on a remote embedding server with
register_remote_embedder. - Add AI-Assisted labeling with adjustable superpixels for creating and editing instance segmentation masks in WASM-capable browsers.
- Python SDK: Enable metric recomputing for instance-segmentation.
- Show the distribution panel on the videos grid in the GUI, with annotation class and metadata distributions.
- Python SDK: Embed images that are added after
register_remote_embedderon the remote embedding server, if the server embeds image bytes. - Python SDK: Store the URL and API key of a remote embedder that
register_default_embedderregistered before an import, so that a later process, such aslightly-studio gui, searches on the server with no new registration. - Python SDK: Embed annotation crops on a remote embedding server that
register_default_embedderregistered, if the server embeds image bytes.
Changed
- Redesign the embedding plot selection tools as visible, sticky Pan, Rectangle, and Lasso buttons in the GUI.
- Show the Rectangle and Lasso drag shortcuts in the embedding plot tool tooltips.
- Show categorical metadata filters in the image sidebar with an optional field picker and expanded value lists.
- Keep the query editor button enabled in the GUI. Clicking "Re-apply" refreshes the results with the current data.
- Move the Tags, Metadata, and Embeddings docs pages under Workflows and add Enterprise sign-up and demo buttons to the docs landing page. The old URLs redirect.
Fixed
- Disable the subpart diversity sampling strategy when no object detection or segmentation annotation sources exist in the collection.
- Combine typicality and similarity computation and sampling into one backend request that continues if the frontend disconnects.
- Fix the
Valuesfield in the distribution panel using a larger font than the other fields. - Fix the focus outline of the bulk annotation panel controls missing its left and right edges.
- Fix instance segmentation masks that cannot be selected with a click when their bounding boxes are hidden in the GUI.
- Fix undoing an annotation deletion restoring it to the default source instead of its original one.
- Return 409 instead of 500 for text and image search on a collection whose embedding space cannot embed the query, and 502 when its embedding server cannot be used.
- Python SDK: Keep the image embeddings stored before an embedder failure, and embed only the images without an embedding when the images are added again with
embed=True.
What's Changed
- Cover RemoteEmbedder protocol and routing edge cases by @iunir in #2388
- Increase Fast Track guardrail timeout by @lukas-lightly in #2436
- fix(gallery): prevent native image drag during selection by @breken-ai in #2310
- Give each remote capability its own timeout by @iunir in #2361
- Run the conformance kit from the command line (conformance kit 2/2) by @iunir in #2433
- Round-trip the remote embedder over a live socket by @iunir in #2362
- feat: introduce tick representation model by @ikondrat in #2440
- provide information about current ticks of mcap sequence.get tick channels by @ikondrat in #2444
- Add method in recording service to return all ticks by @ikondrat in #2449
- feat: introduce get_ticks route function by @ikondrat in #2450
- Replace the embedding plot's built-in tool strip with a sticky Pan/Rectangle/Lasso pill by @gabrielfruet in #2377
- Check remote embedder URLs before connecting by @iunir in #2359
- feat: add instance segmentation to dataset.evaluate() by @lorinczszabolcs in #2400
- add method to query one sample link by sequence id by @ikondrat in #2452
- feat: introduce method to get tick details by @ikondrat in #2453
- feat: introduce api endpoint to provide logs information for given sequence tick by @ikondrat in #2455
- feat: introduce runnable example of indexing mcap from s3 by @ikondrat in #2458
- feat: fix wrong dataset id by @ikondrat in #2459
- feat(point clouds): render camera previews in the MCAP sequence grid by @LeonardoRosaa in #2429
- Add CreateMcap.from_frame_locator by @horatiualmasan in #2464
- Add CreateSensorCalibration from camera intrinsics by @horatiualmasan in #2465
- feat: introduce channel selection component by @ikondrat in #2468
- feat: render timeline frame for channels by @ikondrat in #2461
- Add grid item size control to MCAP grid view by @LeonardoRosaa in #2445
- Build an embedder from the configuration stored on a model row by @iunir in #2446
- Add Annotation QA docs page by @JonasWurst in #2431
- Add McapComponent and look up group components by @horatiualmasan in #2466
- Add a Recording wrapper that stores camera calibrations by @horatiualmasan in #2469
- Fix Values input font size in the distribution panel (#2414) by @rishcx in #2463
- Render channels information info in filtering bar by @ikondrat in #2470
- feat: introduce helper to get frame url by @ikondrat in #2460
- Add annotation details panel to point-cloud workspace by @LeonardoRosaa in #2396
- feat: introduce projection frame component for camera by @ikondrat in #2480
- Use teak details by @ikondrat in #2481
- Add McapSequence to write and read group order by @horatiualmasan in #2471
- Render the Slack announcement as a chat.postMessage payload by @iunir in #2456
- Add the Announce Release workflow by @iunir in #2370
- Camera projection strip by @ikondrat in #2482
- Resolve an embedder from stored config on registry cache miss by @iunir in #2441
- Embed uploaded images from bytes instead of a temp file by @iunir in #2439
- Add McapComponentSpec and the MCAP dataset schema by @horatiualmasan in #2472
- Fix: Typicality and similarity sampling did not finish if frontend disconnects by @IgorSusmelj in #2478
- Add McapDataset.create, load, and load_or_create by @horatiualmasan in #2473
- Add recordings and sequences on McapDataset by @horatiualmasan in #2476
- Extract shared categorical metadata filter by @lukas-lightly in #2488
- introduce types for context by @ikondrat in #2487
- Add a tutorial on exploring and annotating Imagenette with embeddings by @mrpositron in #2454
- Keep a configured backend ahead of a lazily loaded built-in by @iunir in #2447
- Document ImageBytesEmbedder in the API reference by @iunir in #2498
- Introduce point cloud workspace.context by @ikondrat in #2497
- feat: add explicit sequence_id by @ikondrat in #2486
- Introduce createPointCloudWorkspaceContext by @ikondrat in #2501
- Serve image search when the embedder has no bytes capability by @iunir in #2451
- use context in PointCloudLabelingWorkspace by @ikondrat in #2504
- Let a stored config serve capabilities a registered embedder lacks by @iunir in #2507
- Render initial summary by @ikondrat in #2506
- Enable CodeRabbit reviews on draft PRs by @LeonardoRosaa in #2510
- introduce getTickNumberFromHash by @ikondrat in #2509
- Add: Integrate slic npm package to the frontend code by @IgorSusmelj in #2512
- Index one MCAP recording into a sequence of groups by @horatiualmasan in #2489
- Index MCAP recordings from a path by @horatiualmasan in #2493
- Add an MCAP sequence example and export the Python API by @horatiualmasan in #2496
- Skip the temp-file fallback of pasted-image search for a remote space by @iunir in #2508
- Read the embedding space of a remote server without a stored config by @iunir in #2515
- Add resolver functions to store a remote embedder on an embedding model by @iunir in #2516
- Add register_remote_embedder to point an embedding space at a remote server by @iunir in #2514
- Store MCAP capture timestamps on locators. by @horatiualmasan in #2521
- adjust tick details by @ikondrat in #2527
- Add: Toolbar and size controls for ML assisted labeling by @IgorSusmelj in #2518
- feat: render camera frames by @ikondrat in #2529
- Use capture timestamps as MCAP sequence timestamps. by @horatiualmasan in #2522
- Add fallback matching of MCAP timestamps by @horatiualmasan in #2524
- Tell a failed remote embedding server apart from a missing capability by @iunir in #2530
- Raise typed errors when a query embedder cannot be resolved by @iunir in #2531
- Pair MCAP locators on capture time, then log time. by @horatiualmasan in #2525
- Move the threaded embedding server of the round-trip test into a helper by @iunir in #2537
- Index MCAP recordings with capture pairing and topic channel ids. by @horatiualmasan in #2526
- Return 409 or 502 when a query embedder cannot be resolved by @iunir in #2528
- Test text search through register_remote_embedder against a real server by @iunir in #2534
- adjust _point_cloud_message by @ikondrat in #2543
- introduce normalize_point_cloud_intensity by @ikondrat in #2545
- Test pasted-image search through register_remote_embedder by @iunir in #2539
- introduce point_cloud_value by @ikondrat in #2548
- feat: introduce cacher for mcap reading by @ikondrat in #2535
- Add quickstart-enterprise CLI command by @LeonardoRosaa in #2511
- Show drag shortcuts in the plot selection tool tooltips by @gabrielfruet in #2536
- Test the error paths of search through register_remote_embedder by @iunir in #2549
- adjust get camera frame by @ikondrat in #2538
- Serve a dataset with a remote embedder for an e2e test by @iunir in #2550
- Test search through a remote embedder in the GUI by @iunir in #2551
- introduce read_point_cloud_field by @ikondrat in #2544
- Document remote embedders and add an example by @iunir in #2533
- Run the remote embedder e2e test after merge and on dispatch by @iunir in #2554
- add read_point_cloud_colors by @ikondrat in #2541
- Add: Connect segmentation results from superpixel interactions by @IgorSusmelj in #2555
- feat: utilities to arrow blobs - part 1 by @ikondrat in #2567
- add read_point_cloud_layout by @ikondrat in #2568
- feat: introduce build_point_cloud_schema by @ikondrat in #2569
- Build point cloud columns by @ikondrat in #2570
- feat: add serialize_point_cloud by @ikondrat in #2571
- introduce get_point_cloud by @ikondrat in #2542
- Keep the dev API URL out of the built GUI by @mrpositron in #2577
- Add: Preserve ml assisted labeling mode and add superpixel by @IgorSusmelj in #2578
- Add: Superpixel selection for ai assisted labeling by @IgorSusmelj in #2579
- Add CUBOID_3D schema and object-track parent/source ids. by @horatiualmasan in #2553
- Create cuboid_3d annotations on GROUP samples. by @horatiualmasan in #2561
- update mcap example to pass all channels by @ikondrat in #2595
- Embed and commit images in chunks by @iunir in #2596
- feat: Introduce route to fetch information about lidar channel by @ikondrat in #2594
- feat: wire instance segmentation into evaluation recompute by @lorinczszabolcs in #2457
- Add a route to a connected remote embedder (1/4) by @iunir in #2601
- Embed images without an embedding on a rerun by @iunir in #2597
- Document the serving API and the conformance command by @iunir in #2563
- Add compact categorical metadata filter layout by @lukas-lightly in #2592
- Skip existing embeddings on insert by @iunir in #2604
- Introduce types by @ikondrat in #2609
- Delete, copy, and relink cuboid_3d annotation details. by @horatiualmasan in #2564
- Accept video filters in the metadata distribution endpoints by @mrpositron in #2490
- Embed images by path through a remote image-bytes route (2/4) by @iunir in #2599
- add mergeBounds by @ikondrat in #2611
- Move the image distribution panel out of the collection layout by @mrpositron in #2491
- Parse cuboids and frame tags from Foxglove SceneUpdate. by @horatiualmasan in #2565
- add mergeCloudPointFrames by @ikondrat in #2612
- Skip an image that cannot be read or is larger than one request (3/4) by @iunir in #2602
- Prepare the video count hook and the distribution panel for videos by @mrpositron in #2495
- Remove duplicated point cloud types from pointCloudBuffer by @ikondrat in #2623
- Improve Point Cloud Scene Rotation and Translation by @LeonardoRosaa in #2618
- Show the distribution panel on the videos grid by @mrpositron in #2505
- Disable subpart diversity when no croppable annotation sources exist by @LeonardoRosaa in #2625
- add parseCloudPointFrame by @ikondrat in #2613
- Add rgb color mode to point cloud rendering by @ikondrat in #2622
- Rewrite ROS 1 time and duration fields so SceneUpdate CDR decodes. by @horatiualmasan in #2566
- Skip *_labeled annotation MCAPs when indexing recordings. by @horatiualmasan in #2572
- Join annotation MCAP SceneUpdate timestamps to tick groups. by @horatiualmasan in #2589
- introduce cloud buffer updates by @ikondrat in #2610
- fix: undo of an annotation deletion now restores its original source by @kshivam4781 in #2513
- Apply the image-path adapter in the embedder registry (4/4) by @iunir in #2600
- Introduce fetchCloudPointFrame by @ikondrat in #2614
- Retry network errors in the remote transport by @iunir in #2619
- Introduce useCloudPointFrame by @ikondrat in #2616
- Split a batch again when the server answers 413 by @iunir in #2620
- Move categorical metadata filters to sidebar by @lukas-lightly in #2617
- Render lidar details by @ikondrat in #2627
- Round grid thumbnail sizes to fixed steps and release video connections by @mrpositron in #2574
- Persist matched annotation MCAP cuboids, tracks, and frame tags. by @horatiualmasan in #2590
- Lower the Remote Embedder e2e job timeout by @iunir in #2628
- Fix clipped focus ring in the bulk annotation panel by @mrpositron in #2633
- Add helpers for matching annotation MCAPs to indexed sequences. by @horatiualmasan in #2629
- Match annotation MCAPs in a folder to indexed recordings. by @horatiualmasan in #2630
- Attach cuboids from annotation MCAPs in a folder. by @horatiualmasan in #2632
- Fix: Select segmentation masks when their bounding boxes are hidden by @mrpositron in #2585
- Add video embedding region resolver by @lukas-lightly in #2643
- Require Python tests to state the values their assertions depend on by @gabrielfruet in #2636
- initial impelementation of key time frame by @ikondrat in #2642
- Add annotations to the get_tick_details endpoint by @horatiualmasan in #2655
- feat: initial playback implementation by @ikondrat in #2656
- Add placeholder for camera information by @ikondrat in #2658
- Store the server of a remote default embedder on the dataset by @gabrielfruet in #2634
- Document ingestion through a remote embedder by @gabrielfruet in #2659
- Store the static transforms of a recording when indexing it by @horatiualmasan in #2670
- Add the reference frames of a recording to the recording model by @horatiualmasan in #2671
- Use "ingestion" instead of "import" for data in the glossary by @gabrielfruet in #2666
- Allow re-applying a query to refresh the data by @JonasWurst in #2641
- Resolve video embedding regions across queries by @lukas-lightly in #2657
- Replace "import" with "ingest" or "add" for data across docs and GUI by @gabrielfruet in #2669
- Store the reference frames of a recording when indexing it by @horatiualmasan in #2676
- Move image-bytes sending into a shared PreparedImageRoute base by @gabrielfruet in #2667
- Expose the reference frames in the sequence summary by @horatiualmasan in #2677
- Look up the transform between two frames at a point in time by @horatiualmasan in #2681
- Embed annotation crops on a remote embedder by @gabrielfruet in #2668
- Serve point clouds in a target coordinate frame by @horatiualmasan in #2683
- feat: Create proper layout for recording details by @ikondrat in #2679
- Upload frontend build symbols to PostHog for human-readable logs by @LeonardoRosaa in #2479
- Add a Frame select to the point cloud workspace by @horatiualmasan in #2693
- feat: proper channel filtering by @ikondrat in #2684
- Serve tick cuboids in a target coordinate frame by @horatiualmasan in #2694
- Introduce flap plain projection by @ikondrat in #2692
- Request tick cuboids in the point cloud reference frame by @horatiualmasan in #2695
- fix: introduce key to avoid jumping for flat plain by @ikondrat in #2706
- add point cloud workspace state tools and commands.state rotation by @ikondrat in #2691
- Preserve embedding regions in video filters by @lukas-lightly in #2705
- Use polygon regions for video plot selections by @lukas-lightly in #2698
- Render annotations on the 3D scene by @LeonardoRosaa in #2707
- Clear image plot selections on collection change by @lukas-lightly in #2714
- Make cuboid edges visible at any distance by @LeonardoRosaa in #2715
- Test video embedding region request size by @lukas-lightly in #2710
- Add select box to color points by @ikondrat in #2708
- Add Segment wrapper to point cloud annotation list by @LeonardoRosaa in #2717
- Add backend for mcap sequence prev/next navigation by @horatiualmasan in #2709
- Add frontend for mcap sequence prev/next navigation by @horatiualmasan in #2712
- Add EPIC-KITCHENS-100 egocentric video QA tutorial by @JonasWurst in #2697
- Add: Complete ml assisted labeling for instance segmentations by @IgorSusmelj in #2584
- Add better error messages by @horatiualmasan in #2716
- Add better error messages for labeled mcap files by @horatiualmasan in #2721
- Improve Docs Landing Page by @JonasWurst in #2719
- Filter LiDAR and camera channels in the details view by @ikondrat in #2720
- Persist MCAP frame tags on ticks and return tick tags by @horatiualmasan in #2724
- Cache MCAP-derived camera frames and point clouds for one week by @ikondrat in #2731
- Show and edit tick tags in the point-cloud details view by @horatiualmasan in #2732
- Install CPU torch by default on Linux in the dev environment by @michal-lightly in #2734
- Fix: Improve ai assisted labeling controls by @IgorSusmelj in #2729
- Show track number in cuboid annotation panel by @LeonardoRosaa in #2718
- Bump lightly-studio version to 1.1.3 by @github-actions[bot] in #2735
New Contributors
- @breken-ai made their first contribution in #2310
- @rishcx made their first contribution in #2463
- @kshivam4781 made their first contribution in #2513
Full Changelog: v1.1.2...v1.1.3
Released by @gabrielfruet