Feature/bitmask

.github/tasks.md

@@ -1,2 +1,12 @@
 ## Tasks
 
+### Bitmask segmentation annotation mode
+Per-annotation binary masks, COCO-style RLE serialization, brush + erase interaction.
+
+- [x] Phase 1: Data model + RLE utils + tests (`src/mask_utils.ts`, `ULabelSpatialType`, `SPATIAL_TYPE_SET`)
+- [x] Phase 2: Bitmask rendering layer (`draw_bitmask`, dispatch, redraw/clear)
+- [x] Phase 3: Brush/erase paints pixels (begin/continue/finish for bitmask)
+- [x] Phase 4: Undo/redo patch diffs for brush strokes (single-stroke `bitmask_stroke` action)
+- [x] Phase 5: Toolbox + mode registration (mode button, brush enable/disable, keybinds)
+- [ ] Phase 6: Export/import round-trip + tests + demo page
+

CHANGELOG.md

@@ -4,6 +4,20 @@ All notable changes to this project will be documented here.
 
 ## [unreleased]
 
+## [0.25.0] - Aug 5th, 2026
+- Add a `bitmask` annotation mode for raster (per-pixel) segmentation, selectable via `allowed_modes: ["bitmask", ...]`.
+  - Painted with the brush (toggle with `toggle_brush_mode_keybind`, default `g`); erase with `toggle_erase_mode_keybind` (default `e`); resize the brush with `increase_brush_size_keybind` / `decrease_brush_size_keybind` (defaults `]` / `[`) or `alt+scroll`. The brush/erase toggles now apply to both `polygon` and `bitmask` modes.
+  - A paint stroke only joins an existing mask of the currently-selected class; painting a different class (or over empty space) starts a new mask of the selected class. Erase is class-agnostic. This differs from the `polygon` brush, which joins any polygon under the brush.
+  - Hover a mask (with the brush off) to change its class via the ID dialog, or move/delete it like any other spatial annotation.
+  - Each bitmask annotation stores a single binary mask. On export, `spatial_payload` is a COCO-style run-length-encoded object: `{ "counts": <number[]>, "size": [<height>, <width>] }` (column-major, starting with a background run). Fully-erased masks are deprecated (ULabel's delete semantics).
+  - Configurable render opacity for bitmask classes via `mask_annotation_opacity`.
+- Add brush **overlap modes** for bitmask painting, controlled globally and persisted to localStorage: `none` (default), `exclude`, and `overwrite`.
+  - `exclude`: newly-painted pixels never cover pixels owned by other undeprecated bitmask annotations (existing masks win).
+  - `overwrite`: newly-painted pixels are removed from any other bitmask annotation that owned them (the new mask wins); a mask fully carved away is deprecated.
+  - Resolution is deferred to the end of a stroke (you see the active mask cover others, then it snaps to the resolved result on release) and only affects the pixels a stroke adds; erase is unaffected.
+  - Selectable via the Brush toolbox item (shown in bitmask mode) and keybinds `set_brush_overlap_none_keybind` / `set_brush_overlap_exclude_keybind` / `set_brush_overlap_overwrite_keybind` (defaults `shift+n` / `shift+e` / `shift+o`), which also appear in the `Keybinds` toolbox item. Initial value configurable via `default_brush_overlap_mode`.
+  - Brush toolbox buttons have hover tooltips.
+
 ## [0.24.0] - July 22nd, 2026
 - Add `ConfidenceSlider` toolbox item (`AllowedToolboxItem.ConfidenceSlider`) that deprecates (hides) or shows spatial annotations based on their confidence values. Unlike the now-deprecated `KeypointSlider`, it works with all spatial annotation types that have a confidence payload (`bbox`, `bbox3`, `polygon`, `polyline`, `contour`, `tbar`, and `point`), across every subtask.
   - Supports a single global "all" slider and/or per-class sliders, controlled by `class_filter_mode` (`"toggle"`, `"all-only"`, or `"class-only"`).

api_spec.md

@@ -66,6 +66,11 @@ class ULabel({
     toggle_erase_mode_keybind: string,
     increase_brush_size_keybind: string,
     decrease_brush_size_keybind: string,
+    mask_annotation_opacity: number,
+    default_brush_overlap_mode: BrushOverlapMode,
+    set_brush_overlap_none_keybind: string,
+    set_brush_overlap_exclude_keybind: string,
+    set_brush_overlap_overwrite_keybind: string,
     fly_to_next_annotation_keybind: string,
     fly_to_previous_annotation_keybind: string,
     annotation_size_small_keybind: string,
@@ -178,7 +183,9 @@ As you can see, each subtask will have a corresponding list of annotation object
     "spatial_type": "<string>", 
 
     // (nullable) e.g. [[x1, y1], [x2, y2], ...]
-    "spatial_payload": "<array>", 
+    // For "bitmask" annotations this is instead a run-length-encoded object:
+    // { "counts": <number[]>, "size": [<height>, <width>] }. See Bitmask annotations.
+    "spatial_payload": "<array | object>", 
 
     // The class associated with the annotation
     "classification_payloads": [ 
@@ -281,9 +288,49 @@ The full list of `"allowed_modes"` that are currently supported is:
 - `"whole-image"`: A label to be applied to an entire frame
 - `"global"`: A label to be applied to the entire series of frames
 - `"point"`: A keypoint within a single frame
+- `"bitmask"`: A raster (per-pixel) segmentation mask, painted with the brush. See [Bitmask annotations](#bitmask-annotations).
 - `"delete_polygon"`: Allows drawing a polygon around an area, and all annotations within that area will be deleted
 - `"delete_bbox"`: Allows drawing a bounding box around an area, and all annotations within that area will be deleted
 
+#### Bitmask annotations
+
+The `"bitmask"` mode enables raster (per-pixel) segmentation. Each bitmask annotation stores a single binary mask the size of the image.
+
+**Interaction**
+
+- Painting uses the brush, shared with the `polygon` brush. Toggle the brush with `toggle_brush_mode_keybind` (default `g`) or the Brush toolbox item, erase with `toggle_erase_mode_keybind` (default `e`), and resize the brush with `increase_brush_size_keybind` / `decrease_brush_size_keybind` (defaults `]` / `[`) or `alt+scroll`.
+- Starting a paint stroke over an existing bitmask of the **currently-selected class** adds to that mask; otherwise (a different class is selected, or you start over empty space) a new bitmask annotation of the selected class is created. Erasing is class-agnostic — it removes from whichever mask is under the brush. (This class-aware joining differs from the `polygon` brush, which joins any polygon under the brush.)
+- With the brush off, hovering a mask surfaces the usual edit dialogs: change its class via the ID dialog, or move/delete it like any other spatial annotation. Erasing a mask entirely deprecates the annotation (ULabel's delete semantics).
+- Requires the `Brush` toolbox item (`AllowedToolboxItem.Brush`) to be present.
+
+**Overlap modes**
+
+When painting, the brush can enforce mutual exclusivity with *other* undeprecated bitmask annotations. The mode is a single **global** value, persisted to localStorage, and is chosen via the Brush toolbox item (shown in bitmask mode) or the overlap keybinds. Its initial value comes from [`default_brush_overlap_mode`](#default_brush_overlap_mode).
+
+- `"none"` (default): painting only adds to the active mask; other masks are untouched (pixels may be owned by multiple annotations).
+- `"exclude"`: newly-painted pixels never cover pixels owned by other bitmask annotations (existing masks win).
+- `"overwrite"`: newly-painted pixels are removed from any other bitmask annotation that owned them (the new mask wins); a mask fully carved away is deprecated.
+
+Resolution is **deferred to the end of a stroke** and only affects the pixels the stroke adds (pre-existing overlaps are left alone). Erase strokes are unaffected. These modes govern *new strokes only* — they do not retroactively de-overlap already-imported masks.
+
+**Serialization**
+
+A bitmask's `spatial_payload` is a COCO-style, uncompressed run-length encoding:
+
+```javascript
+{
+    // Alternating run lengths in column-major (Fortran) order, always starting
+    // with a background (0) run. A leading foreground pixel is a leading 0.
+    "counts": [<number>, ...],
+    // [height, width] of the mask (matches COCO's size convention)
+    "size": [<height>, <width>]
+}
+```
+
+Note this is the *uncompressed* form (`counts` as an integer array), not the LEB128-packed string used by `pycocotools`. Masks import from and export to this same object shape.
+
+The render opacity of bitmask annotations is configurable via [`mask_annotation_opacity`](#mask_annotation_opacity).
+
 The `resume_from` attributes are used to import existing annotations into the annotation session for each subtask, respectively. Existing annotations must be provided as a list of annotations of the form specified above.
 
 ### `task_meta` and `annotation_meta`
@@ -506,16 +553,31 @@ Keybind to toggle between annotation and selection modes. Default is `u`.
 Keybind to create a bounding box annotation around the `initial_crop`. Default is `f`. Requires the active subtask to have a `bbox` mode.
 
 ### `toggle_brush_mode_keybind`
-Keybind to toggle brush mode for polygon annotations. Default is `g`. Requires the active subtask to have a `polygon` mode.
+Keybind to toggle brush mode for `polygon` and `bitmask` annotations. Default is `g`. Requires the active subtask to have a `polygon` or `bitmask` mode.
 
 ### `toggle_erase_mode_keybind`
-Keybind to toggle erase mode for polygon annotations. Default is `e`. Requires the active subtask to have a `polygon` mode.
+Keybind to toggle erase mode for `polygon` and `bitmask` annotations. Default is `e`. Requires the active subtask to have a `polygon` or `bitmask` mode.
 
 ### `increase_brush_size_keybind`
-Keybind to increase the brush size. Default is `]`. Requires the active subtask to have a `polygon` mode.
+Keybind to increase the brush size. Default is `]`. Requires the active subtask to have a `polygon` or `bitmask` mode.
 
 ### `decrease_brush_size_keybind`
-Keybind to decrease the brush size. Default is `[`. Requires the active subtask to have a `polygon` mode.
+Keybind to decrease the brush size. Default is `[`. Requires the active subtask to have a `polygon` or `bitmask` mode.
+
+### `mask_annotation_opacity`
+The fill opacity (`0`-`1`) used when rendering `bitmask` (raster segmentation) annotations. Default is `0.4`.
+
+### `default_brush_overlap_mode`
+The initial [brush overlap mode](#overlap-modes) for bitmask painting: `"none"` (default), `"exclude"`, or `"overwrite"`. The live value is global and persisted to localStorage, so a user's last choice takes precedence over this default on subsequent sessions.
+
+### `set_brush_overlap_none_keybind`
+Keybind to set the brush overlap mode to `none`. Default is `shift+n`.
+
+### `set_brush_overlap_exclude_keybind`
+Keybind to set the brush overlap mode to `exclude`. Default is `shift+e`.
+
+### `set_brush_overlap_overwrite_keybind`
+Keybind to set the brush overlap mode to `overwrite`. Default is `shift+o`.
 
 ### `fly_to_next_annotation_keybind`
 Keybind to set the zoom to focus on the next annotation. Default is `Tab`, which also will disable any default browser behavior for `Tab`.

demo.js

@@ -13,4 +13,5 @@ console.log(`http://localhost:${port}/frames.html`);
 console.log(`http://localhost:${port}/box-roi.html`);
 console.log(`http://localhost:${port}/resume-from.html`);
 console.log(`http://localhost:${port}/row-filtering-example.html`);
+console.log(`http://localhost:${port}/bitmask-example.html`);
 console.log(`http://localhost:${port}/live_demo.html`);

demo/bitmask-example.html

@@ -0,0 +1,73 @@
+<!DOCTYPE html>
+<html>
+    <head>
+        <title>ULabel - Bitmask Segmentation</title>
+
+        <!-- ULabel Library -->
+        <script src="/ulabel.js"></script>
+
+        <!-- JQuery Library -->
+        <script src="https://code.jquery.com/jquery-3.5.1.min.js" integrity="sha256-9/aliU8dGd2tb6OSsuzixeV4y/faTqgFtohetphbbj0=" crossorigin="anonymous"></script>
+
+        <!-- ULabel Usage -->
+        <script>
+            /* global $ */
+            /* global ULabel */
+
+            $(window).on("load", function() {
+
+                function on_submit(annotations) {
+                    var element = document.createElement('a');
+                    element.setAttribute(
+                        "href", ('data:text/plain;charset=utf-8,' +
+                        encodeURIComponent(JSON.stringify(annotations, null, 2)))
+                    );
+                    element.setAttribute("download", "annotations.json");
+                    element.style.display = 'none';
+                    document.body.appendChild(element);
+                    element.click();
+                    document.body.removeChild(element);
+                }
+
+                let subtasks = {
+                    "segmentation": {
+                        "display_name": "Segmentation",
+                        "classes": [
+                            {
+                                "name": "Vehicle",
+                                "color": "orange",
+                                "id": 10
+                            },
+                            {
+                                "name": "Obstacle",
+                                "color": "green",
+                                "id": 11
+                            }
+                        ],
+                        "allowed_modes": ["bitmask", "polygon", "delete_bbox"],
+                        "resume_from": null,
+                        "task_meta": null,
+                        "annotation_meta": null
+                    }
+                };
+
+                // Initial ULabel configuration
+                let ulabel = new ULabel({
+                    "container_id": "container",
+                    "image_data": "https://ulabel.s3.us-east-2.amazonaws.com/cs-demo-0.png",
+                    "username": "DemoUser",
+                    "submit_buttons": on_submit,
+                    "subtasks": subtasks
+                });
+                // Wait for ULabel instance to finish initialization
+                ulabel.init(function() {
+                    // ULabel is now ready for use
+                });
+
+            });
+        </script>
+    </head>
+    <body>
+        <div id="container" style="width: 100%; height: 100vh; position: absolute; top: 0; left: 0;"></div>
+    </body>
+</html>