Feature/bitmask

[TrevorBurgoyne]

Bitmasks

Description

• Add a bitmask annotation mode for raster (per-pixel) segmentation, selectable via allowed_modes: ["bitmask", ...].
  • Painted with the brush (toggle with toggle_brush_mode_keybind, default g); erase with toggle_erase_mode_keybind (default e); resize the brush with increase_brush_size_keybind / decrease_brush_size_keybind (defaults ] / [) or alt+scroll. The brush/erase toggles now apply to both polygon and bitmask modes.
  • A paint stroke only joins an existing mask of the currently-selected class; painting a different class (or over empty space) starts a new mask of the selected class. Erase is class-agnostic. This differs from the polygon brush, which joins any polygon under the brush.
  • Hover a mask (with the brush off) to change its class via the ID dialog, or move/delete it like any other spatial annotation.
  • Each bitmask annotation stores a single binary mask. On export, spatial_payload is a COCO-style run-length-encoded object: { "counts": <number[]>, "size": [<height>, <width>] } (column-major, starting with a background run). Fully-erased masks are deprecated (ULabel's delete semantics).
  • Configurable render opacity for bitmask classes via mask_annotation_opacity.
• Add brush overlap modes for bitmask painting, controlled globally and persisted to localStorage: none (default), exclude, and overwrite.
  • exclude: newly-painted pixels never cover pixels owned by other undeprecated bitmask annotations (existing masks win).
  • overwrite: newly-painted pixels are removed from any other bitmask annotation that owned them (the new mask wins); a mask fully carved away is deprecated.
  • Selectable via the Brush toolbox item (shown in bitmask mode) and keybinds set_brush_overlap_none_keybind / set_brush_overlap_exclude_keybind / set_brush_overlap_overwrite_keybind. Initial value configurable via default_brush_overlap_mode.
  • Brush toolbox buttons have hover tooltips.

PR Checklist

• Merged latest main
• Version number in package.json has been bumped since last release
• Version numbers match between package package.json and src/version.js
• Updated documentation if necessary (currently just in api_spec.md)
• Added changes to changelog.md

Breaking API Changes

none i hope

[Copilot]

Pull request overview

Adds first-class support for raster “bitmask” segmentation annotations, including RLE (COCO-style) serialization, brush-based painting/erasing, overlap resolution modes, and UI/tooling updates to expose the new mode and controls.

Changes:

• Introduces bitmask as a spatial annotation type with runtime mask caching and COCO-style RLE import/export support.
• Extends the brush UI/keybind system to support bitmask painting and global overlap modes (none/exclude/overwrite) persisted to localStorage.
• Adds tests and a demo page to validate mask utilities and resume/export round-trips.

Reviewed changes

Copilot reviewed 19 out of 22 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/mask_utils.test.js	New unit tests for ULabelMask core operations + RLE round-trips
tests/annotation.test.js	Adds resume/export round-trip test for bitmask annotations
src/version.js	Bumps runtime version constant
src/toolbox.ts	Shows brush toolbox for bitmask and adds overlap controls + tooltips
src/toolbox_items/keybinds.ts	Adds keybind entries for overlap mode selection
src/mask_utils.ts	Adds ULabelMask implementation + RLE encode/decode utilities
src/listeners.ts	Adds keybind handling for overlap mode selection
src/index.js	Implements bitmask rendering, brush pipeline, overlap resolution, and undo/redo action
src/html_builder.ts	Adds the Bitmask mode button to the mode toolbar
src/configuration.ts	Adds config for mask opacity + default/live overlap mode + overlap keybinds
src/blobs.js	Adds BITMASK_SVG icon
src/blobs.d.ts	Exposes BITMASK_SVG typing
src/annotation.ts	Registers bitmask as a valid spatial type and skips point clamping for it
src/actions.ts	Wires bitmask_stroke into action/undo/redo listeners
package.json	Bumps package version
package-lock.json	Updates lockfile version fields to match package version
index.d.ts	Extends public types/APIs for bitmask + overlap modes
demo/bitmask-example.html	Adds a demo page showcasing bitmask mode
demo.js	Prints the new demo URL
CHANGELOG.md	Adds 0.25.0 release notes for bitmask + overlap modes
api_spec.md	Documents bitmask mode, serialization, and new config/keybinds
.github/tasks.md	Tracks implementation phases for the bitmask feature

[Copilot]

Pull request overview

Copilot reviewed 19 out of 22 changed files in this pull request and generated 1 comment.

Suppressed comments (3)

src/mask_utils.ts:105

• has_foreground_in_circle uses the same rounded loop bounds as paint_circle, so with fractional centers it can fail to detect foreground pixels that are inside the circle (false negatives when deciding which mask is under the brush). Use floor/ceil bounds here too.

        const min_x = clamp_int(cx - r, 0, this.width - 1);
        const max_x = clamp_int(cx + r, 0, this.width - 1);
        const min_y = clamp_int(cy - r, 0, this.height - 1);
        const max_y = clamp_int(cy + r, 0, this.height - 1);

src/index.js:1901

• offscreen.getContext("2d") can return null; calling createImageData on a null context will throw. Guard against a missing 2D context before using it (this also avoids crashes in non-browser/limited-canvas environments).

        const offscreen_ctx = offscreen.getContext("2d");
        const image_data = offscreen_ctx.createImageData(box_width, box_height);
        const data = image_data.data;
        const mask_data = mask.data;

src/mask_utils.ts:72

• Using clamp_int (Math.round) to compute the paint loop bounds can shrink the bounding box when the brush center is fractional (mouse coordinates are floats). That can miss pixels that are actually within the circle, causing visible gaps or inconsistent painting. Use floor/ceil bounds so the iteration fully covers the circle's extent.

This issue also appears on line 102 of the same file.

        const min_x = clamp_int(cx - r, 0, this.width - 1);
        const max_x = clamp_int(cx + r, 0, this.width - 1);
        const min_y = clamp_int(cy - r, 0, this.height - 1);
        const max_y = clamp_int(cy + r, 0, this.height - 1);

[Copilot]

Pull request overview

Copilot reviewed 19 out of 22 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/toolbox.ts:453

• When switching from a brush-capable mode to a non-brush mode, calling toggle_erase_mode/toggle_brush_mode here can force the annotation mode back to polygon/bitmask (toggle_brush_mode attempts to switch modes if not already in one). This can override the user’s requested mode change and leave the UI/state inconsistent. Prefer disabling brush/erase state without changing annotation_mode.

                        ulabel.toggle_erase_mode(e);
                    }
                    // Turn off brush mode if it's on
                    if (current_subtask["state"]["is_in_brush_mode"]) {
                        ulabel.toggle_brush_mode(e);

src/index.js:1768

• get_bitmask duplicates the decode/empty logic from set_bitmask_from_rle and then calls set_bitmask. Using the shared helper reduces duplication and ensures any future validation/size checks apply consistently to both code paths.

            let mask;
            const payload = annotation_object["spatial_payload"];
            if (payload != null && payload["counts"] !== undefined) {
                mask = ULabelMask.from_rle(payload, false);
            } else {

src/index.js:1791

• set_bitmask_from_rle decodes unvalidated RLE payloads and does not ensure payload.size matches the current image dimensions. A mismatched (or malformed) payload can lead to incorrect indexing during rendering and hit-testing. Validate the payload and guard against size mismatches before decoding.

    set_bitmask_from_rle(annotation_object, rle) {
        let mask;
        if (rle != null && rle["counts"] !== undefined) {
            mask = ULabelMask.from_rle(rle, false);
        } else {