Skip to content

[AI] Replace brush with click-based interaction for AI object mask#20595

Merged
TurboGit merged 2 commits intodarktable-org:masterfrom
andriiryzhkov:no_brush
Mar 20, 2026
Merged

[AI] Replace brush with click-based interaction for AI object mask#20595
TurboGit merged 2 commits intodarktable-org:masterfrom
andriiryzhkov:no_brush

Conversation

@andriiryzhkov
Copy link
Copy Markdown
Contributor

Simplify the object mask UX and improve selection quality.

Why remove brushing

The brush-first step was my initial idea to improve UX and speed up object selection - let the user quickly paint over the object instead of clicking multiple points. After extensive testing, the effect turned out to be the opposite: in many situations the initial selection was completely wrong (inverted mask, selecting background instead of object, or producing noisy fragmented masks).

The root cause: the brush approach generated 10-30+ dense point prompts from a single stroke, but SAM2 was designed for small amount of precise clicks. This caused several problems in practice:

  • Inconsistent results: the same brush stroke on the same object could produce a good mask or a completely inverted one depending on exact mouse path and speed
  • Inverted selections: SAM often selected the background instead of the brushed object, because the dense points overwhelmed the model's prompt encoder
  • Speed-dependent bias: faster mouse movement produced sparser points, slower movement denser - the model response changed based on how fast the user moved
  • No spatial diversity: all points clustered in the brushed area, giving SAM no information about the rest of the object

A single click on the center of an object consistently produces a better mask than a careful brush stroke. This matches Meta's SAM demo and other segment-anything tools where click-to-select is the standard interaction.

Changes

  • Click to select: first click on an object creates an immediate mask
  • Click +/- to refine: additional clicks add/remove regions (shift+click for background)
  • First click is always foreground: shift is ignored on the first click
  • Removed brush code: brush painting, brush cursor, brush points, brush size config
  • Renamed brush_used to has_selection: cleaner semantics
  • Added feather slider: adjustable path border/softness

@TurboGit TurboGit added this to the 5.6 milestone Mar 20, 2026
@TurboGit TurboGit added the feature: redesign current features to rewrite label Mar 20, 2026
Copy link
Copy Markdown
Member

@TurboGit TurboGit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me, thanks!

@TurboGit TurboGit merged commit 31c5f34 into darktable-org:master Mar 20, 2026
5 checks passed
@andriiryzhkov andriiryzhkov deleted the no_brush branch March 20, 2026 07:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature: redesign current features to rewrite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants