Skip to content

docs: fragments: one is the piece you pointed at, not the biggest one - #554

Merged
JArmandoAnaya merged 1 commit into
mainfrom
docs/fragments-wording
Aug 12, 2026
Merged

docs: fragments: one is the piece you pointed at, not the biggest one#554
JArmandoAnaya merged 1 commit into
mainfrom
docs/fragments-wording

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

The suggestion pipeline's fragments setting has two values, and the single case picks the piece
your points are on
rather than the biggest piece on the frame. That has been the behaviour since
it shipped and it is argued in the code, but nowhere a reader would find it: the table in
docs/inference.md said "one piece or all of them", which invites exactly the wrong guess.

Decision (Armando, 2026-08-12): prompt-anchored selection is kept; the wording follows it. This is
the flag raised in #551's "Flags for Armando" section, settled.

What changed

docs/inference.md now states the rule and its three cases — a point inside a piece picks that
piece, several points across several pieces pick the largest of those because two positives
describe one object, and a point inside none of them picks the nearest, since a mask need not cover
the exact pixel clicked. Negative points never select: they say what the shape is not, and a piece
is chosen before its shape is known. It also says what the rule is not, because that is the
guess worth heading off — picking whichever piece owns the topmost-leftmost lit pixel is a fact
about where the speckle fell rather than about what was asked for.

docs/ui.md gets the short form beside the control.

The Fragments docstring is widened too, and that is the half that matters most. Pydantic does
not publish a StrEnum's per-member comments, so the spec carried only the generic sentence and a
client reading openapi.json had no way to learn any of this. The description now names the rule.
openapi.json and the generated client follow; the only functional line in the diff is four lines
of comment in api.ts.

No behaviour changes. masks._pointed_at is untouched.

The gate, and a declared fallback

Static gates, the generated-artifact drift gates, the script gates and every Python module this
change can reach, all green:

ruff check .            All checks passed!
ruff format --check .   375 files already formatted
mypy src/visionset      Success: no issues found in 156 source files
lint-imports            Contracts: 4 kept, 0 broken.

bash scripts/check.sh generated                              exit=0
  ==> openapi drift / generated client drift / mcp tool reference drift / version sync
bash scripts/check.sh docs                                   exit=0
  ==> docs site build / docs projection is deterministic / docs site internal links
pnpm test:scripts                                            exit=0
uv run pytest tests/server tests/kernel tests/inference tests/scripts   exit=0
frontend/annotator:  Tests  1005 passed (1005)

@visionset/ui-core's suite did not run clean here, and it does not run clean on unmodified
main either.
This machine is at a load average of 139 on eight cores, with several other
sessions and a run of clang compilations on it:

load averages: 139.12 93.77 74.50
188.4%CPU  iTerm2
 67.8%CPU  claude
 57.9%CPU  claude
 55.7%CPU  clang -cc1 …   (five of these)

On a clean, freshly built checkout of main at 567f848 with none of this branch's changes:

pnpm -r build && pnpm --filter @visionset/ui-core test
  Test Files  … failed
      Tests  21 failed

On this branch the same suite failed 5 then 3 tests across two runs, with a different set each
time, and all of them pass when their files are run on their own:

pnpm --filter @visionset/ui-core exec vitest run \
  src/screens/inference.test.tsx src/screens/schemaDraft.test.tsx src/screens/screens.test.tsx
  Test Files  3 passed (3)
      Tests  149 passed (149)

A shifting failure set that reproduces on unmodified main is the machine, not the diff — and a
diff of one docstring and two prose paragraphs has no mechanism to reach a schema-draft test.

So this is the sanctioned fallback, declared rather than taken silently. Not run locally: the
full @visionset/ui-core suite and both browser suites (annotator e2e (chromium), browser cycle (chromium)). CI on clean runners is the arbiter, and this will not be merged until every
required check is green there.

Decision (Armando, 2026-08-12): prompt-anchored selection is kept; the wording
follows it.

The shipped rule was already this — masks._pointed_at picks the piece under the
points, falls back to the largest of several that are pointed at, and to the
nearest one where a mask does not cover the exact pixel clicked. What was
missing was anywhere a reader could learn it: the table said "`one` piece or
`all` of them", which invites the guess that `one` means the biggest.

docs/inference.md now states the three cases and why negatives never select, and
says out loud what the rule is not — picking whichever piece owns the
topmost-leftmost lit pixel is a fact about where the speckle fell rather than
about what was asked for. docs/ui.md gets the short form.

The Fragments docstring is widened too, because that one is the client-facing
contract: pydantic does not publish a StrEnum's per-member comments, so the
spec's description was the generic sentence alone and a caller reading it had no
way to learn any of this. openapi.json and the generated client follow.
@JArmandoAnaya
JArmandoAnaya merged commit 94ef832 into main Aug 12, 2026
15 checks passed
@JArmandoAnaya
JArmandoAnaya deleted the docs/fragments-wording branch August 12, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant