Skip to content

feat(inference): curated model list, closed device and precision fields, download lifecycle - #470

Merged
JArmandoAnaya merged 2 commits into
mainfrom
feat/inference-form-v2
Aug 9, 2026
Merged

feat(inference): curated model list, closed device and precision fields, download lifecycle#470
JArmandoAnaya merged 2 commits into
mainfrom
feat/inference-form-v2

Conversation

@JArmandoAnaya

Copy link
Copy Markdown
Contributor

Closes #469.

The connection form asked a person to type three things it could have offered them, and a
finished download did not show up until the page was reloaded. Two commits: the kernel's half,
then the browser's.

Curated models

The model field is a grouped select over the checkpoints this build has an adapter for, with
Custom model… beside them revealing the same free model id and revision fields the form
had before. Curation guides; it does not restrict.

Every entry was verified before it was curated, against the locked transformers 5.14.1 and
huggingface_hub 1.27.0. All six candidates passed and none was dropped:

Model model_type Resolver set Size License
facebook/sam2.1-hiera-tiny sam2_video segmenter 311.9 MB Apache-2.0
facebook/sam2.1-hiera-small sam2_video segmenter 368.8 MB Apache-2.0
facebook/sam2.1-hiera-base-plus sam2_video segmenter 647.1 MB Apache-2.0
facebook/sam2.1-hiera-large sam2_video segmenter 1.8 GB Apache-2.0
IDEA-Research/grounding-dino-tiny grounding-dino detector 1.4 GB Apache-2.0
IDEA-Research/grounding-dino-base grounding-dino detector 1.9 GB Apache-2.0

Each is pinned to the commit hash read at verification time rather than to main — the form's
own helper text says a moving pointer is not a provenance, so the list practises it. The sizes
are the hub's own figures for those revisions, which is why they are safe to hold as constants:
a pinned revision is an immutable set of files.

The whole list lives in frontend/ui-core/src/screens/inferenceCatalog.ts. The default a new
local connection opens on is facebook/sam2.1-hiera-base-plus, the pinned successor of the
single model the form suggested before there was a list.

Closed device and precision fields

device accepts cpu, cuda, and cuda:N for a second GPU. Everything else was accepted
before and then quietly resolved onto the CPU in full precision, so a connection saying gpu
described a run that never happened and went on displaying gpu while it did not happen.

precision is a Precision enum (fp16, fp32), published on the wire, with float16,
half, float32 and full normalized onto it — those spellings have been honoured since the
field existed, and a vocabulary that closed by refusing them would refuse a stored row on the
way out of the store. It is conditioned on the device: both local adapters resolve half
precision as this device is CUDA and the connection asked for fp16, so cpu + fp16 is a
setting with no effect rather than a slow one. precisions_for() owns that rule, and the form
reads the same shape rather than a second copy of the sentence.

The two vocabularies are published two different ways because one has a member that is not a
fixed word: precision is an enum a client can enumerate, device is a string the kernel
refuses outside its pattern. Both refusals are the kernel's, and both render as prose.

Download lifecycle

  1. Completion refresh (the bug). The 202 invalidated the connection list; the job's
    completion — which is what moves setup_state — invalidated nothing, so a finished
    download left Not set up on screen until a reload. A settled job now invalidates what it
    touched, through the existing background-job observation. No poll that never stops: the
    poll settles on its own and the invalidation fires on the transition.
  2. Failure surfacing. A failed run leaves the row at Not set up and renders the job's own
    words plus what to do about it. Download weights is the retry — there is no second
    control.
  3. Post-Ready repair. download_weights is now legal at {local, ready}, surfaced as
    Verify weights in the row's overflow menu; the prominent button stays Not set up-only
    and the wire action keeps one name. The kernel table, the wire derivation and the contract
    tests moved together.

That retires the retrying flag on require_downloadable / fetch_weights end to end: it
existed only to relax the state half of a gate that no longer refuses, so a re-queued orphan
and a person pressing Verify now take the identical path.

A claim that was checked and was wrong

The dispatch and the existing docstrings said a re-run verifies the cached snapshot against its
hashes. It does not. hf_hub_download short-circuits when the revision is a commit hash and the
pointer path exists — it returns the file without reading it — so a re-run proves the snapshot
is complete, not that it is intact. What it does do, verified in the same read: an
interrupted transfer resumes from what it had (.incomplete + Range), and a file that
arrives is checked against the size the hub published before it is moved into place.

The action still earns its place: an incomplete cache is the failure that actually happens on a
machine where a disk filled or a cache was pruned. But weights.py, capabilities.py,
docs/inference.md and the UI copy now say completeness rather than integrity. Anyone wanting
a true integrity check would need force_download or a hash pass, which is a separate issue and
is not in this PR.

Tests

New: the two vocabularies and their aliases, the cross-field rule at the domain, the service and
the wire; precisions_for as the single owner of the conditioning; the contract tests extended
for download_weights at {local, ready}; the curated list's own rules (every revision a
40-hex commit, the default resolves, the pair is the identity); the grouped dropdown rendered
from the catalog; Custom revealing the free fields; device→precision conditioning; the kernel's
refusal rendering as prose; job-terminal → row refresh with no reload; failure prose plus retry
through the same action; Verify weights offered only where the wire declares it.

Pinned test moved deliberately. test_download_weights_is_declared_on_exactly_one_square
is now ..._on_both_local_squares_and_no_others. It exists to make a widening a decision rather
than a drift, and this is the decision.

Fixtures across five test modules paired device="cpu" with precision="fp16" — exactly the
combination the new rule refuses — and were moved to fp32.

Mutation verification

Each mutation was applied to a committed tree, run, and reverted by its exact diff.

Mutation Result
Hardcode facebook/sam2-hiera-base-plus in InferenceScreen.tsx instead of DEFAULT_MODEL.modelId 5 red in inference.test.tsx
Delete the refresh() call from the settled-job effect refreshes the row when the job finishes, with no reload red
Widen CONNECTION_KINDS[DOWNLOAD_WEIGHTS] to EVERY_CONNECTION_TYPE 2 red in test_capabilities.py
Make the device→precision cross-field check unreachable 4 red across test_inference_connections.py and test_inference.py

Local gate

Run in stages under the harness's command ceiling; the machine was free (load average 3.55 on
20 cores), so this is the full gate split, not the saturation fallback.

Stage Exit
pytest tests/architecture 0
pytest tests/cli 0
pytest tests/examples 0
pytest tests/formats 0
pytest tests/inference 0
pytest tests/jobs 0
pytest tests/kernel 0
pytest tests/mcp 0
pytest tests/packaging 0
pytest tests/server tests/test_versioning.py 0
ruff check . 0
ruff format --check . 0
mypy src/visionset 0
lint-imports 0
pnpm -r build 0
pnpm test (node script gates + vitest, 827 passing) 0
pnpm -r lint 0
openapi drift / generate:client:check / export_mcp_tools --check / version:check 0
playwright test (chromium e2e) 0 — 232 passed, 1 flaky (gallery.spec.ts tile overlap, unrelated, passed on retry)
playwright test -c playwright.cycle.config.ts 0

tests/scripts holds only .mjs files, so pytest there exits 5 by design; those gates run
under pnpm test.

Found, not fixed

  • A workspace whose inference_connection row carries a device outside the new pattern — only
    reachable by typing one into the free-text form or the CLI before this change — now fails
    validation on read. The precision aliases cover the spellings the product documented;
    a device like gpu or mps does not have an honest normalization, and rewriting one behind
    somebody's back would contradict this package's own "no rewriting of what somebody typed".
    No migration is included: inference_connection arrived in migration 6, one day before this
    change, and such a row was never running where it claimed to.
  • #466 (the four inference request models do not forbid unknown fields) is untouched.

…ts can be verified

Two rules the kernel could state and did not.

`device` accepts `cpu`, `cuda` and `cuda:N` and nothing else. Everything
outside that used to be accepted and then quietly resolved onto the CPU in
full precision, so a connection saying `gpu` described a run that never
happened and went on displaying `gpu` while it did not happen.

`precision` is a `Precision` enum — `fp16`, `fp32` — with the spellings this
build already honoured (`float16`, `half`, `float32`, `full`) normalized onto
it, so a row written under the free-text field stays readable. It is
conditioned on the device: both local adapters resolve half precision as *this
device is CUDA and the connection asked for fp16*, which makes `cpu` + `fp16`
a setting with no effect rather than a slow one. `precisions_for` owns that
rule so a form offering a choice and the kernel refusing one read the same
function.

`download_weights` becomes legal at `{local, ready}` as well. The work is
idempotent by the download library's own design — a snapshot already in the
cache is checked against its hashes rather than re-fetched — so the same call
against a ready connection answers "are the weights still there and still
intact?", which had no action at all before. That retires the `retrying` flag:
it existed only to relax the state half of a gate that no longer refuses, and
a re-queued orphan now takes the ordinary path.

cf. #469
…and a download that finishes on screen

The connection form asked a person to type three things it could have offered
them. All three are now lists.

**Model.** A grouped select over the models this build has an adapter for, each
entry pinned to a revision verified against the locked transformers and shown
with the hub's own download size and a line on what it is for. **Custom model…**
reveals the free model id and revision fields, so curation guides without
restricting. The list lives in one module, `inferenceCatalog.ts`, and the form
reads it and nothing else — a model id spelled anywhere in `screens/` is now a
test failure.

**Device and precision.** The kernel's vocabularies, offering-side, with the
precision list following the device because half precision applies on CUDA only.
A row already carrying `cuda:1` keeps it rather than being silently reassigned.
The kernel is still the authority: what it refuses arrives as prose.

**The download's whole life.** A settled job invalidates the connection list, so
a finished download turns the row `Ready` where somebody can see it — the bug
was that only the `202` invalidated, and the completion, which is what moves
`setup_state`, invalidated nothing. A failure renders the job's own words plus
what to do about it, the row stays `Not set up`, and **Download weights** is the
retry — there is no second control. At `Ready` the same declared action appears
as **Verify weights** in the row's overflow, where it re-checks that the
snapshot is complete.

The claim that a re-run verifies the cache against its hashes was checked and is
not true: `hf_hub_download` returns a file already present under a pinned
revision without re-reading it. What a re-run establishes is completeness, and
the docstrings, the docs and the UI copy now say that instead.

cf. #469
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Inference connection form v2: curated model list, closed fields, download lifecycle

1 participant