feat(inference): curated model list, closed device and precision fields, download lifecycle - #470
Merged
Merged
Conversation
…ts can be verified
Two rules the kernel could state and did not.
`device` accepts `cpu`, `cuda` and `cuda:N` and nothing else. Everything
outside that used to be accepted and then quietly resolved onto the CPU in
full precision, so a connection saying `gpu` described a run that never
happened and went on displaying `gpu` while it did not happen.
`precision` is a `Precision` enum — `fp16`, `fp32` — with the spellings this
build already honoured (`float16`, `half`, `float32`, `full`) normalized onto
it, so a row written under the free-text field stays readable. It is
conditioned on the device: both local adapters resolve half precision as *this
device is CUDA and the connection asked for fp16*, which makes `cpu` + `fp16`
a setting with no effect rather than a slow one. `precisions_for` owns that
rule so a form offering a choice and the kernel refusing one read the same
function.
`download_weights` becomes legal at `{local, ready}` as well. The work is
idempotent by the download library's own design — a snapshot already in the
cache is checked against its hashes rather than re-fetched — so the same call
against a ready connection answers "are the weights still there and still
intact?", which had no action at all before. That retires the `retrying` flag:
it existed only to relax the state half of a gate that no longer refuses, and
a re-queued orphan now takes the ordinary path.
cf. #469
…and a download that finishes on screen The connection form asked a person to type three things it could have offered them. All three are now lists. **Model.** A grouped select over the models this build has an adapter for, each entry pinned to a revision verified against the locked transformers and shown with the hub's own download size and a line on what it is for. **Custom model…** reveals the free model id and revision fields, so curation guides without restricting. The list lives in one module, `inferenceCatalog.ts`, and the form reads it and nothing else — a model id spelled anywhere in `screens/` is now a test failure. **Device and precision.** The kernel's vocabularies, offering-side, with the precision list following the device because half precision applies on CUDA only. A row already carrying `cuda:1` keeps it rather than being silently reassigned. The kernel is still the authority: what it refuses arrives as prose. **The download's whole life.** A settled job invalidates the connection list, so a finished download turns the row `Ready` where somebody can see it — the bug was that only the `202` invalidated, and the completion, which is what moves `setup_state`, invalidated nothing. A failure renders the job's own words plus what to do about it, the row stays `Not set up`, and **Download weights** is the retry — there is no second control. At `Ready` the same declared action appears as **Verify weights** in the row's overflow, where it re-checks that the snapshot is complete. The claim that a re-run verifies the cache against its hashes was checked and is not true: `hf_hub_download` returns a file already present under a pinned revision without re-reading it. What a re-run establishes is completeness, and the docstrings, the docs and the UI copy now say that instead. cf. #469
This was referenced Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #469.
The connection form asked a person to type three things it could have offered them, and a
finished download did not show up until the page was reloaded. Two commits: the kernel's half,
then the browser's.
Curated models
The model field is a grouped select over the checkpoints this build has an adapter for, with
Custom model… beside them revealing the same free model id and revision fields the form
had before. Curation guides; it does not restrict.
Every entry was verified before it was curated, against the locked
transformers5.14.1 andhuggingface_hub1.27.0. All six candidates passed and none was dropped:model_typefacebook/sam2.1-hiera-tinysam2_videofacebook/sam2.1-hiera-smallsam2_videofacebook/sam2.1-hiera-base-plussam2_videofacebook/sam2.1-hiera-largesam2_videoIDEA-Research/grounding-dino-tinygrounding-dinoIDEA-Research/grounding-dino-basegrounding-dinoEach is pinned to the commit hash read at verification time rather than to
main— the form'sown helper text says a moving pointer is not a provenance, so the list practises it. The sizes
are the hub's own figures for those revisions, which is why they are safe to hold as constants:
a pinned revision is an immutable set of files.
The whole list lives in
frontend/ui-core/src/screens/inferenceCatalog.ts. The default a newlocal connection opens on is
facebook/sam2.1-hiera-base-plus, the pinned successor of thesingle model the form suggested before there was a list.
Closed device and precision fields
deviceacceptscpu,cuda, andcuda:Nfor a second GPU. Everything else was acceptedbefore and then quietly resolved onto the CPU in full precision, so a connection saying
gpudescribed a run that never happened and went on displaying
gpuwhile it did not happen.precisionis aPrecisionenum (fp16,fp32), published on the wire, withfloat16,half,float32andfullnormalized onto it — those spellings have been honoured since thefield existed, and a vocabulary that closed by refusing them would refuse a stored row on the
way out of the store. It is conditioned on the device: both local adapters resolve half
precision as this device is CUDA and the connection asked for fp16, so
cpu+fp16is asetting with no effect rather than a slow one.
precisions_for()owns that rule, and the formreads the same shape rather than a second copy of the sentence.
The two vocabularies are published two different ways because one has a member that is not a
fixed word:
precisionis an enum a client can enumerate,deviceis a string the kernelrefuses outside its pattern. Both refusals are the kernel's, and both render as prose.
Download lifecycle
202invalidated the connection list; the job'scompletion — which is what moves
setup_state— invalidated nothing, so a finisheddownload left
Not set upon screen until a reload. A settled job now invalidates what ittouched, through the existing background-job observation. No poll that never stops: the
poll settles on its own and the invalidation fires on the transition.
Not set upand renders the job's ownwords plus what to do about it. Download weights is the retry — there is no second
control.
download_weightsis now legal at{local, ready}, surfaced asVerify weights in the row's overflow menu; the prominent button stays
Not set up-onlyand the wire action keeps one name. The kernel table, the wire derivation and the contract
tests moved together.
That retires the
retryingflag onrequire_downloadable/fetch_weightsend to end: itexisted only to relax the state half of a gate that no longer refuses, so a re-queued orphan
and a person pressing Verify now take the identical path.
A claim that was checked and was wrong
The dispatch and the existing docstrings said a re-run verifies the cached snapshot against its
hashes. It does not.
hf_hub_downloadshort-circuits when the revision is a commit hash and thepointer path exists — it returns the file without reading it — so a re-run proves the snapshot
is complete, not that it is intact. What it does do, verified in the same read: an
interrupted transfer resumes from what it had (
.incomplete+Range), and a file thatarrives is checked against the size the hub published before it is moved into place.
The action still earns its place: an incomplete cache is the failure that actually happens on a
machine where a disk filled or a cache was pruned. But
weights.py,capabilities.py,docs/inference.mdand the UI copy now say completeness rather than integrity. Anyone wantinga true integrity check would need
force_downloador a hash pass, which is a separate issue andis not in this PR.
Tests
New: the two vocabularies and their aliases, the cross-field rule at the domain, the service and
the wire;
precisions_foras the single owner of the conditioning; the contract tests extendedfor
download_weightsat{local, ready}; the curated list's own rules (every revision a40-hex commit, the default resolves, the pair is the identity); the grouped dropdown rendered
from the catalog; Custom revealing the free fields; device→precision conditioning; the kernel's
refusal rendering as prose; job-terminal → row refresh with no reload; failure prose plus retry
through the same action; Verify weights offered only where the wire declares it.
Pinned test moved deliberately.
test_download_weights_is_declared_on_exactly_one_squareis now
..._on_both_local_squares_and_no_others. It exists to make a widening a decision ratherthan a drift, and this is the decision.
Fixtures across five test modules paired
device="cpu"withprecision="fp16"— exactly thecombination the new rule refuses — and were moved to
fp32.Mutation verification
Each mutation was applied to a committed tree, run, and reverted by its exact diff.
facebook/sam2-hiera-base-plusinInferenceScreen.tsxinstead ofDEFAULT_MODEL.modelIdinference.test.tsxrefresh()call from the settled-job effectrefreshes the row when the job finishes, with no reloadredCONNECTION_KINDS[DOWNLOAD_WEIGHTS]toEVERY_CONNECTION_TYPEtest_capabilities.pytest_inference_connections.pyandtest_inference.pyLocal gate
Run in stages under the harness's command ceiling; the machine was free (load average 3.55 on
20 cores), so this is the full gate split, not the saturation fallback.
pytest tests/architecturepytest tests/clipytest tests/examplespytest tests/formatspytest tests/inferencepytest tests/jobspytest tests/kernelpytest tests/mcppytest tests/packagingpytest tests/server tests/test_versioning.pyruff check .ruff format --check .mypy src/visionsetlint-importspnpm -r buildpnpm test(node script gates + vitest, 827 passing)pnpm -r lintopenapi drift/generate:client:check/export_mcp_tools --check/version:checkplaywright test(chromium e2e)gallery.spec.tstile overlap, unrelated, passed on retry)playwright test -c playwright.cycle.config.tstests/scriptsholds only.mjsfiles, sopytestthere exits 5 by design; those gates rununder
pnpm test.Found, not fixed
inference_connectionrow carries a device outside the new pattern — onlyreachable by typing one into the free-text form or the CLI before this change — now fails
validation on read. The precision aliases cover the spellings the product documented;
a device like
gpuormpsdoes not have an honest normalization, and rewriting one behindsomebody's back would contradict this package's own "no rewriting of what somebody typed".
No migration is included:
inference_connectionarrived in migration 6, one day before thischange, and such a row was never running where it claimed to.
#466(the four inference request models do not forbid unknown fields) is untouched.