Skip to content

Inference connection form v2: curated model list, closed fields, download lifecycle #469

Description

@JArmandoAnaya

The connection form asks a person to type three things it could have offered them: which
model, which device, which precision. Two of those are closed sets the kernel knows, and the
third is a short list this build actually has adapters for. The result today is a form where
gpu, float32x and a model of a family nothing here serves are all accepted, and where the
only feedback is a refusal much later — or, for device and precision, no refusal at all,
because an unrecognised device silently falls back to the CPU.

Directed by Armando (2026-08-09). cf. #418, #421, #424.

A. Model field → grouped dropdown, plus Custom

A curated, grouped select replaces the free-text model field:

  • Interactive segmentation (point prompts) — the SAM 2.1 ladder:
    facebook/sam2.1-hiera-tiny, -small, -base-plus, -large.
  • Text-prompt detectionIDEA-Research/grounding-dino-tiny, -base.
  • Custom model… — reveals the free-text model id and revision fields exactly as today.
    Any hub id remains accepted: curation guides, it never restricts.

Each entry is verified before it is curated — its model_type must land in the resolver's
supported family sets at the locked transformers, and its download size is read once from
the existing size endpoint. The dropdown shows the name, the size and a one-line hint
("tiny — CPU-friendly"). A candidate that fails verification is dropped and named in the PR
body rather than shipped hopefully.

Curated entries pin the revision hash verified at curation time, not main: the form's own
helper text says a moving pointer is not a provenance, and curation should practice it. Custom
mode keeps a free revision field.

The curated list lives in one module the form reads, so extending it later is a one-line
change.

All candidates are Apache-2.0 checkpoints from their original publishers, consistent with the
recorded neutral-sources decision. No entries from other ecosystems.

B. Closed-vocabulary fields, with conditional enablement

  • device: cpu or cuda, with cuda:N kept as the escape for a machine with more than one
    GPU. The vocabulary is the kernel's, so the CLI, the API and the form all refuse the same
    strings.
  • precision: fp16 or fp32, conditioned on device. fp16 on cpu is not honoured by
    either adapter today — both drop half precision unless the device is CUDA — so a connection
    saying cpu + fp16 describes a run that will not happen. The form offers only what works
    and the kernel refuses the rest; the UI does not compute legality on its own.
  • Defaults: cuda + fp16; a curated model on cpu defaults to fp32.
  • Free text stays free where it is genuinely free: name, custom model id, custom revision,
    endpoint URL.

C. Download lifecycle

  1. Completion refresh (a bug today): when the download job reaches a terminal state, the
    connection row must update without a page reload. The job's completion is the mutation, so
    it invalidates the state and allowed_actions of what it touched — the standing
    invalidation rule, wired to the existing background-job observation, not a poll that never
    stops.
  2. Failure surfacing: a failed download leaves the connection Not set up — the kernel
    guarantees there is never a half-ready row — and the row must say so. Status stays
    Not set up, the job's error renders as what-happened + what-to-do prose near the row, and
    Download weights remains available as the retry. The action is the retry; no separate
    retry button. This covers a network loss mid-download and a corrupt cache entry alike:
    the snapshot download resumes what it has and re-checks it against its hashes.
  3. Post-Ready repair: the download job is already idempotent — re-run against a ready
    connection it verifies the cache and no-ops — so download_weights becomes legal at
    {local, ready} too, surfaced as "Verify weights" in the row's overflow menu. The
    prominent row button stays Not set up-only. This is a capability-table change: the kernel
    table, the wire derivation and the contract tests move together. The wire action keeps one
    name; only the label is contextual.

Tests

  • The grouped dropdown renders the curated set from the single source; Custom reveals the free
    fields; device → precision conditioning; a server-side refusal still surfaces as prose.
  • Contract tests extended for download_weights at {local, ready}, and the pinned
    action-set test updated deliberately — it exists to make this a decision rather than an
    accident.
  • Job-terminal → row refresh with no reload; failure → prose plus retry through the same
    action; the never-half-ready invariant asserted at the UI-facing layer.

Bounds

No new model families — the dropdown lists only what the resolver serves today; OWLv2, SigLIP
and the rest are separate issues. No changes to the suggest route, to masks, or to providers
beyond the capability table.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions