The adapter that answers a connection is chosen from the model_type the model's own
config declares (visionset/inference/providers.py). Two defects there, one visible and
one latent.
1. sam2_video is classified as a text-prompt model
SEGMENTER_FAMILIES holds exactly {"sam2"}. The SAM 2 checkpoints published for image
and video work declare model_type: sam2_video — including
facebook/sam2-hiera-base-plus, which is the model id the product's own connection form
suggests. That name is not in the set, so resolution falls through to the detector
adapter, and a point suggestion is refused with a sentence that is confidently false:
$ family_of(connection) -> 'sam2_video' in SEGMENTER_FAMILIES=False
UnsupportedPrompt: facebook/sam2-hiera-base-plus@main answers text prompts;
it was asked with 'points' points, which it has no way to interpret
Reproduced 2026-08-08 against a ready local connection whose weights are in the
workspace cache. The model is a point-prompt model; the message describes some other
model.
The checkpoint is servable as it stands — Sam2Model.from_pretrained loads a
sam2_video checkpoint and transformers names that case as expected:
You are using a model of type `sam2_video` to instantiate a model of type `sam2`.
This may be expected if you are loading a checkpoint that shares a subset of the
architecture (e.g., loading a `sam2_video` checkpoint into `Sam2Model`)
So this is a classification defect only. No adapter needs to change.
2. An unrecognised model_type falls through instead of refusing
family_of returns the declared type, or "" when the config cannot be read; anything
not in SEGMENTER_FAMILIES is handed to the detector adapter. So a model this build has
no adapter for — and a config nothing could parse — is served by whichever adapter
happens to be the fallback, and fails later with a message about that adapter's prompt
kind rather than about the model.
Defect 1 is one instance of this. A wrong classification produces error messages that lie
about the user's model, which is worse than an honest refusal.
What it should do
- Every SAM 2
model_type the locked transformers can serve for image-level point
prompting resolves to the point-prompt adapter.
- An unrecognised
model_type is refused with a message that says the model type is not
supported and lists the ones that are — never resolved to a family by fallback.
Scope
The resolver and its error message. The suggested default model id is correct and does
not change; nothing about the suggest route or the editor changes.
cf. #424, #418
The adapter that answers a connection is chosen from the
model_typethe model's ownconfig declares (
visionset/inference/providers.py). Two defects there, one visible andone latent.
1.
sam2_videois classified as a text-prompt modelSEGMENTER_FAMILIESholds exactly{"sam2"}. The SAM 2 checkpoints published for imageand video work declare
model_type: sam2_video— includingfacebook/sam2-hiera-base-plus, which is the model id the product's own connection formsuggests. That name is not in the set, so resolution falls through to the detector
adapter, and a point suggestion is refused with a sentence that is confidently false:
Reproduced 2026-08-08 against a
readylocal connection whose weights are in theworkspace cache. The model is a point-prompt model; the message describes some other
model.
The checkpoint is servable as it stands —
Sam2Model.from_pretrainedloads asam2_videocheckpoint andtransformersnames that case as expected:So this is a classification defect only. No adapter needs to change.
2. An unrecognised
model_typefalls through instead of refusingfamily_ofreturns the declared type, or""when the config cannot be read; anythingnot in
SEGMENTER_FAMILIESis handed to the detector adapter. So a model this build hasno adapter for — and a config nothing could parse — is served by whichever adapter
happens to be the fallback, and fails later with a message about that adapter's prompt
kind rather than about the model.
Defect 1 is one instance of this. A wrong classification produces error messages that lie
about the user's model, which is worse than an honest refusal.
What it should do
model_typethe lockedtransformerscan serve for image-level pointprompting resolves to the point-prompt adapter.
model_typeis refused with a message that says the model type is notsupported and lists the ones that are — never resolved to a family by fallback.
Scope
The resolver and its error message. The suggested default model id is correct and does
not change; nothing about the suggest route or the editor changes.
cf. #424, #418