Only allow generative (text/vision) models in the chat model picker - #102
Conversation
The chat model picker was a denylist (not speech), so encoders like google-bert/bert-base-uncased, embeddings, ASR and time-series models stayed selectable. Selecting one pre-loads it as text_generation and aborts server startup (Model type bert not supported), bricking the app. - Detect generative models by causal/seq2seq architecture in addition to the name heuristics, so real chat models (Phi, SmolLM, …) are recognized and encoders are not. - Make the picker an allowlist: only text-generation-capable (.text/.vision) models are selectable. - Skip pre-loading a stale non-generative selection so the server still starts.
|
Please double check this doesn't affect images which are supported in the chat version in images tab |
|
Also, it doesn't affect omni models like Gemma 4 which might use that tag |
|
yeah @Blaizzy, great point about omni models, added a small patch for that, for the img page, i think we can wait to merge this until a new pr for folding the img feature into the chat page. i think @lucasnewman said it'll be a follow up work |
I added that in #93 so you should be good -- model availability is type-aware now for non-language models (image, stt, tts). |
lucasnewman
left a comment
There was a problem hiding this comment.
LGTM but see comment about making sure to pick up all possible model roots.
…olders Add a shared NativSettings.modelSearchRoots (primary folder + user-added additionalModelSearchPaths + HF hub cache) and use it in the chat-picker's non-generative check instead of re-deriving the roots there, which had omitted the user's custom model folders.
wait, confused, the release shows sep img and chat page |
Problem
The chat model picker is a denylist (
LocalModel.isEligibleForLanguageModelPicker=!speechToText && !textToSpeech), so non-generative models a user downloads — encoders likegoogle-bert/bert-base-uncased/FacebookAI/xlm-roberta-base, embeddings, ASR, time-series — still appear and can be selected as the chat model. Selecting one and starting the server pre-loads it astext_generation; mlx-vlm/mlx-lm has no matching architecture, so startup aborts:The app is then effectively bricked — the server won't come up and the menu bar is unresponsive, with no in-UI way to recover.
Fix
.textor.vision); only models with no generative capability — encoders, embeddings, ASR, forecasting, pure image-gen — drop out. Multimodal/VLM models are not excluded..textfrom causal/generation architectures (*ForCausalLM/*ForConditionalGeneration/*LMHeadModel) in addition to the existing name heuristics, so legitimate chat models (Phi, SmolLM, …) aren't accidentally filtered out.config.jsonshows only non-generative architectures, its--modelpre-load flag is dropped (load-on-demand) so the server still starts, with a log note. The check is conservative — any uncertainty leaves the model untouched.Screenshots to follow.