Add any Hugging Face GGUF model, not just the curated catalog#38
Open
sachin-detrax wants to merge 1 commit into
Open
Add any Hugging Face GGUF model, not just the curated catalog#38sachin-detrax wants to merge 1 commit into
sachin-detrax wants to merge 1 commit into
Conversation
The local model list was a fixed array of nine models compiled into
models-catalog.ts. Anything else on Hugging Face was unreachable without
running llama-server by hand or waiting for a maintainer to ship a new
release.
Models are now open-ended. A user-added entry is a full LocalModelDef
with a `custom-` prefixed id, persisted in localModels.customModels
(config v33 -> v34, older files inherit []). loadConfig registers them
in a catalog registry, so getLocalModelDef / isKnownLocalModelId resolve
curated and custom entries through one lookup and every existing
consumer -- daemon start, installer, TUI rows, CLI -- picked them up
without changes.
Entry points:
TUI "+ Add a model from Hugging Face..." pinned under Local text
models. Enter opens a prompt taking either a reference (resolve
and add) or free text (search, then pick by digit).
CLI models add <ref> / models search <query>.
/models add and /models search from the prompt.
References accepted: repo URLs, /resolve/ and /blob/ file URLs,
hf://owner/repo[@rev]/file.gguf, a pasted `hf download ...` command
(one- or two-argument, trailing flags dropped), and bare owner/name.
Naming only a repo picks a 4-bit quant and any mmproj projector, which
makes vision repos come out vision-capable. HF_TOKEN is honoured for
gated repos, including on the weights download.
hf:// is parsed by hand rather than with `new URL`: that puts the owner
in the host slot and lowercases it, and HF owners are case-sensitive, so
hf://Qwen/... would silently 404.
Two rendering defects surfaced while testing this, both pre-existing and
both able to shred the panel on any daemon failure:
- daemonError / errorLine are view-model fields rendered as a single
<Text> in a one-row slot, but a failed llama-server start stored a
4KB, ~25-line log tail. Ink overlaps lines when a frame outgrows its
budget rather than clipping: 48 rows into a 24-row budget. Flattened
on write and on read, with wrap="truncate-end" on the one-row slots
-- flattening alone still wrapped a long single line.
- llm-panel computed its list budget without subtracting the modals
and banners it renders above the list, unlike local-models-panel:
42 rows into 24, and 66 with both defects together.
Flattening the message hid the real cause behind "did not become
healthy", so extractLoadFailure now lifts the diagnostic line out of the
log tail into the message head. Full detail still reaches the runtime
feed and the L logs tab.
Routing between "add this model" and "set this base URL" goes through
looksLikeHuggingFaceReference, stricter than the parser: a bare
owner/name is ambiguous with 192.168.1.5/api and stays behind the
explicit `add` verb. An inline regex here had already drifted once --
it knew about huggingface.co but not hf://, so /models hf://... was
silently persisted as http://hf://... instead of adding the model.
Tests: HF reference parsing and quant/projector selection, add-row
placement and prompt key handling (including panel hotkeys not stealing
characters mid-URL and digits only picking when results show), rendered
frames, and frame-height regressions asserting height <= budget.
Closes AtomicBot-ai#27
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #27.
The local model list was a fixed array of nine models compiled into
models-catalog.ts. Anything else on Hugging Face was unreachable without runningllama-serverby hand or waiting for a maintainer to ship a new release. Models are now open-ended.A user-added entry is a full
LocalModelDefwith acustom-prefixed id, persisted inlocalModels.customModels(config v33 → v34; older files inherit[]).loadConfigregisters them in a catalog registry, sogetLocalModelDef/isKnownLocalModelIdresolve curated and custom entries through one lookup — daemon start, installer, TUI rows and CLI all picked them up without changes.Entry points
+ Add a model from Hugging Face..., pinned under Local text models. Enter opens a prompt taking either a reference (resolve and add) or free text (search, then pick by digit).models add <ref>andmodels search <query>./models addand/models search.References accepted
/resolve/and/blob/file URLshf://owner/repo[@rev]/file.ggufhf download ...command (one- or two-argument, trailing flags dropped)owner/nameNaming only a repo picks a 4-bit quant and any mmproj projector, so vision repos come out vision-capable.
HF_TOKENis honoured for gated repos, including on the weights download.hf://is parsed by hand rather than withnew URL: that puts the owner in the host slot and lowercases it, and HF owners are case-sensitive, sohf://Qwen/...would silently 404.Two pre-existing rendering defects fixed along the way
Both surfaced while testing this, and both could shred the panel on any daemon failure:
toStatusLine()truncates at every error-message write and at the render point, withwrap="truncate-end"on the InkText.LlmPanelframe budget did not account for overlay modal / banner rows, so opening a modal during an error overran the frame.estimateOverlayRows()now covers them.Regression tests pin the frame height for both.
Also
extractLoadFailurepulls the diagnostic reason out of llama-server logs so a failed load reports why rather than a generic message.Tests
35 files changed, +2104/−52, including new suites for the HF reference parser, panel rendering, frame height, and the reducer.
The 6 failures are identical by name on
mainand on this branch (ChatLog/SplashBanner/TuiAppsmoke,llm-panel selectors,persistEmbeddingHybridRecall) — pre-existing, untouched by this change. 38 tests added, none broken.🤖 Generated with Claude Code