Repository navigation
b11714
Pre-releaseui : Discover & Download Models from HF Hub (#29584)
- common : resolve - download tags and list cached sidecars
A Q4_0-mtp style tag now resolves the sidecar file when no model file matches it, so a solo draft or mmproj download actually pulls the file. Cached sidecar files list as their own entries so the state survives a restart, and removing such a tag deletes only the sidecar.
Assisted-by: pi:zai-org/GLM-5.3
- ui : optional sanitized raw HTML in markdown
Add an allowHtml prop to MarkdownContent: raw HTML found in the markdown is
rendered after DOMPurify sanitization instead of being escaped as literal
text. Default stays escaped.
Assisted-by: pi:GLM-5.3-Flash
- ui : add the discover models components
The Discover surface: the curated-catalog list with its search and
skeletons, the details pane with its readme, metadata, Hub stats and
download options, and the standalone download progress bar that replaces
the plain one. The quant download button sizes its select with a new xs
trigger, so the select gains that size and the muted box look. The store
stays incomplete when a repo fails, so the next mount retries it.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
- ui : put the discover view in the models manager
The manager dialog gains its second view: the Discover Models call to
action fades the table out, slides the title with its compass mark, and
fades the discover surface in; the arrow button returns to the table.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
Conflicts:
tools/ui/src/lib/components/ui/button/button.svelte
- ui : add the Enable Discover Models setting
The flag gates browsing and downloading HuggingFace GGUF models, so it is
declared on the discover layer; the repo-tree lookup in the manager layer
above reads it from here.
Assisted-by: pi:llama.cpp/DeepSeek-V4.1-Flash
- ui : resume model downloads from the pause point
A stopped download keeps its partial file instead of deleting it, so
re-posting the tag resumes from disk. The first progress record now
carries the resume offset, the paused snapshot survives into the
resumed entry, and a pause settles as soon as it is requested.
Assisted-by: pi
- ui : lead the discover list with downloads
The discover sidebar lists running downloads above the suggested models,
and steps aside while a query is active. A paused download stays in the
manager table, where it is resumed or dropped.
Assisted-by: pi
- ui : make resuming a paused download work
Re-posting the tag of a paused download hit the router's "already
exists" check, because a paused download parks its entry as DOWNLOADED
until the next reload: a download entry that is not running is now
replaced and resumed. The UI keeps the paused row when the request still
fails, instead of dropping it and looking finished.
Assisted-by: pi
- ui : remember unfinished model downloads
A partial file is named after its content hash, so nothing on disk says
which model it belongs to once the router restarts. The router now keeps
the tags of the downloads it started, clears a tag when the download
finishes or is deleted, and lists what is left as still downloading
without starting anything. The UI adopts those tags as paused on every
list fetch, so a refresh or a fresh open shows an incomplete download and
can resume it from the files already on disk.
Assisted-by: pi
- ui : show a paused download's state on its chip
A paused quant chip reads as paused at rest, the way a running one shows
its spinner, and offers the resume action on hover. The selector's
download row gets a View in Discover action beside its cancel.
Assisted-by: pi
- ui : collapse the download options by default
The copyable download command starts collapsed so the option rows
lead; expanding it stays one click away.
Assisted-by: pi
- ui : keep decision models out of the hub lists
Decision models answer with calibrated probabilities instead of text,
so llama-ui cannot run them: the hub data layer drops them from the
catalog and every search, keyed by the decision-model tag or a
text-classification pipeline.
Assisted-by: pi
- ui : read download tags as quant and sidecar
A <repo>:<quant>-<sidecar> tag now parses into its quant and sidecar
instead of one opaque quant token, and ModelsService.draftSidecarsByRepo
collects the router listing's sidecar-only entries per repo, replacing
the inline copy in the models store.
Assisted-by: pi
- ui : cluster draft downloads under their main model
Tracked downloads group by repo: the main weights lead and the draft
sidecars ride inside their item as compact sub-lines, on the selector,
the discover list and the manager table. The main model's controls
pause, resume and cancel the whole cluster, and the manager's download
rows select into the configuration pane.
Assisted-by: pi
- ui : act on tracked downloads from the table rows
The table passes its clusters' draft sub-lines to the model rows. A
row's actions cascade pause and resume, an in-flight row offers Cancel
download while a paused one keeps Delete from disk, and the confirm
dialog's copy follows the state.
Assisted-by: pi
- ui : give downloads a configuration pane of their own
Selecting a tracked download opens the pane in a download mode: live
progress with pause, resume and cancel, and See in Discover, instead of
metadata and load actions that a model without files cannot offer.
Assisted-by: pi
- ui : make the discover details self-fetching and responsive
ModelsDiscoverDetailsContainer owns the details, tree and readme
fetches so hosts outside the discover view can render the pane, and the
details layout adapts to narrow panes with container queries instead of
viewport breakpoints.
Assisted-by: pi
- ui : download a matching draft with the main model
A main quant download now pulls the quant-matched draft of the most
preferred kind the repo ships (DFlash before MTP), fire-and-forget, so
speculative decoding is ready when the main lands. Manual draft chips
stay available; the options row also collapses to one column in narrow
panes.
Assisted-by: pi
- ui : keep discover off by default and couple it with the hub setting
Discover Models starts unchecked. Enabling it turns the Hugging Face
Hub setting on and disabling it turns it back off, so the checkbox
never lies; with discover off the hub setting stays available on its
own for the display data. Stored and reset state are corrected the same
way on load.
Assisted-by: pi
- ui : open the manage models dialog on the manager view
The dialog now opens on the manager instead of returning to the last
view, discover entries still land on the discover view while the
setting allows it, and the discover surfaces follow the setting's
state.
Assisted-by: pi
- ui : give the discover search room for its focus ring
The sticky search bar insets from the panel edges so the focus ring
stays inside the scroll container, and rows scroll behind an opaque bar
instead of through it.
Assisted-by: pi
- common: settle sidecar names on the listing and drop unfinished partials
An infix or trailing sidecar token is ambiguous: draft heads use it
(Model-MTP-BF16.gguf), so do full models with embedded heads
(Model-Q8D-MTP.gguf). Such a name is a sidecar only next to token-free
main weights of the same family, while mmproj or imatrix anywhere in the
name, or a leading token, decides alone (mmproj_f16.gguf).
A quant tag always carries its bit count, so a sidecar named after no
quant (mtp-Model-it.gguf) gets the bare tag: the listing, the plan and
remove all use it. An imatrix tag resolves into a plan slot of its own,
downloaded but never wired into the params.
remove resolves the tag the way a download does and deletes the partial
files of every planned blob, so cancelling a running download or
dropping one left unfinished by an earlier run leaves nothing on disk.
The router passes its token and releases its lock around the call, and
records the unfinished downloads with an atomic write.
llama download succeeds on a sidecar-only plan and prints its paths.
- ui : move dialog views and draft states into enums and constants
The models dialog's views become the ModelsDialogView enum, the draft
subline's states join ModelRowDownloadState, the view fade and the
copy-feedback reset move to ui constants, and the dialog's compass
marks come from DISCOVER_ICON.
Assisted-by: pi
- ui : drop the unused discover surface
The dead openDiscoverModels twin, the commented-out command divider,
the details pane's unused onClose and its header's close button, and a
redundant type cast leave; the command's copy feedback joins the shared
reset constant.
Assisted-by: pi
- ui : trim the download option comments and highlight the chat template
The comment verbosity the options panel and its chips carried shrinks
to the non-obvious why, and the chat template dialog renders through
SyntaxHighlightedCode as jinja instead of a plain pre block.
Assisted-by: pi
- common: only a .gguf file is a sidecar, and mmproj/imatrix as whole words
A non-GGUF file named like a sidecar (quantization-mmproj-Q8_0.json) is
picked as the mmproj, and a sidecar word inside a longer word of a model
name (Phi-2-OmniMatrix.i1-Q4_K_M.gguf) drops the model.
- ui: dedupe draft sidecar badges per quant
The draft sidecar badges deduped by kind and repo without the quant, so
with a draft cached at two quants only one badge survived, and a delete
of the other main quant found no badge to remove its sidecar with.
Also restores the getDownloadEntries doc lost when the clusters method
landed above it.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: adopt the incomplete download status and drop the paused-ids registry
The store inferred an unfinished download from a downloading entry it
held no local state for, which misread a download running in another
tab as paused and offered a resume the server rejects. The listing
carries the "incomplete" status now, so the adoption keys off it
directly.
The paused ids no longer need to survive in localStorage: the server
remembers them across restarts, and the progress snapshot was stale
after a reload anyway. The stale localStorage key is removed.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: quant-match the pause and cancel cascades
Pause, resume and cancel cascaded to every draft sidecar of the repo
while delete only took the quant-matched ones, so canceling a Q4_0
download also killed a Q8_0 draft that belongs to another quant's
main. Every cascade acts on the quant-matched drafts now, matching
what a delete takes with it.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: collapse the duplicated draft subline markup
The downloaded and in-flight branches repeated the identical connector,
label and badge markup; one block with a conditional tail keeps the two
states in sync.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: scrub positioned styles out of sanitized model card html
The style attribute stays allowed for the KaTeX output, which let a
third party card draw a fixed overlay over the app. The sanitize hook
drops position:fixed and position:absolute values and leaves the KaTeX
layout styles alone.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: run the discover mount fetch once and reuse the shared repoOf helper
fetch() toggles its loading flag synchronously inside the mount effect,
which re-ran the fetch on every toggle and retried failed catalog repos
endlessly; the effect now untracks the call. The local repoOf copy and
the duplicate services import give way to the shared helpers.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: narrow the spec-type map to the draft sidecars
SPEC_TYPE carried empty entries for the aux sidecars it is never
indexed with; the map and the command component now type the badge as
a draft sidecar through a draftBadgeOf helper instead of a cast.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: name the decision-model tags and markdown cache keys
The decision-model hub tag and pipeline tag move into the huggingface
constants next to the other pipeline tags, and the markdown transform
cache prefixes get names instead of bare one-letter tokens.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: read the discover setting through one truthiness check
Three places read the setting with !== false and one with Boolean();
all four read it the same way now.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- ui: compare the models dialog view against its enum
The template compared the view state against the 'manage' literal in
three spots while the script used the enum.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
-
nit: Remove
+before draft sidecar badge -
server: list unfinished downloads as incomplete
A download that never finished was reported as "downloading", the same
status a live download carries, so a client could not tell a running
download from a dead one. The tags remembered in unfinished-downloads
now list under a dedicated "incomplete" status, without an instance.
Assisted-by: pi (zai-org/GLM-5.3-Flash)
- server: download a model with its sidecars as -hf does
Resolution goes back to master. POST /models takes the sidecar kinds to
pull with the model (mmproj by default) and the download child gets them
as -hf flags, so the server picks the files. /models lists real models
only, each with the sidecars it has on disk, and incomplete downloads
keep their sidecars across restarts. A stopped download keeps its
partial files for the next attempt, and DELETE purges them through the
same plan the download follows.
- ui: download sidecars as parameters of their model
The download options keep their chips, but a chip no longer posts a
sidecar tag: the store maps it to its model and the sidecar kinds to
pull, the quant chip takes its companion draft in the same download,
and draft badges come from the sidecars each cached model lists.
The server adds the sidecars a cached model lacks when it is posted
again, and DELETE removes a single sidecar of a cached model.
- ui : mirror the server's sidecar lookup in draft badges
The server resolves a sidecar by the case-sensitive <kind>- token in the same
directory as the weights. The UI matched the token loosely, so a badge could
offer a draft the server never pulls.
Assisted-by: DeepSeek-V4.1-Flash
- ui : post the model's sidecar kinds with its download
The store keeps one download per model: the chip's kinds ride along with the
model's own sidecars, and a download row no longer expands into one row per
sidecar. Draft badges carry no quant of their own yet, so a sidecar of a cached
model is never labelled with the model's quant.
Assisted-by: DeepSeek-V4.1-Flash
- ui : join a quant's draft into its download chip
One chip per quant, its draft badged inside it and its size covering both
files. The draft is the file the server would pick: the same directory, the
exact quant, else the closest bit depth, so a 4-bit model can pair with a
16-bit head.
Assisted-by: DeepSeek-V4.1-Flash
- ui : drop the draft download rows from the manager and selector
A download is one row per model again: the drafts ride with it, and the rows
read the sidecars on disk off the listing instead of pairing sidecar entries
with their model. Pause, resume and cancel act on the model's own download.
Assisted-by: DeepSeek-V4.1-Flash
- ui : keep the discover download list one row per model
The list leads with one row per downloading model, its sidecars as sub-lines.
A row badges only the draft kinds the server can resolve, so a badge never
outlives the file its lookup would pick.
Assisted-by: DeepSeek-V4.1-Flash
- ui : keep the discover chips readable in dark mode
The metadata chips, the chat template button and the View on Hugging Face link
carried a faint border and no surface over the dialog background, so they were
near invisible. They take the tertiary button surface now, and their icons size
with size-* so the variant's base rule leaves them alone.
Assisted-by: DeepSeek-V4.1-Flash
- ui : carry the draft through the download options
One chip per main quant, its draft badged on it and the chip's downloaded state
covering both files. The drafts sit behind an opt-in checkbox with an inline
type pick, and each row opens its own command: it pins the row's quant and only
names the draft type, so -hfd is gone.
Assisted-by: DeepSeek-V4.1-Flash
- ui : line the details skeleton up with the loaded pane
Same root padding and container context as the loaded view, so the container
queries resolve alike. The download options card pads its blocks instead of
itself, the readme gets its card, and a description line stands in for the
catalog paragraph.
Assisted-by: DeepSeek-V4.1-Flash
- common : drop the partials of a removed sidecar
Removing a sidecar alone left the .downloadInProgress files of the sidecars a
download had planned in blobs/. The removal purges them through the same plan
the download follows, and the test covers a finished draft beside an unfinished
one, whose partial goes while the model keeps its files.
Assisted-by: DeepSeek-V4.1-Flash
- server : name why a download is refused
A model that is already downloading, or that is running, answered "already
exists". It now says which: "is already downloading", or "is running, unload it
first", both before the resumable check. The router test covers the two.
Assisted-by: DeepSeek-V4.1-Flash
- ui : post a download once per click
Two quick clicks on the same chip posted twice, and the second answer was
"already exists". The store tracks the model as pending, returns early while it
is pending or in flight, and reads it as in flight before the server answers, so
the chip shows the download at once. The new client test pins the flows: one
POST per double click, the chip reading downloaded once the list reports the
model, a resume keeping its sidecars, a cancel dropping the whole download, and
a reload restoring the paused state.
Assisted-by: DeepSeek-V4.1-Flash
- ui : keep a plain download row the height of the others
The sub-lines wrapper of a download row was rendered even with nothing in it, so
its padding made the row taller than a plain one and the overlaid progress bar
sat in that empty strip. It renders only with sub-lines now, in the selector and
in the discover list, and the selector row takes the phone padding its
neighbours have.
Assisted-by: DeepSeek-V4.1-Flash
- ui : badge a repo's draft once
The args of a loaded model and the listing of a cached one report the same draft
in different words, one with the file's quant and one with the kind alone, so
the merge kept both and a repo row showed the same kind twice. The badges merge
per kind now, the args one winning as it knows the quant, and the test pins both
sources of a real repo.
Assisted-by: DeepSeek-V4.1-Flash
- server : report the sidecars of a preset or a download on its way
A model the preset also names is registered as a preset, so its drafts were
never reported and no row could badge them. A download on its way reported none
either, though the record of the kinds it pulls is right there: those win over
the plan now, the plan resolves through the preset's hf-repo, and a models dir
stays out of it.
Assisted-by: DeepSeek-V4.1-Flash
- ui : badge a download's drafts and open its model from the row
A draft still coming down badges the model id now, the way a landed one does,
while an on-disk draft keeps its checked sub-line. The row itself opens the
model, the status slot beside it pauses or resumes, and the manager resolves a
focus on a download row so its panel opens.
Assisted-by: DeepSeek-V4.1-Flash
- common : report a removed sidecar that only had a partial
Removing a sidecar alone dropped the result of the partial purge, so a sidecar
that only ever landed as a .downloadInProgress blob answered false and the UI
had no reason to refetch. The purge result is folded into the return now, and
the test covers a partial-only removal.
Assisted-by: DeepSeek-V4.1-Flash
- ui : ignore a pause while the download POST is still on its way
The chip reads as downloading before the server answers, so a quick second click
unloaded a model that had not started and left the store neither in flight nor
paused. A pause returns early while the POST is pending, before it touches the
feed or the paused marker. The client test holds the POST open and pins it.
Assisted-by: DeepSeek-V4.1-Flash
- ui : offer a missing draft as its own chip
A model already on disk without its draft had no way to add it: the quant chip
carried the draft, so it read as a download of both. The quant chip keeps to its
own state now, its joined draft shows only while it rides along or is on disk,
and a downloaded quant missing its draft gets a chip of its own - badge, size,
download - that posts the model with that kind, so the server keeps the files
already there and pulls the draft.
Assisted-by: DeepSeek-V4.1-Flash
- server : list a paused sidecar download by the files on disk
A model on disk whose sidecar download stops keeps its unfinished record,
and the listing reported the kinds that record plans as if they were on
disk: the UI checked a draft that never landed and offered no way to add it.
The planned kinds now stand in only while the download runs.
-
chore: Clean up comments
-
server : list sidecars only for a model that resolves from a HF repo
A preset on a local file has no HF repo, yet the listing resolved its name
as one: the plan threw on the invalid repo format and GET /models failed
for every model. The sidecars now come from the repo a preset names or a
cached model is listed under, and a local file lists without them.
- common : fix download cancel regression
Signed-off-by: Adrien Gallouët angt@huggingface.co
- common : preserve model partials when removing sidecars
Signed-off-by: Adrien Gallouët angt@huggingface.co
- server : one sidecar table and an unfinished downloads class
Address review from @ggerganov.
The sidecar kinds live in one table in common/download, mapping each name to
its download option and plan file; the server validates, builds its options
and lists cached sidecars through it instead of its own lists.
The unfinished downloads move into server_unfinished_downloads, a record with
its own mutex and a struct per entry, and server_models exposes them through
unfinished_downloads_get and unfinished_downloads_remove.
- common: one sidecar table, and removals that keep shared files
The resolution, the cached listing and the removal follow a single
table of sidecar kinds, so a kind is declared once with its file
keyword, whether it is a draft, its option and its plan file. The
server reads the same table for the kinds it accepts, the default
sidecars and the child flags.
Removing a tag keeps the files another cached tag still loads, such as
an mmproj named after the removed quant. A sidecar removed on its own
goes for every tag that resolves it.
An unfinished download records the exact paths of its partials when
it starts. A removal deletes those without asking the hub, and keeps a
partial another recorded download also writes. The record is taken
once the child is spawned, and the files go under the models lock, so
a download of the same tag cannot start while they are removed.
- server: drop stale partials and refuse a tag that resolves nothing
Recording a download again deletes the partials of the earlier record
that the new plan no longer writes, such as those of an older hub
revision or of a sidecar the new request leaves out.
A posted tag that resolves no file of the repo is refused before the
download starts. Removing a model follows a single path, whether the
reload erased its entry during the wait or not.
- server: fix the CI race between the router download tests
The pytest workers run in parallel on one shared cache, so a download
test could find the file another test had just fetched and see no
progress, or start from the unfinished downloads record another test
wrote. Each download test now runs on an empty cache of its own.
- server: refuse a DELETE of an unknown name and decide downloads in load()
A DELETE of a name that is neither a model nor a recorded download
went to common_download_remove, and without a tag it wiped the whole
repo cache, including the files of a loaded quant or of a download on
its way. Such a name is now not found, as it was before.
A POST that downloads over an existing entry checked its state outside
the lock, so a load of the same model in between got its instance
killed. The checks now run in load(), under the lock, right before the
spawn. The fake load time of the tests applies to model loads only,
since a download worker loads none.
- server: add incomplete_sidecars to the /models API
A sidecar added to a cached model and then paused kept its record, but
the listing showed only the complete model, so a fresh UI lost the
pause. The model entry now carries incomplete_sidecars, the kinds its
unfinished download has not written yet; a POST with the kind resumes
one and a DELETE with it drops its partial.
- common : drop a kept partial when a resume gets 416
Now that partials survive between attempts, an old or complete
partial makes every retry fail with the same 416.
Delete it so the next attempt can start over from zero.
Signed-off-by: Adrien Gallouët angt@huggingface.co
- common: drop a kept partial when a resume gets 200
A server that ignores the range answers 200 to a resume, and the kept
partial then fails every attempt the same way a 416 does. Drop it as
well, so the next attempt starts over without a range.
- server: refuse a download that shares a partial with one on its way
Two downloads of quants that share a file, the mmproj or an mtp picked
by the closest bits, appended to the same partial and renamed a file
with interleaved bytes. A download whose plan writes a partial that a
running download also writes is now refused until that one is done.
- common : check the saved etag before resuming a partial download
Before resuming, compare the etag saved with the partial to the one
the server now serves. If it is missing or different, start over,
so a resumed download never mixes bytes from two versions of the file.
Signed-off-by: Adrien Gallouët angt@huggingface.co
Signed-off-by: Adrien Gallouët angt@huggingface.co
Co-authored-by: Pascal admin@serveurperso.com
Co-authored-by: Adrien Gallouët angt@huggingface.co
Website:
Attestations:
macOS/iOS:
- macOS Apple Silicon (arm64)
- macOS Apple Silicon (arm64, KleidiAI enabled) DISABLED
- macOS Intel (x64)
- iOS XCFramework
Linux:
- Ubuntu x64 (CPU)
- Ubuntu arm64 (CPU)
- Ubuntu s390x (CPU)
- Ubuntu x64 (Vulkan)
- Ubuntu arm64 (Vulkan)
- Ubuntu x64 (CUDA 12) - CUDA 12.8 libraries
- Ubuntu x64 (CUDA 13) - CUDA 13.4 libraries
- Ubuntu arm64 (CUDA 13) - CUDA 13.4 libraries
- Ubuntu x64 (ROCm 10.0)
- Ubuntu x64 (OpenVINO)
- Ubuntu x64 (SYCL FP32)
- Ubuntu x64 (SYCL FP16)
- Linux arm64 (Snapdragon: CPU, Adreno GPU, Hexagon NPU) - setup guide
Android:
Windows:
- Windows x64 (CPU)
- Windows arm64 (CPU)
- Windows arm64 (OpenCL Adreno)
- Windows x64 (CUDA 12) - CUDA 12.4 DLLs
- Windows x64 (CUDA 13) - CUDA 13.4 DLLs
- Windows arm64 (CUDA 13) - CUDA 13.4 DLLs
- Windows x64 (Vulkan)
- Windows arm64 (Vulkan)
- Windows x64 (OpenVINO)
- Windows x64 (SYCL)
- Windows x64 (ROCm 10.0)
openEuler:
- DISABLED
- openEuler x86 (310p)
- openEuler x86 (910b, ACL Graph)
- openEuler aarch64 (310p)
- openEuler aarch64 (910b, ACL Graph)
UI: