You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
GET /api/loaded gives each model a status, loaded or sleeping, and taraengine ps shows it in a STATUS
column; taraengine load and taraengine doctor say when a model sleeps.
An import from hf:// or a local path takes its native context window from the file's header when it is pulled,
and its default window is no larger; imports pulled by 0.1.0 get theirs when the daemon starts.
An import keeps the capabilities it showed while it ran, across unloads and restarts.
pkg/client: Error.NotJSON marks a refusal whose body is not the engine's error JSON; such a body is shown as
one line of at most 300 bytes.
Changed
The memory figure and the capabilities of a running model are recorded whoever started it: a model that a /v1
request brings up gets them from a later read of the model list. Reading a model's properties never makes
llama-server load it.
POST /api/load answers at once for a model that sleeps at the requested window, and waits for a model that is
still loading instead of refusing the load.
The end of llama-server's log that GET /api/models/{name} carries keeps line breaks and tabs and drops every other
control character.
A pre-release tag such as v0.2.0-rc.1 is released with the [Unreleased] section of this file as its notes,
unless it has a section of its own.
The catalog runs a q8_0 KV cache (cache-type-k and cache-type-v) for gemma4:e4b, glm-4.7-flash, qwen3.6:27b,
qwen3.8:27b and qwen3.6:35b, which holds less memory at the same window; the README's catalog table names each
model's KV cache. A key set in preset_global wins over a catalog model's own preset for that key, and with
flash attention turned off there a catalog model keeps an f16 V cache, which llama.cpp quantizes only with it.