Repository navigation
lmgw on Windows, as a native app with its own installer and updater; lmgw as a Podman image for a server; and a VRAM admission that starts what would run, with a warning, instead of refusing it.
- lmgw on Windows. A per-user installer (no administrator needed) and an updater that checks the download's sha256 before it starts the installer. There is no Podman on Windows: llama.cpp, stable-diffusion.cpp and audio.cpp run as native processes, each in a Windows job that ends with lmgw, from the engines' own release builds.
- Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under
%LOCALAPPDATA%\lmgw\engines. A CUDA build with no kernels for the card is noticed, and the start goes on with the Vulkan build. The Engines page has a Windows tab with each build, why it was chosen and Try CUDA. - It looks like Windows. The window follows the system's dark or light mode and the accent colour, and the title bar takes the sidebar's colour.
- The machine check, VRAM accounting (per-process figures from the driver), the private files and keys (owner-only ACLs, Credential Manager) and links (the shell's open verb) all have Windows versions.
- Not yet on Windows: agents, git engine builds, PDF text extraction, the tray menu in dark mode and the Windows 10 theme path.
- Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under
- lmgw as a server image.
lmgw-serveris the gateway and dashboard without the tray app, in a Podman image that runs its model, agent and build containers with a nested Podman, started by a rootless Quadlet unit. A start with a pending migration first copies the database tobackups/. Settings the unit pins show read-only, with Settings → About saying where each came from.docs/server.mdis the guide.- TLS on the main port (TLS 1.2 and 1.3, the pair reloaded when its files change) and proxy mode for a trusted reverse proxy's forwarded scheme, host and client.
- Accounts. Names and passwords (argon2id) with sessions, set-password links, a sign-in throttle and
lmgw-server accountcommands. The server always has them; the desktop has a switch under Settings → Network & access → Accounts. Keys and devices say who made them. - Audit log. With accounts on, every administrative write, writing tool call and sign-in is an entry, secrets redacted, kept for a number of days you set, shown as the Activity list.
- The device CA's key is sealed under the image's wrapping key, and the update notice and the machine check speak for the server.
- VRAM admission starts what would run. A model that does not wholly fit the card's free memory is no longer refused when it can work partly on the CPU; it starts and says "It will work but only fits partially on the GPU, expect slowdowns", on the Models page, in the start answer and in the test.
- Image rows with
offload_to_cpuor CPU/disk parameters are charged their on-card part, learned per engine build. A pipeline that is all on the card and larger than the card is still refused, with what offloading would do. - llama.cpp rows are charged what lands on the card: a partial
--n-gpu-layers,--no-kv-offloadand MoE expert offload. A row that llama.cpp's--fitwould shrink is fitted by lmgw (context, then experts, then layers) against its own figures, and the start renders the result as explicit flags. - Several cards in one engine's view are not fitted yet.
- Image rows with
- Auto headroom. Under Settings → GPU → Admission the headroom is auto by default: the largest rise of other programs' memory use over the last 7 days plus a margin per engine class while a model starts, or the fixed value. It uses the fixed value until a day of readings exists. Forget learned growth… clears the history.
- MoE offload. Chat rows have Experts kept in memory (
--n-cpu-moe) and GPU cache for experts in memory (--moe-cache-mib), and admission charges only what stays on the card, by tensor name, draft model included. - Dashboard. Settings → About; read-only pinned settings; the Activity list; the Accounts card.
Breaking / upgrade notes
- Clients: Kai, Kai mobile and Kai for Firefox move to this release. The update manifest gains an
nsisentry besiderpm, and thelmgw-updatecrate's artifact type changed with it. - Migrations 0101 to 0106 run on start: accounts and sessions (0101), the audit log (0102), image idle and working-set learning (0103, 0104, 0105: an image row's learned peak and budget now belong to the engine build they were learned under and are learned again after an engine change) and the outside-use history (0106).
- New settings and fields:
vram.headroom_mode(auto or fixed) besideheadroom_mb;n_cpu_moeandmoe_cache_mibon chat rows;trusted_proxies,tls.*andaccounts.audit_days. Warnings on start/restart answers and image tests; the op list hasengine_releases. - Auto headroom replaces the fixed one by default; set
headroom_modeto fixed to keep the value you had. - The server image pins its listeners and keys; on the desktop an account switch starts at off.
Fixes
- A partial image start is held to what admission gave it and learns from a failed generation; a fitted llama-server container is adopted at boot.
- A first llama.cpp start under a declared budget is no longer refused as too large before the cards are known.
- A new row's id is settled under the routing-write lock, so two Get it runs cannot pick the same one.
- A size over
max_body_mbanswers 413 on Windows, not a reset; the main TLS paths must be absolute; a download without a dialog goes to the Downloads folder. - Podman's known-harmless stderr lines are no longer quoted back as errors.