Skip to content

lmgw 0.7.0

Latest

Choose a tag to compare

@github-actions github-actions released this 11 Nov 23:29
· 2 commits to main since this release

lmgw on Windows, as a native app with its own installer and updater; lmgw as a Podman image for a server; and a VRAM admission that starts what would run, with a warning, instead of refusing it.

  • lmgw on Windows. A per-user installer (no administrator needed) and an updater that checks the download's sha256 before it starts the installer. There is no Podman on Windows: llama.cpp, stable-diffusion.cpp and audio.cpp run as native processes, each in a Windows job that ends with lmgw, from the engines' own release builds.
    • Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under %LOCALAPPDATA%\lmgw\engines. A CUDA build with no kernels for the card is noticed, and the start goes on with the Vulkan build. The Engines page has a Windows tab with each build, why it was chosen and Try CUDA.
    • It looks like Windows. The window follows the system's dark or light mode and the accent colour, and the title bar takes the sidebar's colour.
    • The machine check, VRAM accounting (per-process figures from the driver), the private files and keys (owner-only ACLs, Credential Manager) and links (the shell's open verb) all have Windows versions.
    • Not yet on Windows: agents, git engine builds, PDF text extraction, the tray menu in dark mode and the Windows 10 theme path.
  • lmgw as a server image. lmgw-server is the gateway and dashboard without the tray app, in a Podman image that runs its model, agent and build containers with a nested Podman, started by a rootless Quadlet unit. A start with a pending migration first copies the database to backups/. Settings the unit pins show read-only, with Settings → About saying where each came from. docs/server.md is the guide.
    • TLS on the main port (TLS 1.2 and 1.3, the pair reloaded when its files change) and proxy mode for a trusted reverse proxy's forwarded scheme, host and client.
    • Accounts. Names and passwords (argon2id) with sessions, set-password links, a sign-in throttle and lmgw-server account commands. The server always has them; the desktop has a switch under Settings → Network & access → Accounts. Keys and devices say who made them.
    • Audit log. With accounts on, every administrative write, writing tool call and sign-in is an entry, secrets redacted, kept for a number of days you set, shown as the Activity list.
    • The device CA's key is sealed under the image's wrapping key, and the update notice and the machine check speak for the server.
  • VRAM admission starts what would run. A model that does not wholly fit the card's free memory is no longer refused when it can work partly on the CPU; it starts and says "It will work but only fits partially on the GPU, expect slowdowns", on the Models page, in the start answer and in the test.
    • Image rows with offload_to_cpu or CPU/disk parameters are charged their on-card part, learned per engine build. A pipeline that is all on the card and larger than the card is still refused, with what offloading would do.
    • llama.cpp rows are charged what lands on the card: a partial --n-gpu-layers, --no-kv-offload and MoE expert offload. A row that llama.cpp's --fit would shrink is fitted by lmgw (context, then experts, then layers) against its own figures, and the start renders the result as explicit flags.
    • Several cards in one engine's view are not fitted yet.
  • Auto headroom. Under Settings → GPU → Admission the headroom is auto by default: the largest rise of other programs' memory use over the last 7 days plus a margin per engine class while a model starts, or the fixed value. It uses the fixed value until a day of readings exists. Forget learned growth… clears the history.
  • MoE offload. Chat rows have Experts kept in memory (--n-cpu-moe) and GPU cache for experts in memory (--moe-cache-mib), and admission charges only what stays on the card, by tensor name, draft model included.
  • Dashboard. Settings → About; read-only pinned settings; the Activity list; the Accounts card.

Breaking / upgrade notes

  • Clients: Kai, Kai mobile and Kai for Firefox move to this release. The update manifest gains an nsis entry beside rpm, and the lmgw-update crate's artifact type changed with it.
  • Migrations 0101 to 0106 run on start: accounts and sessions (0101), the audit log (0102), image idle and working-set learning (0103, 0104, 0105: an image row's learned peak and budget now belong to the engine build they were learned under and are learned again after an engine change) and the outside-use history (0106).
  • New settings and fields: vram.headroom_mode (auto or fixed) beside headroom_mb; n_cpu_moe and moe_cache_mib on chat rows; trusted_proxies, tls.* and accounts.audit_days. Warnings on start/restart answers and image tests; the op list has engine_releases.
  • Auto headroom replaces the fixed one by default; set headroom_mode to fixed to keep the value you had.
  • The server image pins its listeners and keys; on the desktop an account switch starts at off.

Fixes

  • A partial image start is held to what admission gave it and learns from a failed generation; a fitted llama-server container is adopted at boot.
  • A first llama.cpp start under a declared budget is no longer refused as too large before the cards are known.
  • A new row's id is settled under the routing-write lock, so two Get it runs cannot pick the same one.
  • A size over max_body_mb answers 413 on Windows, not a reset; the main TLS paths must be absolute; a download without a dialog goes to the Downloads folder.
  • Podman's known-harmless stderr lines are no longer quoted back as errors.