Repository navigation
Releases
lmgw 0.7.0
lmgw on Windows, as a native app with its own installer and updater; lmgw as a Podman image for a server; and a VRAM admission that starts what would run, with a warning, instead of refusing it.
- lmgw on Windows. A per-user installer (no administrator needed) and an updater that checks the download's sha256 before it starts the installer. There is no Podman on Windows: llama.cpp, stable-diffusion.cpp and audio.cpp run as native processes, each in a Windows job that ends with lmgw, from the engines' own release builds.
- Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under
%LOCALAPPDATA%\lmgw\engines. A CUDA build with no kernels for the card is noticed, and the start goes on with the Vulkan build. The Engines page has a Windows tab with each build, why it was chosen and Try CUDA. - It looks like Windows. The window follows the system's dark or light mode and the accent colour, and the title bar takes the sidebar's colour.
- The machine check, VRAM accounting (per-process figures from the driver), the private files and keys (owner-only ACLs, Credential Manager) and links (the shell's open verb) all have Windows versions.
- Not yet on Windows: agents, git engine builds, PDF text extraction, the tray menu in dark mode and the Windows 10 theme path.
- Engines download for you. lmgw picks the build per engine from the card and driver (CUDA 13, CUDA 12 where it reaches the card, Vulkan, CPU), checks it against its pin and unpacks it under
- lmgw as a server image.
lmgw-serveris the gateway and dashboard without the tray app, in a Podman image that runs its model, agent and build containers with a nested Podman, started by a rootless Quadlet unit. A start with a pending migration first copies the database tobackups/. Settings the unit pins show read-only, with Settings → About saying where each came from.docs/server.mdis the guide.- TLS on the main port (TLS 1.2 and 1.3, the pair reloaded when its files change) and proxy mode for a trusted reverse proxy's forwarded scheme, host and client.
- Accounts. Names and passwords (argon2id) with sessions, set-password links, a sign-in throttle and
lmgw-server accountcommands. The server always has them; the desktop has a switch under Settings → Network & access → Accounts. Keys and devices say who made them. - Audit log. With accounts on, every administrative write, writing tool call and sign-in is an entry, secrets redacted, kept for a number of days you set, shown as the Activity list.
- The device CA's key is sealed under the image's wrapping key, and the update notice and the machine check speak for the server.
- VRAM admission starts what would run. A model that does not wholly fit the card's free memory is no longer refused when it can work partly on the CPU; it starts and says "It will work but only fits partially on the GPU, expect slowdowns", on the Models page, in the start answer and in the test.
- Image rows with
offload_to_cpuor CPU/disk parameters are charged their on-card part, learned per engine build. A pipeline that is all on the card and larger than the card is still refused, with what offloading would do. - llama.cpp rows are charged what lands on the card: a partial
--n-gpu-layers,--no-kv-offloadand MoE expert offload. A row that llama.cpp's--fitwould shrink is fitted by lmgw (context, then experts, then layers) against its own figures, and the start renders the result as explicit flags. - Several cards in one engine's view are not fitted yet.
- Image rows with
- Auto headroom. Under Settings → GPU → Admission the headroom is auto by default: the largest rise of other programs' memory use over the last 7 days plus a margin per engine class while a model starts, or the fixed value. It uses the fixed value until a day of readings exists. Forget learned growth… clears the history.
- MoE offload. Chat rows have Experts kept in memory (
--n-cpu-moe) and GPU cache for experts in memory (--moe-cache-mib), and admission charges only what stays on the card, by tensor name, draft model included. - Dashboard. Settings → About; read-only pinned settings; the Activity list; the Accounts card.
Breaking / upgrade notes
- Clients: Kai, Kai mobile and Kai for Firefox move to this release. The update manifest gains an
nsisentry besiderpm, and thelmgw-updatecrate's artifact type changed with it. - Migrations 0101 to 0106 run on start: accounts and sessions (0101), the audit log (0102), image idle and working-set learning (0103, 0104, 0105: an image row's learned peak and budget now belong to the engine build they were learned under and are learned again after an engine change) and the outside-use history (0106).
- New settings and fields:
vram.headroom_mode(auto or fixed) besideheadroom_mb;n_cpu_moeandmoe_cache_mibon chat rows;trusted_proxies,tls.*andaccounts.audit_days. Warnings on start/restart answers and image tests; the op list hasengine_releases. - Auto headroom replaces the fixed one by default; set
headroom_modeto fixed to keep the value you had. - The server image pins its listeners and keys; on the desktop an account switch starts at off.
Fixes
- A partial image start is held to what admission gave it and learns from a failed generation; a fitted llama-server container is adopted at boot.
- A first llama.cpp start under a declared budget is no longer refused as too large before the cards are known.
- A new row's id is settled under the routing-write lock, so two Get it runs cannot pick the same one.
- A size over
max_body_mbanswers 413 on Windows, not a reset; the main TLS paths must be absolute; a download without a dialog goes to the Downloads folder. - Podman's known-harmless stderr lines are no longer quoted back as errors.
lmgw 0.6.0
A machine check that sets up the graphics card, Find a model, the OpenAI platform as its own upstream, Anthropic's blocks carried natively, image and audio tools in the Chat, and the Chat as a crate other apps can host.
- The machine check. At start lmgw checks whether this computer can run local models and says what is missing in plain words: the graphics card and its driver (Secure Boot included, amdgpu on AMD), NVIDIA's container toolkit, Podman, the model settings against the card, free space and the engine downloads. It shows as the "This computer" card first in Settings → GPU, as a card on the Overview and above the Chat's empty state, and as a banner in the Chat while local models are involved (a cloud-only setup is not nagged; "Not now" quiets all three). The tray adds Graphics card needs attention… when the verdict leaves ready.
- Set it up installs the toolkit through a fixed root helper,
/usr/libexec/lmgw/lmgw-machine-helper, behind lmgw's own confirmation (every command and file under "Show the commands") and one polkit password prompt. It runs only from the app's window and as a system task (lmgw-machine-setup.service), so closing lmgw doesn't interrupt it. Where the helper can't apply (no dnf, an image-based system) the check lists the commands to type instead. - lmgw writes the card's CDI description itself, without root, into its own runtime directory, rewrites it after a driver or package update and passes it to its containers with
--cdi-spec-dir; there is nonvidia-ctk cdi generateto run for them. - Model settings made for the other vendor's card (AMD Vulkan or NVIDIA CUDA) can be switched in one confirmed step, with every change listed.
- A start the computer can't make answers
503 gpu_setupwith the reason, or the model's fallback answers it. The check and its fix are also themachine_checkop and thelmgw__machine_checktool.
- Set it up installs the toolkit through a fixed root helper,
- Engine images arrive with the model. A model download also fetches the image its class runs in, and a start whose image is missing downloads it before admission, showing its bytes, without taking a place in the queue. A failed image download answers
engine_download. - Find a model. Models → Find a model takes a name as someone said it ("qwen 3.6") or any Hugging Face link, searches the hub live (originals before uploads, no model list of lmgw's own) and answers one card: the original, the upload and the file that fits this machine's card, judged by the listed sizes and then a Range read of the GGUF header, before anything downloads. Get it downloads the file with its companions, plans it, fits its context to the card and creates the row without starting it; Say hello opens a thread. Send to a friend copies the model's name and its public Hugging Face address, and a pasted share text plans that exact file. Add from Hugging Face marks every file by its fit and preselects the recommended one. Self-admin agents get
lmgw__model_find,lmgw__model_getandlmgw__model_get_status.- Downloads resume from their
.partafter a restart, are checked by length and sha256 before they are renamed into place, and pause on a metered connection. - A fresh install needs no settings to download: it keeps models under
~/.local/share/lmgw/models/, one folder per class, shown and changed under Settings → Runtimes.
- Downloads resume from their
- Binary units everywhere. Every byte figure lmgw shows or says (the machine check, downloads, image pulls, disk, card memory, body sizes, notes, tool sentences) is in KiB, MiB, GiB or TiB, through one formatter.
- The OpenAI platform gets its own egress. The
openaiprotocol now means api.openai.com only; every other OpenAI-shaped provider isopenai_compatible./v1/responsesgoes to the platform natively, metered and priced, unless only lmgw's loop can serve the request (an MCP label lmgw serves, aprevious_response_idlmgw stored). Hosted tools pass through on the native path. Stored responses belong to the key that made them, andGET/DELETE/input_itemson a native id go through its upstream.- Chat sends to the platform go over the Responses API unless they set
stop,seed, a penalty or carry audio. Encrypted reasoning survives across turns: in the Chat, as a thinking block'ssignatureon/v1/messagesand asencrypted_contenton/v1/responses. A model's refusal is its own part. - Service tiers. lmgw asks for
service_tier: autowhen the client set none and prices each answer at the tier it reports. Price rows have a tier; traffic at a tier with no price stays unpriced with a note, and the Prices page lists it. - Budget floors. A tier with no price, or a stream cut before its usage, still counts against budgets at a floor (the standard price, or a local token count). A budgeted caller is refused with
budget_unpricedon a model with no price for what it is billed in, flex without a flex price included. max_completion_tokensfor the output cap; llama.cpp samplers are left out and named inx-lmgw-params-ignored; OpenRouter'sreasoningobject becomesreasoning_effort; OpenAI's error codes,Retry-Afterandx-ratelimit-*reach the client.
- One fallback per model and alias. Every model and alias has one Fallback setting, used for every reason it can't answer: the GPU hold, a benchmark, VRAM taken outside lmgw, a deferred background request, a card that isn't set up, an exhausted quota, an unreachable upstream or a disabled row. A fallback is served as if the client had named it, so it can hand on to its own;
x-lmgw-fallback-chainlists each hop with its reason. A fallback must be of the same model type and may be a local model; a loop is refused. lmgw remembers an upstream that is out of quota and skips straight to its fallback until a re-check (quota_recheck_seconds, under Usage & cost) or Check now. - Anthropic, natively. Between a Messages client and an Anthropic upstream, lmgw now carries what it used to flatten, whole and streamed: thinking blocks with their signatures, blocks it doesn't model (
redacted_thinking, server tool blocks, newer types) verbatim, text block citations, compaction blocks andcontext_managementedits, and Files API sources for images and documents. Every other route gets the nearest equivalent (the compaction summary as text, citations dropped) andx-lmgw-params-ignorednames what was left out. Signed and verbatim blocks go back only to the upstream that produced them. The Anthropic dialect of/v1/modelscarries Anthropic's capability blocks. - Stalled streams end. An upstream that stays silent past its timeout ends the client's stream with the protocol's own error. Keep-alive comments only hold a stream open while the model may be thinking out of sight or a provider-run tool is running.
- Structured output, translated everywhere.
response_format,text.formatandoutput_config.formatare one field that every egress writes in its upstream's form (Anthropicoutput_config.format, GeminiresponseJsonSchema). A model whose capabilities say it can't is refused early withresponse_format_unsupported; upstream schema refusals arrive asinvalid_json_schemaorresponse_format_unsupportedwithparam. - Image and audio tools for the Chat.
image__generate,image__edit,image__upscale,audio__speakandaudio__transcribeare a built-in toolset on/mcpand, in Chat threads, the labelsimageandaudio. A running call shows its stage, step progress and previews and can be cancelled. A generated picture gets a card with Re-roll, Vary, Edit and Upscale that work without the chat model (POST /chat/api/attachments/{id}/media). Default models per tool are set in Settings → Chat, a folder or a thread. - The Chat on the Chat plane. The dashboard's Chat reads only
/chat/api/*and/v1routes, scoped to the caller (new reads for its settings, toolsets, tools and knowledge bases), and is a crate of its own,lmgw-chat-ui, with a narrow layout down to 280 px and no inline scripts, so a paired device can host the same Chat. Kai for Firefox shows it in a Firefox sidebar.- A message can carry its
origin, where the user writes from, kept out of the system prompt. - Conversations can be renamed (menu or F2;
POST /chat/api/threads/{id}/title), New chat starts in the open thread's folder, the composer is one input bubble, and side panels can be dragged wider or narrower.
- A message can carry its
- Device-to-device transfers. A paired device can offer a file to another device of the same app (an app id declared at pairing), relayed through lmgw: a resumable ranged upload checked by sha256 (
/transfers/api/transfers), the offer on the Chat feed astransfer.updated, a ranged download, and expiry. The Devices card gets a Transfers section, and enrolment carries the device's own name. - API compatibility checks. Every gate run checks each body lmgw sends upstream and each
/v1answer and stream event against OpenAI's OpenAPI spec, Anthropic's SDK schemas and Gemini's discovery document.scripts/compat-live.shdrives multi-turn scenarios through the official SDKs against a running lmgw; its results are published at https://lmgw-dev.github.io/docs/compat/. New on the way:POST /v1/responses/input_tokens, and a sampling parameter a model refuses is retried once without it, left out from then on and named inx-lmgw-params-refused. - Background requests.
x-lmgw-background: onmakes one request a guest on the GPU: it uses a local model only where that disturbs nobody, otherwise the fallback answers or a503 gpu_deferredsays to retry. The Models page shows a model's deferrals over the last 24 hours. - The Voices lab. Under Labs: rounds of candidate voice samples from any text-to-speech alias, saved to the voice library with their exact transcript (also
lmgw__voice_generate,lmgw__voice_sample_save). - Audio.
/v1/audio/alignmentsanswer...
lmgw 0.5.0
Realtime voice over WebRTC, an encrypted and mutually authenticated link for paired devices, decision models, and labs that keep their history. lmgw is MIT-licensed from this release.
- WebRTC realtime calls.
POST /v1/realtime/callsis OpenAI's WebRTC transport for/v1/realtimesessions, in OpenAI's shape: post an SDP offer (rawapplication/sdp, or multipart with asessionpart) with your key and get the answer back. Audio is Opus over UDP, encrypted with DTLS-SRTP; events travel on theoai-eventsdata channel;POST /v1/realtime/calls/{call_id}/hangupends a call. OpenAI's browser example and the Agents SDK'sOpenAIRealtimeWebRTCwork with the base URL pointed at lmgw. lmgw answers as ICE-lite with one host candidate, the address the POST arrived on, and has no STUN or TURN: the client must reach that address directly, on the LAN or through a VPN. The UDP port is the HTTP port's number unless Settings → Realtime → WebRTC calls names another; that group also lists the live calls. - The device link. Paired devices connect to a TLS listener of their own and authenticate with a certificate; bearer device keys are gone. The plain listener stays HTTP for local OpenAI clients and the dashboard.
- lmgw runs a small private CA. The pairing QR code or link carries the CA's fingerprint, a one-time token and the URLs to try (LAN, VPN). The device makes its own key, enrols the public half with the token and gets a certificate, and both sides check each other from then on.
- Each enrolment is confirmed on the Devices card: the device and the card show the same short code, and the device opens nothing until you click Confirm. The card also copies the CA certificate and rotates the CA. The CA's key is encrypted under the desktop's Secret Service.
- The listener is off until you pair a device; the pairing form offers to turn it on and, for another machine, to widen it. Its port is the HTTP port's number plus one (Settings → Network & access → Devices).
- The Chat feed is a push channel for devices.
GET /chat/api/feedtakeskeepalive_s(up tochat_feed_keepalive_max_s, default 900), so a phone can hold the connection open in the background.hellocarries the keep-alive, the device's URLs, certificate andopen_approvals; a livecaught_upends the catch-up; every message write is recorded asmessage.added,message.updatedormessage.deleted. - Device-hosted tools and the device-tools grant. A key's policy gains
device_tools, the device labels (for example["kai"]) whose hosted tools it reaches; the tool scope then filters them like any server's. An allow pattern such askai__*no longer reaches a device by itself (the upgrade grants the labels existing patterns named). A Chat thread's attached label that offers nothing says why in the error frame (tool_unreachable,tool_offline, ...). - Decision models.
POST /v1/systemonerelays the System One wire (a state and typed questions, answered with calibrated probabilities) to a local llama.cpp decision model (b11361 or later) or to an OpenAI-compatible upstream such as OpenRouter. A GGUF whose header declaresdecision.typeis a third aux kind, published asdecide/<id>; aux rows gainubatch_size,batch_size,mmproj_path,backendandthreads. OpenRouter's decision, embedding, rerank, speech and image models now appear in/v1/modelsunder their own tasks. - The lab store and the Decision lab. The Decision, Image and Audio labs keep their runs in lmgw's database. The new Decision lab asks a decision model typed questions, compares and perturbs inputs, and saves question sets with a confidence threshold per question. A set run sends a CSV or JSONL file through a set as a job and reports accuracy, a confusion matrix, reliability bins and the threshold curve; batch agents can classify through a set. The Image and Audio labs save the files their runs made (one file, or a zip with a manifest).
- Save As dialogs. Every download in the desktop app asks where to save, through the desktop portal where there is one.
- VRAM and plan accuracy. A chat or aux row's footprint charges llama.cpp's compute buffers, and the KV cache follows llama.cpp's layout (sliding windows, shared layers, no V cache under MLA, MTP and draft contexts). A plan that cannot fit the card says so, and plans load nothing.
- GGUF licence badges. The Models page shows each file's licence from its header beside the row.
- MIT licence. lmgw was FSL-1.1-MIT; it is plain MIT now.
Breaking / upgrade notes
- Paired devices must pair again. Every existing device row becomes v1: its bearer opens nothing, and the refusal says so. Pair again from the Devices card and confirm the code; v1 pairing links are refused.
key_rotaterefuses a device. - Kai needs 0.2.0, which pairs over the device link. Kai mobile 0.1.0 is released alongside.
- VRAM estimates rise for chat and aux rows (compute buffers), so admission can refuse or evict where a start used to fit.
- Removed spellings are refused, naming the valid one: the op
embed_model_set(useaux_model_set);embedas an auxtarget(useaux);upstream_setprotocolopenaiwith kindllama_server(usellama_cpp); theprobeparameter of the model plan tools. - Run arguments are lists of strings (
extra_run_args,extra_args) on every tool, op and answer; the newline or shell text form is refused. - Answers changed:
key_createanswerskey(wasplaintext);routableisroutable_candidates; unknown values are left out instead of""ornull; unknown ids are404 not_foundon the docs routes; a failed dictation answers{code, message}; the self-admin tools use the dashboard's names and types. - Migrations 0079 to 0084 run on start. Building needs
opus-devel; the RPM depends onopus.
Fixes
- Container starts no longer pass the desktop app's open files or
LD_LIBRARY_PATHtopasta, which caused SELinux denials. - A failed agent run no longer keeps the agent's token in its error text.
- Saving an alias no longer resets its min-p, repeat, presence and frequency penalties.
- Price sync leaves models billed per character, second or image unpriced instead of pricing them wrong.
- The Audio lab's voice list starts no model when the page opens.
- A voice-mode reply bubble stops spinning when its done frame is bad.
lmgw 0.4.4
lmgw's own HTTP API is documented and typed, and a developer document covers what clients, devices and agents can call.
-
/v1/openapi.jsonis the developer document. It lists every operation a non-owner credential reaches: the inference routes, the Chat API that paired devices use, the agent run API, andGET /api/version, with a map of which key kind may call what. The owner's admin plane stays in/api/openapi.json. -
Documented for the first time:
- the rest of the Chat API: thread create, read, settings, delete, pin, move, archive and keep; send, continue, regenerate and edit with their SSE frames; message delete, read-aloud, voice warm-up, speech stop, search, folder patch and delete, attachments, and the exports (including the
lmgw.chat.v1bundle); - the Knowledge bases API, the Audio lab and Image lab APIs, and
GET /chat/api/threads/rows.
Only the dashboard's pages, the
/uiredirects and the agent app and MCP proxies stay undocumented. - the rest of the Chat API: thread create, read, settings, delete, pin, move, archive and keep; send, continue, regenerate and edit with their SSE frames; message delete, read-aloud, voice warm-up, speech stop, search, folder patch and delete, attachments, and the exports (including the
-
Typed answers. Every answer, op result and SSE frame of lmgw's own API is an
lmgw-api-typestype that the handler builds, and the schema comes from that type. Fields the server always sends are listed asrequired; request bodies of shared types are<Name>Inputcomponents. audio.cpp's own JSON (/v1/tasks/*,/v1/audio/alignments) is relayed verbatim and documented as such. The/v1protocol routes still pass unknown request parameters to the upstream. -
Kept honest by tests. Every drift test validates the live answer against the built document, including every Chat turn frame and the
/v1/chat/completionsstream. A test refuses new untyped answers, and a check holds the count of inline JSON in handlers and ops to a ratchet (README, "Typed answers"). -
Behaviour changes, malformed input and error paths only:
- A JSON body of the wrong shape is
422 bad_requeston every Chat route (send'svoice, continue, regenerate and the job answer were 400). Invalid JSON stays 400. - Audio lab bodies of the wrong shape are
422 {error}. Before, a non-stringtranscriptcleared the clip's transcript and a non-stringaliasfell back to another model. GET /audio-lab/api/refs/{name}errors are JSON{error}(were plain text). The Audio lab's speech and task hand-offs answer a bad body in the OpenAI envelope. Image lab errors gainparam,typeandcodewhere they lacked them.
- A JSON body of the wrong shape is
-
Fixes: a tool turn could overflow a 2 MiB stack in a debug build; GitHub CI runs the full test suite again.
lmgw 0.4.3
Personality profiles, approvals and late results for MCP tools, and client devices that host tools of their own.
- Personality profiles. A profile bundles a system prompt, reasoning settings and a voice, and shapes a thread's turns.
- Assign one to a thread or a folder's defaults; the
chat_profilesetting names the Chat's profile for new threads. A folder with no profile inherits the Chat's; it does not mean "none". A spoken session follows its thread's profile from the next response. - Edit them at
/chat/profiles(also Settings → Chat profile), with a drawer and folder picker, header chip and voice-panel chip in the Chat. Preview, voice Test and Speak work on an unsaved draft; a built-in profile can be reset. Unsaved edits ask before another profile opens, and Discard on a new profile drops the draft. - The self-admin tools
lmgw__profiles,lmgw__profile_setandlmgw__profile_deletemanage them. A device may change a profile that steers the admin tools only while it may write with them (403 profile_in_admin_use), and a device's TTS alias is checked against its key's scope.
- Assign one to a thread or a folder's defaults; the
- Approvals for MCP tools.
require_approvalon a thread (or folder default) stops a turn at a gated call.- The thread reports
pending_approvals;POST /chat/api/threads/{id}/approvalsdecides, and the resumed reply streams on as its starter. A new message declines what waits.approval.requestedandapproval.decidedevents on the feed, the tool frame and the realtimemcp_approval_*items carry the call'scall_id. - The dashboard shows Approve and Decline on the call's card, per card or as a batch.
- A device can only tighten the rules: a write that gates fewer calls is
403 approval_loosen_refused, and it decides only within its own reach (approval_out_of_scope). The owner's approval floor per thread and folder always holds, under either spelling of a tool's name.
- The thread reports
- Late tool results (MCP Tasks). An MCP server that answers a tool call with a task no longer holds the turn.
- The result enters the thread when it arrives, as an
lmgw__job_resultpair placed in chronological order, never while a turn runs, and a send, edit or regenerate keeps it. GET /chat/api/threads/{id}/taskslists a thread's jobs,POST /chat/api/threads/{id}/tasks/{task}/cancelcancels one, andPOST /chat/api/threads/{id}/answerlets the model answer results that arrived while nobody was talking.- Bound voice sessions hear
lmgw.task.doneand the continuation is spoken. The dashboard shows a card (Job · tool · status) and a strip of the thread's jobs with Cancel; the server list counts open tasks (open_tasksonGET /api/mcp-servers). device.revokedis a stored feed event for a deleted paired device, so a host can cancel the jobs that device started. A disable says nothing.
- The result enters the thread when it arrives, as an
- Devices host MCP tools. A paired device can serve tools over
GET /mcp/host;initializetells it the link's limits (lmgw/host_limitsin_meta). Closes say why (a deleted key is 4003; of two links the newer stays). A run calls only the tools it was offered. The dashboard's reach list shows a device chip. /mcppasses resources through, namespaced by tool prefix, with MCP Apps metadata:tools/liststamps each tool with_meta["lmgw/server"], and the Chat tool frame gainsserver_label,ui_resourceandstructured_content. App-only tools stay out of model runs.- Colliding tool names no longer shadow each other. Both tools move to
<server>__<tool>(the server's own name still goes upstream; the frame saystool_moved), stable across restarts. A key's deny list holds under every name a tool has; an allow list admits only the name it names; a built-in's switch does not reach a server's tool. structuredContentstays out of the model when a result also has text content: clients and app views get it, the model reads the text.- Gemini tool loops keep their thought signatures. The signature rides in the call's id and goes back to Gemini; an unsigned step gets the skip value. Chat completions, Messages and
/v1/responsesall pass it. - Fixes.
- The Chat page follows results stored before its own send in place, without a reload.
- Test flakes were fixed (a Stop sent before a start, the VRAM sampler, realtime and agent waits);
scripts/flake-hunt.shruns the suite under load. - The self-update code moved into the
lmgw-updatecrate; the feed's auth header goes only to the feed's own origin.
Upgrading: migrations 0071 to 0078 run on start (profiles, device MCP rows, tasks, pending approvals, a message's task, the approval floor, a message role index, known tools). Tools whose names collided are now prefixed by their server, so clients that called the bare name of the shadowed tool should list tools again.
lmgw 0.4.2
Client apps as paired devices, prices beyond tokens, and a Chat that follows every client.
- Client apps as paired devices. Pair a client (a desktop voice client first, a phone later) from Usage → Keys → Devices with an
lmgw-pair:link.- Each device gets its own key, tool scope and budget. It reaches the Chat API and
/v1, and nothing of the admin plane. - Disable, Rotate and Delete end its connections at once. The realtime close says why: 4003 with a kind token for a revocation, 4004 for a thread out of reach.
- A device can be given its own level of lmgw's admin tools (off, read only or full), capped by the gateway's own setting.
- Each device gets its own key, tool scope and budget. It reaches the Chat API and
- A Chat change feed and ongoing folders.
GET /chat/api/feedstreams thread and folder changes, running turns, voice sessions and the GPU hold, and resumes across restarts.- A folder can be one ongoing conversation that every client continues (
POST /chat/api/folders/{id}/current). A new thread starts after the folder's idle time or on "New conversation". - The new
lmgw-clientcrate holds the realtime protocol, the voice state and the feed reader for clients; the dashboard's voice panel runs on it.
- A folder can be one ongoing conversation that every client continues (
- The dashboard's Chat follows other clients live. A
chatframe on/api/eventsnames what changed, and the page reads it again without leaving the tab. A draft whose conversation was deleted elsewhere waits, unsaved, until you send it. - Prices beyond tokens. A model can be priced per audio minute, per 1M characters, per generated image and per request, beside its token rates.
- Requests record the audio length, characters and images they measured. Nothing is estimated: a quantity lmgw can't measure stays unknown, never 0.
- Usage and Traffic show the quantities.
- The catalog sync brings in request fees and cache prices, and a fallback is priced as the alias that answered.
- Supertonic speaks typographic text. A reply containing a character the model's vocabulary lacks, such as „ or an emoji, no longer fails its speech. Such characters become plain equivalents or are dropped, and
x-lmgw-speechnames them. A text with nothing left to say isempty_inputon every TTS model. - No more "database is locked". Every write takes SQLite's write lock up front, so concurrent writes wait their turn instead of failing.
- MCP servers connect or fail. An HTTP/SSE server's handshake is bounded by its
timeout_ms.- Podman-isolated servers run in labelled containers, which are removed when the server stops and swept at the next start.
- Settings saves and the GPU hold no longer wait for MCP reconnects.
- Streams follow
stream_options.include_usage. Streamed chat completions end with a usage chunk only when the client asks for it, as OpenAI's do. - A clean shutdown. Quit, an update's restart and SIGTERM stop the gateway first: feeds and voice sessions close with 1001, and cut turns keep their partial reply.
- The API reference reads as documentation for client developers.
Upgrading: migrations 0061 to 0070 run on start. If an old price row used a removed unit (per_second, per_char), lmgw refuses to start and names the row. Delete it and enter the price again in the new units. Run the price sync once after upgrading, so request fees and cache prices arrive.
lmgw 0.4.1
Tools in voice conversations, llama.cpp as a backend of its own, and an app window that feels native.
- Tools in voice conversations. A
/v1/realtimesession can offer MCP tools the way OpenAI's Realtime API does ({"type": "mcp"}): lmgw runs the calls on its own MCP servers, by label, and reports them asmcp_callitems; the client sends the follow-up.GET /v1/mcp/serversand/v1/mcp/servers/{label}list the servers and their tools for a client's picker./v1/responsesnow takesallowed_toolsas{tool_names: [...]}too and refuses a malformed one instead of allowing every tool. - llama.cpp gets its own backend. llama-server rows speak their own protocol (existing rows migrate on start): what a model can do comes from the server's
/props, and images returned by tools reach a model that sees them. ik_llama.cpp rows use the same path. - Configured fallbacks always apply. A fallback that cannot see gets placeholders for the images (a PDF's pages go as its text) instead of the request waiting for the local model, and the response header
x-lmgw-images-omittedcounts them. A voice turn goes as audio to whichever model answers if it hears audio, cloud and LAN models included (the setting is now off/on, a storedlocalreads as on); otherwise it goes as the transcript. Voice-clip transcription follows the configured fallback. The Traffic page marks every request whose content was reduced this way. - No orphaned model containers. A client that hangs up while its model loads no longer leaves the container running unregistered, which used to block that model with
vram_queue_timeout. After start, lmgw adopts or removes the containers it owns, and a request its client gave up on is logged as a 499 row. - A native app window on KDE. KWin draws the frame, the app uses the desktop's fonts and the title bar's colours, the traffic pulse and the GPU status sit at the foot of the sidebar, right-click gives a desktop menu instead of a browser's, the window opens fully drawn, and the scrollbars are thin. The default interface scale is now 125%.
- Browser clients with a gateway key. CORS preflights now echo the requested headers, so a browser can send
Authorization.
Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm, or let the app offer the update.
lmgw 0.4.0
Voice: talk to your models, in the app and over the API.
- Realtime voice API.
/v1/realtimespeaks OpenAI's Realtime protocol over a WebSocket and answers with your own models: turn detection inside lmgw (Silero VAD, and Smart Turn forsemantic_vad), speech to text, a chat model with tools, and text to speech. Interrupting cuts the answer, and the history keeps only what was heard. - Voice in the Chat. Dictation into the composer, read-aloud of replies (code blocks and tables are announced, not read), and a voice mode that talks with the open thread. Choose the microphone and the output in the app. Each thread can have its own speech models, voice, style and languages: the language you speak and the one the reply is spoken in can differ. A local model that hears audio can get your voice turn as audio (a setting, off by default).
- Natural speech output. Replies go to text to speech in whole sentences, several to a request, and the silence an engine pads its audio with is capped by a new setting, Settings → Realtime → Longest pause.
- Audio models. Catalog downloads follow the revision a model's spec pins, a package can be completed in one go, and an audio model can run on the CPU with its own thread count, where it keeps serving while the GPU hold is on.
- Many fixes around audio containers, the GPU hold and the catalog.
Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm, or let the app offer the update. The README has the voice setup.
lmgw 0.3.0
First public release.
lmgw, short for local model gateway, is a Linux desktop app. It runs models on your own GPU and serves them through local copies of the OpenAI and Anthropic APIs. Cloud providers sit behind the same aliases, picked by you or used as a fallback.
Download the RPM below and install it with sudo dnf install ./lmgw-*.x86_64.rpm. Requirements and a quick start are in the README. From now on the app checks this page itself and offers each new release.