Repository navigation
LLM Router 2026.11.3
This release adds router-wide SYCL keep-alive to keep models loaded on headless Intel GPUs, hourly per-API-key usage tracking with reports, and a fix so the playground preset picker defaults to the model that's actually loaded.
SYCL keep-alive for headless Intel GPUs
Headless Intel GPUs (no monitor attached) can have the driver drop VRAM, silently evicting loaded models. The router can now supervise a keep-alive process that prevents this.
- New bundled sycl-keepalive tool (source + prebuilt exe) shipped with the app.
- New SYCL Keep-alive settings (Off / Headless / Always) with a keep-alive section on the Devices page: mode, oneAPI setup command, executable path override, and save/stop/start/re-check actions. Disabling an enabled SYCL GPU releases the keep-alive on it first.
- Headless detection walks the device tree per GPU (works in session 0) to find attached monitors and PCI addresses, so the keep-alive targets only headless cards.
- The decision logic lives in a pure, unit-tested
SyclKeepAlivePlanner; a hosted service starts/stops/rechecks the process each minute and exposes status.
Shared ownership across router installs
- Keep-alive ownership is recorded machine-wide under
%ProgramData%\LLM Router(PID, start time, config, install path), so the next router to run — e.g. adotnet runbuild after the service — takes the keep-alive over to keep, retarget, or stop it instead of starting a second one. - A keep-alive running as another user (e.g. SYSTEM vs a dev build) is treated as running externally rather than failing to stop and doubling up; the page surfaces a "run elevated to take it over" state when takeover can't inspect the process.
- The keep-alive intentionally outlives the router, so it now runs from a staged copy in
%ProgramData%\LLM Router\sycl-keepalive\<hash>\— named by content hash so a new build gets a new path and the old process is replaced, and stale copies are pruned on start. This stops the executable from locking the install folder during rebuilds/updates. Keep-alives started from an older install-folder copy are still recognized and taken over.
Hourly per-API-key usage tracking & reports
- New
ApiKeyUsageHourlyrollup: prompt and completion token counts recorded per API key, model, and hour as requests flow through the router. - Model attribution is normalized across all API gateways (embeddings, images, passthrough, Ollama, System One) so rollups use the resolved preset name.
- New Usage view on the API keys page plus an
/api/stats/usage-reportendpoint, linked from the Dashboard.
Playground: preset picker defaults to the loaded model
- A remembered or previously selected preset could point at a model that's no longer running, making the first send silently swap models. The playground pages now report which presets have running servers, and the picker prefers a loaded preset over the remembered one — unless an explicit
?presetId=was given.
What's Changed
- Add router-wide SYCL keep-alive for headless Intel GPUs by @sockless-coding in #16
- feat: hourly per-API-key usage tracking and reports by @sockless-coding in #17
Full Changelog: 2026.11.2...2026.11.3