2026.10.0

Release 2026.10.0

New Features

Strata engine support

• Added first-class support for Strata, a MoE-offloading engine with a Python HTTP frontend, alongside llama.cpp
• Generalized the engine build system so installs are tracked per-engine (EngineBuild/EngineBuilds with a ServerEngine column); servers now bind to same-engine installs
• New engine discovery abstractions (IEngineDescriptor, IEngineCatalog, IEngineInstallHandler) with an EngineCatalog that resolves registered engines
• Strata install/update jobs: side-by-side release installs (clone tag, .venv, unattended setup), in-place updates (git pull --ff-only + setup.py --update), engine rollback, and stop-tracking without deleting files
• Split LlamaCppProvider into a shared ManagedServerProviderBase plus engine-specific providers; the new StrataProvider runs Strata's server.py on the router-assigned port, with /props and /slots mirroring llama.cpp so slot gating and KV usage work unchanged
• On first preset start, the model is prepared with Strata's setup.py in existing-GGUF mode (build pack, download MTP draft, write a per-PC run config), with progress streamed to the server status
• Per-engine launch options are stored on ModelPreset via a new EngineSettings JSON column (keyed <engine>.<setting>), so engines can add their own options without a column per option
• Restructured the Engines page into per-engine tabs; server create/edit pages now offer engine selection and engine-appropriate install folder fields
• Preset form gains a Strata section: context length, GPU, images, KV cache type, low-RAM mode, and parallel requests (blank = Strata's recommendation)

Split GGUF support

• New SplitGguf helper parses -NNNNN-of-MMMMM filenames; the model library lists a split model once (first shard, total size) and delete removes every shard
• ModelDownloadService downloads all shards in sequence, pinned to the resolved revision, with combined progress
• The Hugging Face picker shows split models as a single entry with a parts count

Routing: model aliases & fallback presets

• Model aliases with * wildcards (e.g. gpt-4o, claude-*) so clients with hardcoded model ids resolve to a preset (new ModelAliasMatcher)
• Per-preset fallback chain: serves when the primary's server is errored, and takes overflow when the primary is at capacity and the fallback is already loaded; request stats/logs are retargeted to the preset that actually served

Memory management

• Servers sharing a GPU form a memory group with a VRAM budget; before a model loads, least-recently-used idle servers in the group are stopped until the estimated footprint fits (MemoryGroupScheduler + PresetMemoryEstimator)
• Per-server idle unload after N minutes (IdleUnloadService)
• Preset "keep loaded" flag exempts a server from automatic unloading

Embeddings

• OpenAI POST /v1/embeddings and Ollama POST /api/embed / POST /api/embeddings, forwarded verbatim via a new provider SendRawRequestAsync + RouteRequest.BackendEndpoint

Playground

• New dashboard feature for chatting with any preset and comparing runs under different launch settings
• Chat page with per-reply timings (TTFT, prompt/generation tok/s, cached tokens, speculative-decoding draft acceptance), run labels, Repeat xN, and CSV export
• Run history page: runs persist to a new PlaygroundRuns table with a snapshot of the preset name and launch settings (hash of the generated command line), grouped by preset/label/settings; unaffected by request-log retention and soft-deleted when the preset is removed
• Parse llama.cpp draft_n / draft_n_accepted into LlamaCppTimings and backfill them in OpenAiHandler when only scalar response values exist

Servers

• Added a Delete button to the server list with a confirmation dialog and CSRF-protected POST; the new OnPostDeleteAsync 404s unknown servers and 409s when a server is mid-transition (starting/stopping/reconnecting), and drops the server's memory group once no other instance references it

Improvements

• Service install script — install-service.ps1 now defaults -PublishDir to $PSScriptRoot instead of a hardcoded bin/Release/.../publish path, so the script locates the executable relative to itself
• Release asset verification — GitHub release assets now carry a Digest field (SHA-256) used to verify downloaded engine assets against GitHub's published digest
• Engine variant selection — release installs now pick an engine build (CUDA, CUDA 12, or AMD/HIP) and download the matching verified asset

Fixes

• Engine build commit capture — when a source build ran under an env setup command (e.g. oneAPI setvars), the command's banner was recorded as the build's commit SHA. CaptureGitAsync now returns only the last non-empty output line, and a reconcile pass sanitizes stored CommitSha values (extracting the trailing 40-char hex SHA, or clearing the value if none is found) so stale rows self-heal

What's Changed

• Add Strata engine support and generalize engine build management by @sockless-coding in #2
• Add Playground: preset chat with per-reply stats and benchmark run history by @sockless-coding in #3
• Add model aliases & fallback presets, memory groups & idle unload, and embeddings by @sockless-coding in #4
• Add delete server action to servers page by @sockless-coding in #5

New Contributors

• @sockless-coding made their first contribution in #2

Full Changelog: 2026.9.3...2026.10.0