v250

Lots contributors to thank on this release:

• @einhander for greatly improving the cmd/vllm-wrapper and @depohmel for the initial contribution.
• @chaserhkj for adding /v1/task/run endpoints for audio.cpp
• @cphlipot for improving llama-server context compatibility and adding the /models endpoint
• @noctrex for the cool new capabilities labels in the model UI (see screenshot below)

This release also updates the unifed docker container to come with:

• audio.cpp
• llama-bench
• vllm-wrapper

Enjoy!

PS: check https://github.com/mostlygeek/retrocraft - Qwen 3.8 27B demo

Changelog

• 60226b6 docker/unified: add llama-bench, vllm-wrapper, audio.cpp (#1011)
• 44a17d7 README.md: reorder list of features
• 3bccbef Support argv-based vLLM startup in vllm-wrapper (#979)
• be683dc ui-svelte,internal/server: show capability tags on Models page (#1007)
• 2edfa73 api: support /v1/task/run for audio.cpp (#982)
• be50598 AGENTS.md: tweak rules around pull requests
• 3e583ad CONTRIBUTING.md: update rules
• 038ae45 expose meta.n_ctx on models endpoint (#984)
• 6333ee3 router: add /models endpoint (#983)

v249

If you use ComfyUI with llama-swap this release is for you. A new /comfyui endpoint was added to make comfy and llama-swap more compatible. Now an idle comfyUI doesn’t block swapping or background HTTP requests cause comfy to load.

To use it create a model named comfyui_auto and llama-swap will take care of the rest. See issue #1001 for a comfyui and docker example to get started quickly.

Changelog

• f94c94a various: improvements to code layout and naming (#1003)
• 249797f internal/server: add ComfyUI compatibility endpoint (#1002)

v248

Changelog

• 00515cd internal/server,shared: preserve percent encoded in upstream (#988)

v247

Changelog

• 40027d6 ui-svelte: security update
• 298848d internal/hw: detect inference host hardware (#978)

v246

Changelog

• 22df230 ui-svelte: show configured selectors and strategies on Models page (#975)
• 8d61908 internal/perf: fix rocm-smi GPU memory utilization (#973)
• 1f3c68e docker: install rocm-smi for vulkan backend (#968)

v245

Changelog

• 30470a4 ui-svelte: show profiles on model page (#966)
• 27782ab AGENTS.md,CONTRIBUTING.md: update contribution guidelines (#964)
• b9c3738 docker: add -watch-config (#963)

v244

Changelog

• 0c42333 internal/matrix: solve matrix expressions symbolically (#960)

v243

Spicy maintainer notes:

v243 is a quality of life release.

1. cmd/vllm-wrapper helps vLLM users avoid the long load times by communicating with an already running vLLM to load and sleep a model via its API. It hijacks the model.cmd and model.cmdStop parameters to send the commands. Neat. Thanks @depohmel for the contribution!
2. Matrix sets can finally use full model IDs and using var is now optional. Using var was annoying for users who ultimately ran face first into a "WTF is this naming limitation" moment. Even this frozen heart is melted by enough user tears. I'll work on lowering that threshold (maybe).
3. #955 makes macros work where they are expected to (hopefully) in the configuration. The new approach does a multi-pass expansion over the yaml source. When there are no more macros to expand the configuration is unmarshalled into the typed configuration structure. Hopefully this is macro's final evolved form. I consider this pokémon fully evolved.

Changelog

• 4c50064 internal/config: relax matrix model reference constraints (#957)
• 4bffcfa cmd/vllm-wrapper: add helper for sleep/wake (#941)
• fa607fe internal/config: refactor macro expansion (#955)

v242

A bunch of changes today:

• removed the 25MB filesize limit for audio files. This was pulled over by the agents (claude!!!!) from the backend file limit which was fixed a little while ago. Now, upload big files and make your local machines cry!
• fixed a bug found by @teru358 in #946 where TTL and model request could cause a deadlock. Opus 5 released just in time. It burned all the credits in my 5hr limit but the fix was pretty nice after a bit of back and forth.
• introduce fully qualified peer model names. Remove the constraint that local and peer model Ids have to be unique.
• with FQN for peer models the spillover selector can spill over to a remote model. Fun if you have multiple llama-swap rigs and want to tag team in the other one for more parallelism! Qwen 3.6 27B Q8 at 25 tok/sec on the Framework Desktop is still faster than waiting at 0 tokens per second on the busy 3090s.

Changelog

• 7aa7f52 internal/server,ui-svelte: add peer model namespaces (#950)
• 5e9f887 Fix process start race during concurrent stop operations (#949)
• 12af351 ui-svelte: remove audio file size limit (#948)

v241

Changelog

• 8b61e3d internal/server: implement selectors (#942)
• c6adf57 internal/server,shared: reduce request model body functions to one (#940)
• 3e87c02 internal/server: add support for profiles (#935)
• 378a26e ui-svelte: add support for different v1/audio/voices response formats (#932)
• fc8b29f docker: add FFmpeg support for whisper.cpp (#785)