Repository navigation
0.4.1
Changes since the last Release
Added
servebindsPOST /v1/chat/completions/input_tokensand/v1/responses/input_tokens, the OpenAI-shape counterparts of/v1/messages/count_tokens.
Changed
-
Update llama.cpp to b11146 (v0.5.0) (from v0.4.1). Upstream's HTTP server now listens on a list of addresses.
--hoststill takes one. -
release.ymlandrelease-gpu.ymlaccept a taglessworkflow_dispatch. An emptytagbuilds the dispatched branch aschimera-dev-<sha7>-*artifacts and skipspublish, so the workflows can be tested before tagging.
Fixed
-
The embedded web UI's Stop button now stops generation. The UI tags each chat stream with
X-Conversation-Id, which makes upstream keep the generation running after client disconnect and cancel it only throughDELETE /v1/stream.servedid not bind that route, so Stop got a 404 and generation ran tomax_tokens.servenow bindsGET/DELETE /v1/streamandPOST /v1/streams/lookup. Tagged streams still survive disconnect by design, so a reloaded tab can reattach. -
SIGTERM no longer hangs
servewhile a tagged stream is running. Shutdown finalized stream sessions without cancelling them. The request's HTTP worker then waited for results from the stopped task loop, and exit required a second signal.scripts/patches/llama.cpp-stream-cancel-on-shutdown.patchcancels them first; seescripts/patches/README.md.