Skip to content

v0.6.1

Choose a tag to compare

View all tags
@github-actions github-actions released this 10 Oct 19:37
· 1056 commits to main since this release
v0.6.1
24074df

ignis v0.6.1

Fixes

  • A turn no longer ends inside its reasoning. An end-of-turn token drawn while a <think> block is open becomes </think>, so the model goes on to the answer or the tool call instead of stopping with reasoning only. This applies to the 27B and Flash-Next, in prefill, plain and verify rounds. The all-reasoning WARN now names how the turn ended, not the token budget alone (#315; ADR 0048)
  • Chat completions accept "content": null on an assistant message, the shape OpenAI clients send after a tool call. On any other role it is a named 400 (#312)
  • The Playground never switches the model. Its requests name no model, so the server serves the one loaded, and the header and the context limit follow it, read again from GET /v1/models when a turn starts. Before this fix, an open Playground tab and another client asking for a different model reloaded the server on alternate requests (#305)
  • make SPEC=off turns speculation off. SPEC= leaves the choice to the server, and the default rtx5090 profile speculates with dflash2 on the 27B. make help, make config and the docs used to say "off" for it.
  • A queued socket response is admitted when a running one is cancelled. The engine signalled the free slot before the step that released it, so the head of the queue could try too early, find the engine full, and wait forever (#282)
  • A request whose client hangs up while it waits for admission is cancelled. A client gone between the submit and its reply (a /v1/decide fan-out cut short, a queued socket response cancelled as it is admitted) left the admitted request running for nobody

Notes

  • opencode with two ignis models: model and small_model naming different ignis ids reload the server back and forth, since opencode sends title and summary requests to small_model. Point both at the same id, or start with --switch-allow-implicit false.
  • The Flash-Next MTP head (opt-in, --spec-backend mtp) keeps one draft entry made from the stop id after a redirect. This affects drafting only; the target's tokens are exact. The served A/B shows no throughput change (+0.08%).
  • Tests: the Responses-socket ordering tests are deterministic (they flaked the v0.6.0 release run), and a kernel test that still counted backend 3 as unknown (it is MTP since #307) is fixed. The two engine races above were older than v0.6.0 and showed up as Linux CI failures; on a 4-core Linux box they now pass 200 runs out of 200.

Decisions

  • ADR 0048: an end of turn drawn inside an open reasoning block closes the block instead

Container image (linux/amd64)

podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.6.1

(docker: --gpus all in place of --device.) The image carries
the CUDA runtime but no driver — the host's NVIDIA driver is
injected by the container runtime. It is built from the same
compile as the Linux tarball below, so the two always carry the
same binaries.

Binaries

The kernel is compiled for SM120a (RTX 5090 / Blackwell
consumer) only, and the engine needs a .ninfer artifact, which
is not part of this release. The Linux tarball needs the CUDA 13
runtime on the host; the Windows zip carries cudart64_*.dll.
Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep
their own terms (NOTICE-kernel).

Full Changelog: v0.6.0...v0.6.1

Assets 6
Loading