Skip to content

v0.7.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 16 Oct 21:08
· 728 commits to main since this release
v0.7.0
003c08c

ignis v0.7.0

Changes

  • An image model. Qwen-Image-2.1-Turbo is served as ignis-image-2.1-turbo: POST /v1/images/generations and POST /v1/images/edits (1 to 10 condition images, multipart), the checkpoint's presets and 1024x1024, transparent backgrounds, progress as server-sent events, and prompt enhancement by a text model with Qwen's own system prompts. Its own .ninfer artifact on Hugging Face (ignis-server model download ignis-image-2.1-turbo), its own leaf on FP8 GEMM and 8-bit attention. It is one model at a time with the text models: a request naming it switches the server, and the Playground's Generate and Edit do the same from a text model. On the 5090 a 2048x2048 generation takes 7.87 s end to end, a two-image edit 9.48 s, the 27B to the image model 10.0 s and back 11.3 s. Its VRAM plan is on /metrics and in the Monitor (#318-#322, #324; ADR 0050, 0051)
  • Reasoning-loop guard, on by default. On chat completions and /v1/responses (HTTP and WebSocket), a loop in a request's reasoning is taken back once it persists, the model is nudged in its own words, and the generation goes on in the same response; after two nudges the next loop closes the reasoning. The reasoning shows a tag where the loop was, usage.completion_tokens_details.rewound_tokens (output_tokens_details on Responses) counts the tokens taken back, and a reasoning sent back with its tags is repaired to what the model saw. include: ["reasoning.encrypted_content"] on /v1/responses gives each reasoning item that view. Every knob is a config field, patchable live: --model-loop-guard off|log|rewind, the detector's ratio, window, persistence and the two nudges (ADR 0052, 0053)
  • Playground: an image studio. The Image view keeps image sessions of steps, each picture edited from the last, marked (drawn or as a mask, "change only the marked area"), transparent, stored in the browser. The chat gets a make_image tool that switches to the image model and back, and a picker for the next message's model. Seeds are u64 end to end, and an edit is not sent the seed behind its inputs unless "Same seed between edits" is on (#326, still open: its live checks on the GPU are pending)
  • A restore through a retained prefix is reported: the done line's reuse_source, reused_prompt_tokens and reuse_kind, and the prefix hits counted on the tier the state came from (#325)
  • Metrics are on by default in the server itself, on 127.0.0.1:9464, keyless and never exposed. A second server that finds the default port taken warns and serves without the listener (#90)
  • Flash-Next's host plan: the margin it leaves of the available RAM is --model-host-margin-bytes, 4 GiB by default (it was a 6 GiB constant), and make gives Flash-Next its own 2G KV-RAM arena (FN_KV_HOST_POOL_BYTES), so a switch from the 27B no longer meets the 27B's 8G

Fixes

  • A server that exits joins the model thread instead of detaching it, and waits for a switch or an unload under way
  • A warm-up whose request the engine ended is no longer taken for a warm-up

Upgrade notes

  • The loop guard rewinds by default. A client may see [LOOP DETECTED, REWINDING n TOKENS / m CHARS · h] in the reasoning, with the characters before it taken back. --model-loop-guard off (make LOOP_GUARD=off) is the previous behaviour, and what a benchmark should run.
  • Metrics are on without asking. --server-metrics false (make METRICS=0) turns them off; make METRICS=1 now passes nothing.
  • The image model's weights are under the Qwen Research License: non-commercial use only. Do not serve it in a commercial product.

Notes

  • Tests: the loop guard's stop-sequence test gives the looping request a budget the mock takes seconds to spend, so the rewind's cancel always reaches it first; at 4,000 tokens a loaded Linux runner finished it before the cancel, and the first v0.7.0 release run failed on it.

Work in progress (issues still open)

  • Playground image studio: the live checks of make_image, the switch back from the image model, and the canvas paths (#326)

Decisions

  • ADR 0050 — ignis serves an image model, one model at a time
  • ADR 0051 — kernel code is ours, adapted from a named source, or vendored
  • ADR 0052 — a reasoning loop is taken back, the model is nudged, and the client is told
  • ADR 0053 — a loop is acted on once it persists, and a rewind keeps something
  • Amended: ADR 0017, 0026, 0030, 0032, 0043, 0048

Container image (linux/amd64)

podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.7.0

(docker: --gpus all in place of --device.) The image carries
the CUDA runtime but no driver — the host's NVIDIA driver is
injected by the container runtime. It is built from the same
compile as the Linux tarball below, so the two always carry the
same binaries.

Binaries

The kernel is compiled for SM120a (RTX 5090 / Blackwell
consumer) only, and the engine needs a .ninfer artifact, which
is not part of this release. The Linux tarball needs the CUDA 13
runtime on the host; the Windows zip carries cudart64_*.dll.
Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep
their own terms (NOTICE-kernel).

Full Changelog: v0.6.2...v0.7.0

Assets 6
Loading