Skip to content

v0.9.5

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Nov 03:15
· 558 commits to main since this release
v0.9.5
2a20152

ignis v0.9.5

Transcribe: running notes and a recap

  • Notes in two forms (#428, ADR 0076, guide): running notes, kept during the call by an append-only notes thread that writes each kind's live sections, and a recap, written over the whole call once it ends. ADR 0076 records why the running notes send the notes back to the text model and the recap does not, measured on three public conversations.
  • Corrections, renames and your own edits reach the notes: an update marks what it changed, and the notes can be edited by hand.
  • Context for every text-model task: the notes, the recap and Ask are told the transcript's context.
  • Context images: add a picture of what the call is looking at by file, paste or Capture, crop it and black out what should not be seen; the text model reads it once, with vision, and keeps it in context.
  • Exports: a transcript exports to a PDF, and the .md export carries the notes and Ask's answers.
  • Speech: the guard against a transcript that copies its context can also take near copies, a line a word or two off, as no speech: speech.context_copy_edits (0 to 3, 0 by default, an operator's opt-in), since on real speech every value above 0 also emptied a line someone actually said (#442, speech to text).

The Server view

  • A status bar over two tabs (#454, guide): the bar says what each engine runs (the main model, the speech model, the diarizer) and how the last switch ended; Models groups the catalog by the engine a model is for, under a line of what the listed models take on disk and the room left under download.path; Configuration holds the form. A draft, a search and a scope stay when you change tab.
  • Find a configuration field by name, key, flag, variable or description, in a search beside the scope (#450).
  • A model family's scope shows only what that family can set for itself, and the form opens on All models with the running family marked (#452).
  • The model list says each model's kind in a Type column, and its columns line up (#448).
  • GET /v1/models/catalog gives every row its kind, before the model is downloaded too (#448), and carries disk: {path, free_bytes}, the download path and the room left on its volume (#453, model switching).

The engine

  • An image conversation resumes instead of re-prefilling the picture on the 27B: when an image spans the generation opener's page floor, the capture links the opener's page and captures the checkpoint over it, so an identical repeat and the next turn resume from it. A 16,404-token request on the RTX 5090: repeat 1,621 → 171 ms, next turn 1,631 → 233 ms, with outputs equal to a cold prefill split at the same boundaries (#249, ADR 0029 amended).
  • Admission no longer deadlocks behind a checkpoint claim: a waiting request gives up the claim it holds when that is what keeps a higher-ranked entry out, and cached_tokens counts only what a request actually resumed (#445, ADR 0045 amended).
  • A request that published its prefix or lent its pages can be moved to make room: the pages only reclaimable retained state holds besides it now count as freed (#444, ADR 0045 amended).
  • Prefix misses are counted per tier (device and KV-RAM) beside the checkpoint misses, and the Monitor shows the count where it read "not measured" (#222, ADR 0017 amended).
  • A request's media.* ride the line it ends with, ignis.request.done or ignis.request.cancelled, so a decision answered by its prefill and a question past its deadline report what their image cost (#245).

Other changes

  • ignis-bench: g3 decides its gate against ignis (each engine's lanes get the EOS control it honours) and ttft, g5 and small-card prove their prompts distinct across cells, so prompt reuse serves no sample warm (#426, #443).
  • Flaky tests fixed: the MoE slot-trap and nvml_memory_gpu GPU tests (#424), span_logits_gpu (#425) and the Playground's mock metrics test (#455); the GPU profile runs with --no-fail-fast.
  • Findings: a cold 396,032-token prefill on the 5090 takes 98.7 s (4,012 tok/s) (#332); NVML counts a fresh allocation late under WDDM (#424).
  • The README's engine comparison uses better-sourced figures (#446).

Decisions

  • ADR 0076: the running notes send the notes back to the text model and the recap does not
  • ADR 0029: cross-request state reuse (amended)
  • ADR 0045: the KV pool and evicted state (amended)
  • ADR 0017: opt-in Prometheus metrics (amended)

Container image (linux/amd64)

podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.9.5

(docker: --gpus all in place of --device.) The image carries
the CUDA runtime but no driver — the host's NVIDIA driver is
injected by the container runtime. It is built from the same
compile as the Linux tarball below, so the two always carry the
same binaries.

Binaries

The kernel is compiled for SM120a (RTX 5090 / Blackwell
consumer) only, and the engine needs a .ninfer artifact, which
is not part of this release. The Linux tarball needs the CUDA 13
runtime on the host; the Windows zip carries cudart64_*.dll.
Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep
their own terms (NOTICE-kernel).

What's Changed

  • Bump rustls from 0.23.45 to 0.23.46 in the cargo group across 1 directory by @dependabot[bot] in #400

Full Changelog: v0.9.4...v0.9.5

Contributors

dependabot
Assets 6
Loading