v0.9.3

ignis v0.9.3

Vision no longer costs context

A vision load now takes a vision residency: --vision-residency host|device, vision.residency in the config (#413, ADR 0075, guide).

• host, the default. The 27B's vision tower (296 MB) stays in pinned host RAM, and the image embedding pool is host-mapped.
  • The tower: for each image it is staged into the prefill scratch a block at a time, through three 25 MB slots, while the block before it computes.
  • The pool: the encode writes each page through the GPU, and the prefill reads the image's columns in place.
  • Between images, neither takes VRAM.
• device: today's load, with the tower and the pool in VRAM. It is the choice for a card with VRAM to spare on a host short of RAM.
• Profiles:
  • rtx5090 takes host;
  • rtxpro6000 takes device;
  • rtx50-16gb now turns vision on with a 4,096-token image envelope, under host.

Measured on an RTX 5090 (finding):

• KV pool on a 16 GB budget under rtx50-16gb: with vision on, it equals the text-only pool, 7,956 pages at both the 1,024- and the 4,096-token envelope. Under device it is 7,437 and 7,384 pages.
• Encode: a 1,024-token encode costs about 7 ms more (49 ms against 42). A fan-out of questions over one image encodes it once.
• Answers: /v1/decide answers are identical at the same context, and the image's embedding is the same to the bit.
• Decode: unchanged, at 99.7-100.3% of 0.9.2.

The staging rebases each stage's weights onto its slot, following the visit_weights of #407 by ampersandru.

Other changes

• kernel/build.ps1 builds in a shell that already has an MSVC developer environment: it imports vcvars only when cl.exe is not on PATH, so a toolchain the Visual Studio installer does not list builds too. By ampersandru, from #407.

Upgrade notes

• More host RAM under host. The host plan pins the tower and the pool beside the token embedding. Under the make defaults on a 32 GB card, a start needs about 18.6 GB, where it needed about 18 (host RAM).
• A host start that cannot pin them is refused before anything loads, naming both ways out: --vision-residency device (vision.residency: device), or vision off. It never falls back to the device on its own.
• rtx50-16gb turns vision on. For the text-only load of 0.9.2, pass --vision-enabled false.
• The context and the rope can move. On a card whose start adapts its context to free VRAM, the KV that host gives back can raise that context, and with it the YaRN factor that the default auto rope scaling picks. Answers then move slightly, as any change of free VRAM does. Pin --model-max-context to compare against 0.9.2.
• Task Manager on Windows counts the pinned tower and pool as shared GPU memory, as it does the token embedding. That is host RAM, not VRAM oversubscription.

Decisions

• ADR 0075: the vision tower is staged and the embedding pool host-mapped, unless the load asks for the device
• ADR 0035: the vision embedding outlives its encode (amended)
• ADR 0063: the profile follows the card class, and its defaults adapt to the budget (amended)

Container image (linux/amd64)

podman run --rm --device nvidia.com/gpu=all -p 8000:8000 \
  -v /path/to/models:/models:ro \
  -e IGNIS_MODEL_ARTIFACT=/models/<name>.ninfer \
  ghcr.io/gpillon/ignis:0.9.3

(docker: --gpus all in place of --device.) The image carries
the CUDA runtime but no driver — the host's NVIDIA driver is
injected by the container runtime. It is built from the same
compile as the Linux tarball below, so the two always carry the
same binaries.

Binaries

The kernel is compiled for SM120a (RTX 5090 / Blackwell
consumer) only, and the engine needs a .ninfer artifact, which
is not part of this release. The Linux tarball needs the CUDA 13
runtime on the host; the Windows zip carries cudart64_*.dll.
Both need an NVIDIA driver new enough for CUDA 13.

Apache-2.0 (LICENSE, NOTICE); the vendored reference ops keep
their own terms (NOTICE-kernel).

Full Changelog: v0.9.2...v0.9.3