v0.20.0: Qwen3-VL vision, /v1/rerank, GBNF grammars, first-party NVFP4 quantizer

Highlights

imp reads images: Qwen3-VL runs end to end, from imp-cli --image and from /v1/chat/completions (image_url parts, data-URI or fetched). Dynamic resolution means no fixed image size, so a 1795x2397 photo becomes 972 image tokens; DeepStack taps are injected at the LM's first layers and M-RoPE runs three real axes. Several images in one request are encoded in prompt order into one concatenated embedding. Text-only models and text-only prompts are bit-identical to before.

POST /v1/rerank in the Cohere/Jina/vLLM shape, scoring query and document jointly in one forward rather than recomposing them from two embeddings. Validated against llama.cpp serving the same GGUF: top-1 agreement on 3/3 queries, median per-document score delta 0.0014.

GBNF grammar-constrained decoding: response_format: {"type":"grammar"}, llama.cpp's grammar and vLLM's guided_grammar. It is a nondeterministic pushdown simulator, so recursive and bracket-balanced formats are enforceable and not just regular ones. A grammar it cannot honour (left recursion, undefined rules, a missing root) is refused at compile time rather than mis-enforced. Regex constraining ships alongside it as response_format: {"type":"regex"} / guided_regex.

imp-quantize with AWQ-class activation calibration, still EXPERIMENTAL. A dense BF16/FP16 SafeTensors checkpoint with no published export can reach the NVFP4 path at all. Perplexity over ppl_corpus_45k.txt: Qwen3-0.6B 30.10 to 28.48 against a BF16 24.06, Qwen3-1.7B 20.43 to 19.21. Against a bit-identical Modelopt export of Qwen3-14B on the same corpus and engine: Modelopt 10.0301, imp-quantize without calibration 9.9252.

One server serves more than one model (server.model_swap, default on): in-flight generations drain first and are never cancelled, and a failed load restores the previous model rather than leaving the server empty.

Also in here

• A single-page web UI at GET /, embedded in the binary, with per-token latency bars and a separate thinking channel
• usage.prompt_tokens_details.evicted_tokens tells the caller when StreamingLLM dropped context, in all three dialects. Absent unless eviction fired, so its presence is the signal
• External agent gates: aider, Claude Code and the OpenAI Agents SDK each drive imp over a different dialect and must land a real edit in a throwaway repo
• Several silent failures fixed: an image spanning a prefill chunk boundary took the wrong half of itself; the persisted prefix cache dropped the KV scales; a MoE checkpoint whose experts imp could not read loaded and generated garbage; the prefix cache could serve one request's image to another; decode paid for context capacity it never used (up to -38 % on the served path); top_k above 128 sampled from the previous step's candidates; streaming leaked the chain of thought as the answer whenever tools were present; streamed non-ASCII text was corrupted
• Removed: speculative.recycle_loop (verify-in-loop) and its ~1.5k LOC, after a nine-class sweep found no prompt class where the loop beat the same configuration with it off

Gate

make verify-fast on the tagged tree, Qwen3-8B-Q8_0 on one RTX 5090: decode 288.96 tok/s against the pinned 287.19 baseline, peak VRAM 20718 against 20716 MiB.

No breaking changes. Full detail: CHANGELOG · 131 PRs since v0.19.2.