v0.1.8

0.1.8 (2026-08-11)

Features

• cli: allow --packed-slots with --total > 1 (#122 fixed) (1f55d68)

Bug Fixes

• cli: reject --packed-prefix with --total > 1 (e93a394)
• cli: use generate_async in the stdin loop (c464538)
• deps: upgrade lru to 0.18 (RUSTSEC-2026-0253) (740c88c)
• deps: upgrade lru to 0.18 (RUSTSEC-2026-0253) (6728209)
• engine: add BatchAborted error variant so NACKs can't be misclassified as connection-fatal (af1e8e2)
• ov-runtime: fail fast and loud after the packed downstream link is poisoned (64a28b4)
• ov-runtime: harden the packed multi-stage wire against loss and silence (#122) (3b0eefd)
• ov-runtime: surface the step error when a NACK send also fails (8fec1a1)
• runner: apply deferred cancels on lock acquisition and bound the queue (984dbcd)
• runner: never block tokio workers on the engine mutex (#122) (48047c7)
• runner: wake parked streams even when a step panics (84b81ab)
• runner: wake parked streams on submit's NotLoaded early return (628ef9b)

Refactor

• runner: enforce wake-on-release via an engine lock guard (3633f22)

Documentation

• correct lock-protocol and blocking-API comments (d2f5e16)
• multi-stage packed is available again; record the #122 root cause (da8a892)

Testing

• runner: cover NACK-driven batch aborts and close-while-parked (d157382)

v0.1.7

0.1.7 (2026-08-06)

Features

• metrics: Prometheus /metrics endpoint — request, generation, engine, and transport metrics (#16) (adcd3cd)
• metrics: Prometheus /metrics endpoint — request, generation, engine, transport metrics (#16) (fa0a326)

Bug Fixes

• engines: stop reporting engine failures as empty successful completions (a895407)
• metrics: count an over-window prompt rejection (9ec49e7)
• metrics: count every pre-generation engine rejection, not two of them (9b67680)
• metrics: review round — cancel accounting, teardown, QueueFull capacity, timing artifacts (d9b1f8f)
• ov-genai: a failed generate() must emit an error chunk, not an empty success (14b6d49)
• runner: fail loud on shutdown, and book teardown deterministically (bcf5a11)

Documentation

• metrics: fix the scrape example and the HELP strings that contradict their docs (48b98a3)

Testing

• cover the teardown HTTP surface and the qwen36 empty-prompt rejection (a503ad6)
• metrics: make three assertions that could not fail actually fail (8c10cbf)
• runner: pin metric attribution across concurrent streams (b75d4a3)

v0.1.6

0.1.6 (2026-08-04)

Features

• npu: continuous batching on the NPU via packed multi-slot decode (7cd3190)
• npu: multi-stage packed wire + per-slot cancel (63e44ee)
• npu: packed multi-slot substrate for continuous batching (seq-as-batch) (f36c408)
• npu: prefix caching via a shared read-only KV region (14c9903)
• npu: wire packed multi-slot execution into OvRuntimeEngine (9ab6746)

Bug Fixes

• dist-spec: remove a byte-offset slice that could panic the worker (6ba63b6)
• npu: answer 413, not 503, when a prompt cannot fit a packed slot (165fcdd)
• npu: bound the packed multi-stage reply wait (6560dda)
• npu: complete packed prefill inside one step + report prompt_tokens (481174b)
• npu: keep attention sinks when a packed slot's KV region slides (62efd11)
• npu: packed usage double-count + f16 wire dtype; document parity findings (866544a)
• npu: packed wire must use the block_in_place-aware dispatch (126b232)
• npu: refuse a packed prompt that cannot fit its slot's KV region (fee9c33)
• npu: refuse a packed variant narrower than its own slot count (c5e64c7)
• npu: withhold multi-stage packed — it can lose a token frame and wedge (db18d0d)
• runtime: keep attention sinks when the single-task KV ring slides (feff6b1)
• runtime: stop the ov-runtime delta duplicating on a resolved glyph (428a875)

Performance

• npu: skip the prefill-variant compile in packed mode + NPU e2e results (205f894)

Documentation

• npu: fix stale plan-frame shape in perf doc — [1,3,S], not [1,2,S] (0435285)
• npu: point the multi-stage gate at its tracking issue (637cb96)
• npu: reconcile packed-slots with #116 continuous batching (1ef825f)
• npu: record the packed-slots end-to-end run (1bd4009)

Testing

• npu: add solo-only capture mode (8f0efe4)
• npu: end-to-end accuracy parity harness for packed multi-slot decode (3a034a9)
• npu: scored long-form accuracy benchmark (6eff7c3)
• npu: scored task-accuracy benchmark for packed decode (10a76f0)

v0.1.5

0.1.5 (2026-07-30)

Bug Fixes

• shim: explicit bool return type in AVX-VNNI probe lambda (828b401)
• shim: explicit bool return type in AVX-VNNI probe lambda (1cebab4)

v0.1.4

0.1.4 (2026-07-29)

Features

• cli: warn when --cb is enabled on a CPU device (c8b6c63)
• ov-genai: continuous batching via ContinuousBatchingPipeline (#20) (2229af0)

Bug Fixes

• cli: reject --cb on an NPU device (280e1b5)
• ov-genai: bound the cb liveness heartbeat so a wedged batch still fails (2a83442)
• ov-genai: cb step() signals liveness so long prefills survive the stall guard (c7f14f1)
• ov-genai: cb warmup reports failure instead of always logging ok (6420243)
• ov-genai: report a cb scheduler eviction as an error, not a clean stop (48a342b)
• ov-genai: stop admitting work onto a dead cb pipeline (3b2812c)
• runner: deliver a stream's chunks in the order the engine produced them (abccce5)
• shim: apply the chat template on the cb path (88a3a76)
• shim: move the cb UTF-8 hold-back to Rust and make it resync (e24e2c8)

Refactor

• shim: drop the unused cb has_unfinished entry point (13320a4)
• shim: tie a CbHandle's lifetime to its pipeline via Arc (da219dd)

Documentation

• ov-genai: characterise when --cb helps and when it hurts (b55c754)
• ov-genai: correct the --cb example and three overstated claims (951a13b)
• shim: correct the cancel-vs-stop and request_id claims (8e1c824)

Testing

• ov-genai: cover the cb engine state machine behind a pipeline seam (f28dad0)
• ov-genai: pin cb/LLMPipeline chat-template parity (cb34bb9)
• shim: cover the cb resync path with unit tests (b06158d)

v0.1.3

0.1.3 (2026-07-25)

Features

• cli: --prefill-device / --no-chunked-prefill + shard --static-prefill-seq (46c81b6)
• engine: --park-prefill — release prefill weights between prefills (381c08b)
• engine: chunked multi-token prefill + per-phase device on the static path (cff6933)
• engine: consume AOT NPU blobs — .blob sibling of the IR imports instead of compiling (638afa9)
• engine: npuw bank probe + park-without-cache warning (2fb27d4)
• export: emit chunked-prefill static IR variant (--static-prefill-seq) (c1482cb)
• hybrid NPU+CPU execution — chunked prefill on one device, decode on another (ae008f8)
• shim: AOT blob-import FFI + probe — NPU compile spike moves off-box (6e97af2)
• shim: AVX2 GEMV path, CSE weights-tag, residency probe + spike notes (df9a770)
• shim: CascadiaInt4Gemv extension op — decode GEMV from the .bin mmap (spike) (4274b5d)
• shim: oneDNN embedding probe — endgame closed by data (fork-only kernels) (ccb47b0)
• shim: PERF_COUNT profiling FFI + flat kernels — gap fully attributed (487c5fb)
• shim: sibling GEMV fusion (q/k/v, gate/up) — measured perf-neutral (e93a3e2)
• weight-residency paths for the hybrid split — --park-prefill, NPUW bank probe, in-place GEMV RFC (7f4f37a)

Bug Fixes

• engine-openvino: lower the near-tie guard to the first token (10 -> 1) (c87c09d)
• engine: review findings — park on cancel/failure, parked-check before ring mutation, warmup ensure, doc placement, probe resilience, 1-based labels (44a78fe)
• engine: review findings — per-device prefill plugin props, cross-stage geometry check, loud argmax on truncated logits (4bd6a51)
• engine: review findings — window-parity cap, sub-chunking, guards, reuse (fb5d9aa)
• engine: warn on an unverifiable AOT blob instead of silently trusting it (701cbf4)
• export: remove stale prefill variant on re-export (5814e59)
• review: gemv CACHE_DIR strip via props, stale-blob guard, Linux SIMD/tbb build, matcher+fusion guards, tellg check, pacing safety, profiling truncation, xfer wait bug (506c254)
• shim: reject an empty CascadiaInt4Gemv weights_tag (CSE-merge guard) (4177b9d)
• transport: pace large tensor payloads into bounded bursts — DERP-relayed links drop ~750KB single bursts intermittently (8ec831a)
• transport: warn on an unparseable CASCADIA_SEND_BURST_BYTES instead of silent OFF (9be0d40)
• transport: warn when CASCADIA_SEND_BURST_BYTES is clamped up to the 64 KiB floor (566cdb5)

Performance

• shim: gemv-offload 66-73% -> 75-79% of stock; frontier mapped (25d4133)

Documentation

• experiments: track the gemv-offload spike notes incl. NPU cache-import init warning (f49dcaf)
• experiments: track the NPUW weights-bank probe notes (gitignore exempts) (dff68e0)
• perf: 70B blocker dossier — frame black-holing on DERP links, everything else ruled out (eed8d82)
• perf: add 2-stage hybrid pipeline smoke numbers (6db3ccd)
• perf: big-model NPU routes — 2-stage PP, AOT blob import, NPUW folding (all measured) (d3ea8ce)
• perf: correct over-window semantics + short-prompt and placement notes (62000f5)
• perf: device x model-size matrix (1B/3B/8B x CPU/GPU/NPU) on LNL 32GB (fbd8970)
• perf: finalize tier results — 32B 3-box measured, 70B health-validated with isolated blockers (0f80217)
• perf: hybrid NPU+CPU phase split — design, quickstart, measured results (328ef7e)
• perf: link PR #107 in status line (85cc925)
• perf: NPU TTFT tier benchmarks — method, 14B results, attribution, fleet deployment learnings (08369df)
• perf: parking measurements + npuw bank probe results (d15484a)
• perf: post-fix re-validation numbers (33x long-prompt, over-window leg, warm smoke) (0610d22)
• qualify the remaining token-exactness claims for near-tie tolerance (aeba7eb)
• rfcs: narrow the in-place GEMV ask per spike evidence (8f1e37a)
• test: parking leg is near-tie-tolerant parity, not token-identical (527c9bc)

Testing

• cli: cover shard --static-prefill-seq validation + forwarding (4e6d109)
• cli: cover the worker phase-split flag guards (619a3be)
• engine-openvino: CASCADIA_PARITY_SOFT knob for GPU/cross-device sweeps (64d9f93)
• engine-openvino: CASCADIA_STATIC_TASKS knob — steady-state TTFT past the NPU cache-import init (109603e)
• engine-openvino: CASCADIA_WARM_INFER — isolate graph-specific first-inference pathologies (db06dd5)
• engine-openvino: cover import_plugin CACHE_DIR strip (d2dba1c)
• engine-openvino: parking leg honors CASCADIA_PARITY_SOFT (ca937a0)
• engine-openvino: sequential cache-warm probe — no overlapping NPU compile transients at pipeline bring-up (988b1b9)
• engine-openvino: tolerate near-tie prefill forks; correct the token-exact claim (7623bbb)
• engine-openvino: unit-test the parity verdict (no hardware) (968e2c4)
• engine: chunked-prefill ring equivalence + phase-split parity gate ([f65701b](https://github.com/lab...

v0.1.2

0.1.2 (2026-07-21)

Features

• api: real /health readiness (was a hardcoded 200) (c9b6ebc)
• cli: serve a clean model name (basename of --model, or --served-model-name) (0895c2d)
• deepseek v4 support in the sparse-moe engine (4651129)
• dsv4: DeepSeek-V4-Flash exporter + CPU reference model (660e56a)
• dsv4: optional OpenVINO int4 expert backend (GPU/CPU/NPU) (7ee21a0)
• dsv4: ship R1 chat_template so chat completions render instruct prompts (8571cff)
• dsv4: sparse-MoE inference engine + distributed pipeline (c4c9c26)

Bug Fixes

• build, CLI and docs bugs found by running every documented command (c03c68c)
• build: enforce MSRV 1.89, and make the Dockerfile build (a4aa45a)
• cli: usable errors for models, python and deps (5173325)
• dsv4: apply the finish_reason + truncation fixes to the single-stage path (9e749d7)
• dsv4: bound context, fix seeded-sampling parity, and harden the worker (061c912)
• dsv4: bound the pipeline reply-recv so a dead peer fails fast (no wedge) (8f197cb)
• dsv4: exporter carries chat_template.jinja + serving sidecars (a35340a)
• dsv4: harden streamed prefill against mid-stream failure (fe56bb6)
• dsv4: out-wait a cold slice load on the downstream connect (ceb7f7f)
• dsv4: reject a manifest whose compress_ratios can't cover its layers (b3a9480)
• dsv4: reject stage load when manifest exported_layers misses the range (eb6f760)
• dsv4: remove the unusable ov_ir expert export mode (910ba4c)
• dsv4: report finish_reason=length when the context window caps decode (bf637e8)
• dsv4: use one per-token reply deadline (drop the batched-prefill x10) (ca4df33)
• dsv4: warn instead of silently dropping an over-budget prompt tail (e3f7427)
• e2e: find cascadia.exe on Windows (1189b5b)
• engine: ov-genai requires the tokenizer IRs (d37db22)
• pin Intel's key properly, and the bugs review found (a13d28c)
• review-pass follow-ups (d0a2a73)
• scripts: install Intel's current GPU drivers, safely (c5167d7)
• transport: TCP keepalive on inter-rank pipeline sockets (ad6f5ea)

Performance

• dsv4: AVX2 batch expert kernel (on-node bit-exact) (2a2604f)
• dsv4: AVX2+FMA dot product in GEMV, chunked mmap expert dequant (d21c0ee)
• dsv4: batch-union expert kernel for prefill (forward_batch) (7e17b1d)
• dsv4: batch-union MoE in forward_layers_prefill (49ea601)
• dsv4: batched prefill across the pipeline (ForwardBatchPrefill) (9ecbf06)
• dsv4: env-gated per-section decode profiler (DSV4_PROFILE) (c5dd062)
• dsv4: fused AVX2 int4 dequant-dot for mmap experts (1520a83)
• dsv4: parallelize GEMV across cores with rayon (27bae87)
• dsv4: parallelize the o_proj wo_a GEMV (4913f43)
• dsv4: store attention projections as bf16 to halve GEMV bandwidth (37f01e9)
• dsv4: stream prefill one-way to pipeline it across ranks (0375173)

Documentation

• add a CLI reference (2a74a2f)
• dsv4: correct the mmap-vs-eager "bitwise" claim (08d663e)
• dsv4: fix ForwardPrefill attribution in the architecture doc (ed23f4f)
• dsv4: trim architecture doc to essentials, document decode perf (e708d24)
• fix the commands that don't work (5dfb645)
• install from Intel's unified suite in the by-hand block (9d29e59)

Testing

• dsv4: commit the tiny export fixture tensors so CI can load it (4b0262e)
• dsv4: skip golden tests when the gitignored fixture is absent (33dd33d)

Miscellaneous

• deps: bump spin off yanked 0.9.8 (de4df2e)

v0.1.1

0.1.1 (2026-07-09)

Features

• #77 Part B Path 1 � qwen3_5_moe gate, aliases, single-stage docs (e01faf4)
• api: add /v1/completions (OpenAI legacy completions) (#19) (d194c94)
• api: add /v1/completions (OpenAI legacy completions) (#19) (8989b4e)
• api: expose render_chat_prompt for standalone callers (63b5477)
• api: non-stream chat emits tool_calls + tool_calls finish_reason (#44) (e41306f)
• api: Ollama dialect + tool-call plumbing (9fe99ed)
• api: OpenAI sampling params + finish_reason + streaming usage (#14) (58143e6)
• api: OpenAI sampling params, finish_reason, streaming usage (#14) (21c15f2)
• api: OpenAI tool calling (function calling) [cascadia-enterprise#44] (780cbf4)
• api: parse_tool_calls for Llama + Qwen tool output (#44) (79ad3bc)
• api: render chat_template from tokenizer_config.json (Jinja2) (339091a)
• api: renderer forwards tools + message tool fields (#44) (150748e)
• api: SSE streaming, logprobs, cancellation, introspection, tracing (d1e858a)
• api: streaming emits single indexed tool_calls delta (#44) (ee140c0)
• api: tool-calling request/response schema types (#44) (963c5c2)
• cascadia-api: expose render_chat_prompt for in-process embedders (0382ec3)
• cascadia-api: expose render_chat_prompt for in-process embedders (db2a3a0)
• chunk-level n_tokens for accurate tok/s with spec-decode (7c02e55)
• cli: add --version flag (4dd173e)
• cli: add doctor, run, discover, and completions subcommands (acce033)
• cli: plumb OpenVINO performance properties through all engines (525757f), closes #13
• cli: plumb OpenVINO performance properties through the CLI (#13) (f10cf40)
• cli: profile-devices subcommand � per-device tok/s for #41 step 1 (#45) (8a20154)
• cli: type OV mode flags as ValueEnum; warn on ignored perf flags (db171f1)
• cluster: mDNS discovery + master election + automatic placement (70cd54b)
• dashboard: chat playground with streaming SSE + decode stat bar (ab266f1)
• dashboard: cluster web UI + chat playground (tahoma-dashboard crate) (73444fd)
• dashboard: ClusterStrip on the chat surface (191f5e7)
• dashboard: live request/token stats + per-node system specs (2994caf)
• dashboard: node cards + latency matrix on the Cluster page (88fcec7)
• dashboard: scaffold tahoma-dashboard crate with /api/topology + /api/stats (560a83f)
• dashboard: scaffold Vite + React + Tailwind SPA with cascadia design tokens (17534a5)
• dashboard: serve dashboard alongside API in tahoma worker --api (660e13e)
• dashboard: TCP latency probe loop populates the matrix (871c09e)
• dashboard: use the Cascadia brand logo + favicon (70335e6)
• dist_spec: per-round streaming (was buffering all output to one chunk) (bf0d68e)
• download: HuggingFace model registry + pull endpoints (00f6019)
• engine,transport,runner: pipeline-link reliability & dead-peer recovery (d10e44a)
• engine: Engine::step returns EngineResult so failures are visible (826a01b)
• engine: Gemma 4 runtime (--engine gemma4) � single + multi-stage pipeline-parallel (3ff7809)
• engine: implement engine-side cancel() propagation (#15) (4634c74)
• engine: implement engine-side cancel() propagation (#15) (e6a5ce2)
• engine: KV cache via forward_layers_cached (ccec98c)
• engine: real cancel on ov-runtime + mock; document trait contract (cc430ea)
• engines: add ov-genai single-stage LLMPipeline engine (6a22922)
• engines: add ov-genai single-stage LLMPipeline engine (0c95e1a)
• engine: single-stage OpenVINO Runtime engine via optimum-intel (8d7551c)
• engines: plumb OV plugin properties through ov-runtime + ov-dist-spec (ccd2aa8)
• export: --free-source-shards for in-place re-quantization (371fbfc)
• export: MiniMax-M2 exporter to OV-IR sparse-MoE layout (bb1f222)
• export: NF4 (distribution-matched 4-bit) expert quant (e70f656)
• export: per-component precision flags (--shell-quant / --head-quant) (dfda2c1)
• export: streaming per-layer full-model export for MiniMax-M2 ([8046c2f](https://github.com/labscommunity/cas...