Skip to content

v0.1.3

Pre-release
Pre-release

Choose a tag to compare

View all tags
@github-actions github-actions released this 28 Jul 09:35
v0.1.3
fc16203
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

0.1.3 (2026-07-25)

Features

  • cli: --prefill-device / --no-chunked-prefill + shard --static-prefill-seq (46c81b6)
  • engine: --park-prefill — release prefill weights between prefills (381c08b)
  • engine: chunked multi-token prefill + per-phase device on the static path (cff6933)
  • engine: consume AOT NPU blobs — .blob sibling of the IR imports instead of compiling (638afa9)
  • engine: npuw bank probe + park-without-cache warning (2fb27d4)
  • export: emit chunked-prefill static IR variant (--static-prefill-seq) (c1482cb)
  • hybrid NPU+CPU execution — chunked prefill on one device, decode on another (ae008f8)
  • shim: AOT blob-import FFI + probe — NPU compile spike moves off-box (6e97af2)
  • shim: AVX2 GEMV path, CSE weights-tag, residency probe + spike notes (df9a770)
  • shim: CascadiaInt4Gemv extension op — decode GEMV from the .bin mmap (spike) (4274b5d)
  • shim: oneDNN embedding probe — endgame closed by data (fork-only kernels) (ccb47b0)
  • shim: PERF_COUNT profiling FFI + flat kernels — gap fully attributed (487c5fb)
  • shim: sibling GEMV fusion (q/k/v, gate/up) — measured perf-neutral (e93a3e2)
  • weight-residency paths for the hybrid split — --park-prefill, NPUW bank probe, in-place GEMV RFC (7f4f37a)

Bug Fixes

  • engine-openvino: lower the near-tie guard to the first token (10 -> 1) (c87c09d)
  • engine: review findings — park on cancel/failure, parked-check before ring mutation, warmup ensure, doc placement, probe resilience, 1-based labels (44a78fe)
  • engine: review findings — per-device prefill plugin props, cross-stage geometry check, loud argmax on truncated logits (4bd6a51)
  • engine: review findings — window-parity cap, sub-chunking, guards, reuse (fb5d9aa)
  • engine: warn on an unverifiable AOT blob instead of silently trusting it (701cbf4)
  • export: remove stale prefill variant on re-export (5814e59)
  • review: gemv CACHE_DIR strip via props, stale-blob guard, Linux SIMD/tbb build, matcher+fusion guards, tellg check, pacing safety, profiling truncation, xfer wait bug (506c254)
  • shim: reject an empty CascadiaInt4Gemv weights_tag (CSE-merge guard) (4177b9d)
  • transport: pace large tensor payloads into bounded bursts — DERP-relayed links drop ~750KB single bursts intermittently (8ec831a)
  • transport: warn on an unparseable CASCADIA_SEND_BURST_BYTES instead of silent OFF (9be0d40)
  • transport: warn when CASCADIA_SEND_BURST_BYTES is clamped up to the 64 KiB floor (566cdb5)

Performance

  • shim: gemv-offload 66-73% -> 75-79% of stock; frontier mapped (25d4133)

Documentation

  • experiments: track the gemv-offload spike notes incl. NPU cache-import init warning (f49dcaf)
  • experiments: track the NPUW weights-bank probe notes (gitignore exempts) (dff68e0)
  • perf: 70B blocker dossier — frame black-holing on DERP links, everything else ruled out (eed8d82)
  • perf: add 2-stage hybrid pipeline smoke numbers (6db3ccd)
  • perf: big-model NPU routes — 2-stage PP, AOT blob import, NPUW folding (all measured) (d3ea8ce)
  • perf: correct over-window semantics + short-prompt and placement notes (62000f5)
  • perf: device x model-size matrix (1B/3B/8B x CPU/GPU/NPU) on LNL 32GB (fbd8970)
  • perf: finalize tier results — 32B 3-box measured, 70B health-validated with isolated blockers (0f80217)
  • perf: hybrid NPU+CPU phase split — design, quickstart, measured results (328ef7e)
  • perf: link PR #107 in status line (85cc925)
  • perf: NPU TTFT tier benchmarks — method, 14B results, attribution, fleet deployment learnings (08369df)
  • perf: parking measurements + npuw bank probe results (d15484a)
  • perf: post-fix re-validation numbers (33x long-prompt, over-window leg, warm smoke) (0610d22)
  • qualify the remaining token-exactness claims for near-tie tolerance (aeba7eb)
  • rfcs: narrow the in-place GEMV ask per spike evidence (8f1e37a)
  • test: parking leg is near-tie-tolerant parity, not token-identical (527c9bc)

Testing

  • cli: cover shard --static-prefill-seq validation + forwarding (4e6d109)
  • cli: cover the worker phase-split flag guards (619a3be)
  • engine-openvino: CASCADIA_PARITY_SOFT knob for GPU/cross-device sweeps (64d9f93)
  • engine-openvino: CASCADIA_STATIC_TASKS knob — steady-state TTFT past the NPU cache-import init (109603e)
  • engine-openvino: CASCADIA_WARM_INFER — isolate graph-specific first-inference pathologies (db06dd5)
  • engine-openvino: cover import_plugin CACHE_DIR strip (d2dba1c)
  • engine-openvino: parking leg honors CASCADIA_PARITY_SOFT (ca937a0)
  • engine-openvino: sequential cache-warm probe — no overlapping NPU compile transients at pipeline bring-up (988b1b9)
  • engine-openvino: tolerate near-tie prefill forks; correct the token-exact claim (7623bbb)
  • engine-openvino: unit-test the parity verdict (no hardware) (968e2c4)
  • engine: chunked-prefill ring equivalence + phase-split parity gate (f65701b)

Miscellaneous

  • transport: default send pacing off — did not resolve the DERP frame loss; kept as experiment knob (4592fe2)
Assets 3
Loading