v1.3.1 — fused short-prefill attention (TTFT win)

The default cuBLAS prefill attention has per-call overhead (handle setup, ~70
batched-GEMM + softmax launches/layer, N×N score-buffer traffic) that dominates
at small sequence length. genie now dispatches the fused F32 query-tiled kernel
below a ~768-token crossover (cuBLAS's tensor-core GEMMs win at long context).

Prefill tok/s (cuBLAS-only → length-dispatch), Gemma 4 E2B on Orin

length	before	after	Δ
~76	224	461	2.05×
~236	475	690	1.45×
~547	587	632	1.08×
~1547	640	639	—
~9098	537	537	—

Typical 50–300-token interactive prompts prefill ~1.4–2× faster (TTFT); long
context unchanged. vs llama.cpp real cold prompt-eval (llama-simple), genie
short-prefill is 2–7× faster (llama pays a ~1–2 s warmup genie does not).
JLLM_ATTN_SHORT_TILED=0 reverts.

Full Changelog: v1.3.0...v1.3.1

v1.3.0 — split-K decode attention (beats llama.cpp at depth)

Split-K (flash-decoding) decode attention — fixes genie's decode-at-depth
collapse and pushes decode past llama.cpp at every context depth.

The decode-attention kernels launched grid=(n_heads) — only 8 blocks on the
8-SM Orin (ncu: ~11 % occupancy, 3 % throughput), and each per-head block
walked the whole KV serially, so decode throughput fell with depth. Split-K
splits the KV range across n_splits blocks per head, each computing an
online-softmax partial; a reduce kernel combines them per head. Default-on
(JLLM_SPLITK=0 reverts).

Decode-at-depth vs llama.cpp (same Orin, same gemma-4-E2B-it-Q4_K_M, MAXN_SUPER)

Context depth	v1.2.1 (no split-K)	v1.3.0	llama.cpp tg	v1.3.0 vs llama
~512	25.7	31.8	20.1	1.58×
~1024	19.7	30.5	20.8	1.47×
~2048	14.2	26.8	20.4	1.31×
~4096	8.6	21.0	19.1	1.10×

Without split-K genie was 2.2× behind llama.cpp at 4096 (8.6 vs 19.1); with
it genie leads at every depth. Shallow decode is unchanged (the single-block
kernel is kept below 64 tokens).

Details

• flash_decode_splitk_partial_kernel + flash_decode_reduce_kernel, wired into
  both the per-step and CUDA-graph decode paths (the graph bakes n_splits at
  capture; the kernel derives each split's KV range from *d_pos at replay).
• Handles FP16 + INT8 KV, Gemma's cache_head_dim sliding layers, and the
  sliding-window mask. Numerically exact vs the single-block kernel (standalone
  tests/test_flash_decode_splitk.cu: max abs diff 0.0, 3.2–7.3× at seq 512–4096).

Full Changelog: v1.2.1...v1.3.0

v1.2.1 — Gemma 4 E2B decode CUDA-graph fix

Fixes the Gemma 4 E2B decode CUDA-graph path and enables it for the E2B
model (which uses KV sharing). The graph was added in v1.2.0 but excluded for
shared-KV models and carried three latent bugs — none ever shipped to users
(the path was guard-disabled for every real Gemma 4 model).

Bugs fixed

• flash_attention_decode_dyn ignored cache_head_dim — derived the KV
  position stride/offset from head_dim, so Gemma sliding layers (head_dim
  256, cache slot 512) read each KV slot at half stride.
• PLE input buffer aliased scratch — the captured graph reused the same
  scratch offsets for its per-layer buffers and overwrote the PLE values
  mid-replay. Fixed with a dedicated persistent buffer.
• scale_embedding missing from the graph path (dominant bug) — the
  forward ran on an embedding ~39× too small (Gemma ScaledWordEmbedding). Input
  RMSNorm hides it from attention, but the residual stream and the per-layer
  embedding block read the embedding directly → degeneration.

Also adds a device-position Q-only RoPE (rope_inplace_dyn) so the captured
graph can rotate Q for the trailing KV-sharing layers.

Performance (Orin Nano Super, gemma-4-E2B-it-Q4_K_M, MAXN_SUPER)

Clean equal-token decode A/B:

Context depth	graph OFF	graph ON	Δ
shallow / typical	30.7	32.4	+5.5 %
512-token depth	23.9	24.0	~neutral

The graph collapses ~448 per-token kernel launches into one cudaGraphLaunch,
recovering host-launch overhead that matters most when tokens are fast (shallow
context); at depth the per-token GPU work dominates and the win tapers to
neutral. Default-on; JLLM_DECODE_GRAPH=0 reverts. Greedy output matches the
per-step path (a 511-token prompt crossing the sliding window is byte-identical),
modulo the existing int8-GEMM-class borderline tie-breaks.

Full Changelog: v1.2.0...v1.2.1

v1.2.0 — decode optimization cycle

A decode-focused optimization cycle on top of v1.1.0, for Gemma 4 E2B on
Jetson Orin Nano Super. No correctness change (greedy output unchanged) and
no change to the Qwen3 path — every optimization is env-gated (JLLM_*=0).

What's in it

• perf(rope): precomputed cos/sin table — no per-decode-step trig.
• perf(attn): warp-parallel Q·Kᵀ + block softmax in the decode attention kernel.
• perf(gemv): dp4a triple QKV GEMV (Q4_K+Q4_K+Q6_K), one shared q8_1 activation.
• perf(kernels): half2-vectorized vec_add / RMSNorm / GeGLU / SwiGLU / scale.
• perf(engine): Gemma 4 CUDA-graph support — inert for E2B (20 shared-KV
  layers → graph correctly skipped); benefits only a non-shared-KV variant.
• fix(engine): decode-graph skip messages now print once (were per-token).
• chore(bench): scripts/bench_v11x.sh decode profiler.

Measured (Orin Nano Super, gemma-4-E2B-it-Q4_K_M, MAXN_SUPER)

Same-conditions decode A/B (v1.1.0 vs v1.2.0 binaries, identical prompt, 3
interleaved runs each, ≤0.3 tok/s variance), at three context depths:

Context depth (prompt len)	v1.1.0	v1.2.0	Δ
shallow (~11 tok)	29.1	31.0	+6.5 %
~450 (421 tok)	22.1	24.4	+10.4 %
512 (511 tok)	21.1	23.8	~+12 %

The decode gain grows with context depth — the warp-parallel decode
attention does more work per token as the KV cache grows, so it saves more
there. The 512-depth row is the apples-to-apples comparison with v1.1.0's
"23.5 tok/s at parity" methodology (v1.1.0 reads 21.1 here on
jetson_clocks-unlocked MAXN_SUPER). The gain comes from the attention kernel,
the dp4a QKV triple, and the half2 kernels; the RoPE table and the Gemma 4
graph path contribute negligibly for E2B.

Full Changelog: v1.1.0...v1.2.0

v1.1.0 — Gemma 4 E2B support + prefill/decode kernel cycle

Adds Gemma 4 E2B support and a Gemma-4-focused kernel optimization cycle.
Gemma 4 runs greedy-identical to llama.cpp on Jetson Orin Nano Super (including
contexts past the 512-token sliding window). No change to the Qwen3 path —
every optimization is env-gated.

Performance — Gemma 4 E2B Q4_K_M, Orin Nano Super (MAXN_SUPER)

Metric	Before	v1.1.0	Δ
Decode (tg)	12 tok/s	23.5 tok/s	+96 % (llama.cpp parity)
Prefill (1261-tok)	152 tok/s	620 tok/s	+308 % (4.1×)
Prefill (8192-tok)	63 tok/s	126 tok/s	2× (long-context collapse fixed)

Covers #96–#101 (Gemma 4 support), #102–#106 (decode parity), #117–#128
(prefill cycle). See CHANGELOG.md for full notes.

Tag backfilled 2026-06-20 (release was merged via #129 but not tagged at the time).

v1.0.0 — first stable release

First stable release of genie-ai-runtime — the LLM inference engine for Jetson Orin Nano Super 8 GB. No engine behavior change vs v0.1.0-alpha.12 — same Qwen3 path, same INT8 KV default, same kernels. v1.0 cuts the -alpha.N suffix and ships the README/ROADMAP rewrite + --version flags.

Headline numbers (Qwen3-4B Q4_K_M, 25 W MAXN SUPER)

Metric	alpha.2	v1.0.0	Δ
Prefill (33-tok cold)	8.2 tok/s	38.0 tok/s	+363 %
Decode	7.5 tok/s	9.9 tok/s	+32 %
Cold TTFT	2200 ms	877 ms	−60 %
Warm-turn TTFT (Path F, 67 % prefix)	n/a	444 ms	new
KV pool memory @ 1024 ctx	n/a	74 MB	new (−49 % vs alpha.11 FP16)

vs llama-bench pp18 = 17.97 ± 0.65 tok/s: +115 % prefill on the same hardware + model.

What v1.0 ships

• Path A → I complete: tensor-core MMQ Q4_K prefill, INT4-uint32 decode GEMV, batched prefill, persistent KV with longest-prefix hydrate, INT8 KV default (FP16-ULP-bounded drift). See ROADMAP.md for the path-by-path narrative.
• Engine library (libjetson_llm_core.a) — genie-claw embeds this directly.
• CLI (jetson-llm) — single-prompt, interactive chat, persistent-KV, Qwen3 reasoning mode, cold/warm load timing.
• HTTP server (jetson-llm-server, opt-in -DJLLM_BUILD_SERVER=ON) — OpenAI-compatible, SSE streaming, reasoning_content split, systemd unit + installer.
• Docs rewritten — README.md is now a clear intro; ROADMAP.md carries the technical history; docs/server.md is the full HTTP surface.

Target hardware

Jetson Orin Nano Super 8 GB (SM 8.7, L4T R36.x, CUDA 12.6). Build refuses to configure off aarch64. No prebuilt binaries — clone and cmake --build on the Jetson itself. Prereqs in GeniePod/genie-os#1.

Quickstart

```bash
git clone https://github.com/GeniePod/genie-ai-runtime.git
cd genie-ai-runtime
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
./build/jetson-llm -m /path/to/model.gguf -p "Hello"
```

Known follow-ups (not blockers for v1.0)

• #67 — Path F format v3 to persist INT8 KV scales (server can drop FP16 default and unify with CLI once this lands)
• #7 — 24-hour soak + packaging + genie-claw default flip
• #4 — 100-iter × 1000-token stability soak (harness shipped in PR #70, actual run pending)
• #58 — Path G v2 decode-throughput optimization
• #56 — Cold-TTFT reduction beyond Path E + F

Full release notes: CHANGELOG.md.

v0.1.0-alpha.12 — Path I: INT8 KV cache (default, ~50 % KV memory saved)

Path I — INT8 KV cache (now default)

`--int8-kv` now works directly (was warned-and-ignored in alpha.11) and is the default for both CLI and embedded callers. `--fp16-kv` opts back into full-precision KV.

The flag had been a half-implemented stub since alpha.2 — Path I (six phases I1–I6) wired the per-head scale machinery end-to-end and validated quality on Jetson Orin Nano Super 8 GB before flipping the default.

Headline result

	FP16 KV (alpha.11)	INT8 KV (alpha.12 default)	Δ
KV pool memory	144 MB	74 MB (72 MB body + 2.25 MB per-head scales)	−49 %
Decode tok/s (33-tok prompt + 40 decode)	10.0	9.9	within noise
Prefill (33 tok)	857 ms / 38.5 tok/s	867 ms / 38.0 tok/s	+10 ms (scale-lookup overhead)
TTFT	867 ms	877 ms	+10 ms
Output	reference	sensibly-identical, FP16-ULP-bounded word-level drift	✓

The real win is memory headroom for longer contexts (8 K, 16 K) and concurrent voice/HA on the 8 GB Jetson. Throughput is essentially unchanged at chat-typical contexts because attention is only ~2 % of decode wall at N≈40; the INT8 savings on attention's KV-read bandwidth compound at longer contexts.

Path I series

PR	Phase	What
#63	I1 — per-head scale storage	New 2.25 MB scales region in KVCachePool; INT8-mode-only allocation
#64	I2 — fp16_to_int8 wiring + kernel bug fixes	per-head conversion + scale capture; fixed uninit row_max and nullptr scale_out crash on the formerly-stub path
#65	I3 — per-position scales through flash_attention + remove broken fallback	attention dequant reads k_scales[kv_pos * n_kv_heads + kv_head]; removed the kv_int8 → N-sequential-decode fallback that made alpha.11 hang on medium prompts
#66 (closed, superseded)	I5 — JLLM_INT8_KV_EXPERIMENTAL env gate	engineer-only switch for the I5 quality eval; obsolete once default flipped
#68	I6 — default flip + Path F interop guard	this release

Full umbrella + audit: #62.

Quality eval (the pivot, I5)

Single-prompt sensibility check at temp=0:

Run	Output (key phrase)
FP16 reference	"...high performance, low power consumption, and advanced AI capabilities, making it suitable for edge computing and real-time applications."
INT8 short	"...high performance, low power consumption, and advanced AI capabilities, making it suitable for edge computing applications requiring real-time processing and efficiency."
INT8 long (200-tok decode)	Full coherent paragraph about Jetson Orin Nano Super's hardware/software fit for smart-home LLM serving — comparable structure to the FP16 long-prompt baseline

Identical opening (24 tokens character-identical to FP16), then minor word-level paraphrase of the same point. Coherent, on-topic, no garbage / repetition / off-topic drift even at non-trivial context. v1-track will broaden to multiple prompts + perplexity holdout.

Path F (#45) interop guard

The on-disk format from F4a (#50, v2) doesn't yet carry the per-head INT8 scales that attention needs at hydrate time. Saving INT8 bytes without the scales would silently produce garbage on the next turn. alpha.12 skips Path F save when `kv_int8` is active, with a one-time stderr note. Use `--fp16-kv` if you need persistent KV today.

Path F format v3 (persist scales region) tracked separately as #67 — the follow-up that closes this gap and lets INT8 KV + persistent KV coexist for multi-turn warm-start.

vs llama.cpp (unchanged from alpha.11)

	Prefill (pp18)	Decode (tg64)
llama.cpp `-ngl 22`	17.97 ± 0.65 tok/s	6.33 ± 0.25 tok/s
genie-ai-runtime alpha.12	38.0 tok/s	9.9 tok/s
Δ	+111 %	+57 %

Cumulative since alpha.2

	alpha.2	alpha.12	Δ
Prefill (33-tok cold)	8.2 tok/s	38.0 tok/s	+363 %
Decode	7.5 tok/s	9.9 tok/s	+32 %
Cold TTFT	2200 ms	877 ms	−60 %
Warm-turn TTFT (Path F)	n/a	444 ms (with --fp16-kv)	new in alpha.9
KV pool memory	n/a	74 MB (−49 % vs alpha.11)	new in alpha.12
Cache safety (Path F5)	n/a	1 GB LRU cap	new in alpha.10

Not in this release

• Path F format v3 — persist INT8 scales (#67). The follow-up that restores INT8 KV + persistent KV multi-turn interop. Will land separately.
• Quality eval on a corpus, not just one prompt. alpha.12 ships on the single-prompt sensibility check (alpha-track bar); v1-track will broaden to multiple prompts + perplexity holdout.
• Cosmetic post-hydrate prefill-log over-reporting (Path F4b artifact).

v0.1.0-alpha.11 — release-hygiene cleanup (--int8-kv off, perf re-baseline)

What this release is

Release-hygiene cleanup. No throughput changes; no kernel work. Two honesty fixes that close known user-facing footguns:

1. --int8-kv flag is warned-and-ignored. The INT8 KV path has been half-implemented since alpha.2/3 — setting the flag either produced numerical garbage or hung on medium-length prompts. CLI now warns and runs with FP16 KV. Proper INT8 implementation tracked separately under Path I.
2. Performance numbers re-baselined honestly. The 'Decode: 9.1 tok/s' figure had carried through 6 releases without re-measurement; today's measurement is 10.0 tok/s on the same short-prompt baseline. README perf table also gains a serving-realistic second row.

--int8-kv triage

Symptom	Root cause
Hangs on 57+ token prompts	flash_attention_prefill_batched unconditionally fell back to N sequential decode calls when kv_int8=true (attention.cu:285)
Numerical garbage output	flash_attention_decode_kernel got kv_scales=nullptr → defaulted scale=1.0 when dequantizing INT8
Lost scales	fp16_to_int8 called with nullptr scale output and rows=1 (should be rows=n_kv_heads)

alpha.11 stops the silent-broken behavior. Proper fix needs:

• Per-(layer, pos, kv_head) scale storage in KVCachePool (~2.25 MB additional)
• Wire scales through fp16_to_int8 + flash_attention_decode + flash_attention_prefill_batched
• Path F save/load (#46–#51) must persist the scales region too, or invalidate caches when kv_int8 toggles
• Quality eval (perplexity holdout) before flipping default

That's ~1–2 days careful work — tracked separately.

Honest performance baseline (Jetson Orin Nano Super 8 GB, 25 W MAXN SUPER, 918 MHz)

Shape	Prefill	TTFT	Decode
33-tok cold (canonical short prompt)	38.8 tok/s	859 ms	10.0 ± 0.005 tok/s
57-tok prefill + 200-tok sustained decode (longer-prompt realistic)	40.8 tok/s	1410 ms	9.5 tok/s

Two rows because they answer different questions. The 33-tok row is the apples-to-apples comparison vs alpha.2 → alpha.10 baselines. The 57-tok row captures dispatcher amortization on longer prompts (prefill rate goes up a bit) and attention growth with KV context length (decode rate dips slightly).

vs llama.cpp's pp18 = 17.97 ± 0.65 tok/s and tg64 = 6.33 ± 0.25 tok/s: we still lead +115 % prefill / +50 % decode on the canonical short prompt.

Cumulative since alpha.2

	alpha.2	alpha.11	Δ
Prefill (33-tok cold)	8.2 tok/s	38.8 tok/s	+373 %
Decode	7.5 tok/s	10.0 tok/s	+33 %
Cold TTFT	2200 ms	859 ms	−61 %
Warm-turn TTFT (Path F)	n/a	444 ms	new in alpha.9
Cache safety (Path F5)	n/a	1 GB LRU cap	new in alpha.10

Not in this release

• Path I — proper INT8 KV. Next planned phase. Will land after a quality eval pass.
• Path G G2-full (Q6_K uint16 weight loads). G2-lite (#60) measured −0.09 tok/s, closed as no-op. G2-full needs layout redesign for ~+1 tok/s decode; not committed.
• Streaming (#5). The actually-biggest user-perceived improvement still ahead — doesn't change tok/s but makes TTFT feel near-zero. Open in the alpha-track queue.
• Cosmetic post-hydrate Prefill: N tokens log over-reporting (counts hydrated tokens in N).

v0.1.0-alpha.10 — Path F5: production-safe persistent KV (1 GB LRU cap)

Path F5 — LRU eviction + size cap + stale-tmp cleanup

Production hardening for the persistent KV cache that shipped in alpha.9. The cache directory now self-caps at 1 GB by default and evicts oldest *.bin files by mtime when over budget. Stale *.tmp files left behind by crashed prior saves are cleaned at the next save. Same throughput / TTFT / hydrate behavior as alpha.9 — purely operational safety.

Defaults work out of the box

Var	Default	Effect
JLLM_KV_CACHE_MAX_MB	1024	Total *.bin budget in MB. 0 disables eviction.
JLLM_KV_CACHE_STALE_TMP_S	60	Age (s) past which *.tmp files get unlinked. 0 keeps everything.

Verified on Jetson Orin Nano Super 8 GB

Test	Result
Defaults, 5 MB save	Silent — no eviction triggered (1 GB ≫ 5 MB).
MAX_MB=8, 3 × 3.7 MB saves	conv-1 + conv-2 saved silent; conv-3 triggers evicted conv-1.bin (oldest LRU). Final dir = conv-2 + conv-3, total 7.3 MB ≤ 8 MB ✓
old.tmp aged 5 min, default 60 s threshold	cleaned stale tmp .../old.tmp (age 302s) then file gone from disk ✓

Save latency overhead ~5–10 ms (scandir + stat + optional unlink). Within the day-to-day eMMC jitter we already see on the save path.

Throughput / TTFT vs alpha.9

	alpha.9	alpha.10	Δ
Cold prefill	38.7 tok/s	38.7 tok/s	unchanged
Decode	10.0 tok/s	10.0 tok/s	unchanged
Warm-turn TTFT (24 / 36 hydrate)	444 ms	444 ms	unchanged
Cache dir growth	unbounded	capped at 1 GB (default)	bounded ✓
Stale .tmp from crashed saves	accumulate	cleaned (default 60 s)	recovered ✓

Path F series complete

PR	Phase	What
#46	F1	Serialization round-trip test
#47	F2	--conv-id surface (CLI + HTTP)
#48	F3	Save on turn end (full pool, replaced by F3b)
#49	F3b	Pack only used_tokens (34× save speedup)
#50	F4a	Format v2: persist token IDs + off-by-one fix
#51	F4b	Hydrate on turn start + longest-common-prefix
#53	F5	LRU eviction + size cap + stale-tmp cleanup

Path F plan: #45.

Cumulative since alpha.2

	alpha.2	alpha.10	Δ
Cold prefill	8.2 tok/s	38.7 tok/s	+372 %
Decode	7.5 tok/s	10.0 tok/s	+33 %
Cold TTFT	2200 ms	~860 ms	−61 %
Warm-turn TTFT	n/a	~444 ms	new (Path F)
Cache safety	n/a	1 GB cap + LRU + .tmp cleanup	new (Path F5)

Not in this release

• Multi-process flock around scandir/eviction. Single-process today.
• Cache index file (scandir-on-every-save is < 10 ms at our sizes).
• Cosmetic prefill-log fix (post-hydrate Prefill: N tokens still counts hydrated tokens in N). Will pair with the next focused fix.

v0.1.0-alpha.5 — Path C: Q4_K uint32 decode weight loads

Stacked on top of v0.1.0-alpha.3's Path B batched prefill. Validated on Jetson Orin Nano Super 8 GB, Qwen3-4B Q4_K_M, 25 W MAXN SUPER, GPU locked at 918 MHz.

Headline

	alpha.3	alpha.5	Δ
Decode	7.5 tok/s	9.1 tok/s	+21%
Prefill	15.4 tok/s	15.2 tok/s	unchanged
TTFT	1181 ms	~1180 ms	unchanged
Output	reference	bit-identical	✓

Cumulative from the alpha.2 byte-path baseline: prefill +85%, decode +21%, TTFT −46%. Output bit-identical at every merged step.

What shipped — Path C (Q4_K uint32 weight loads)

Each lane reads four packed q-bytes as one `uint32_t` from `blk.qs` instead of one byte. Eliminates the byte-by-byte inner loop on the hot decode kernels. Same math, fewer issued load instructions, constant-per-lane scales. Output bit-identical to the byte path at FP16.

PR	Kernel	Cumulative decode
#25	Wo (residual-fused)	8.0 tok/s
#26	gate/up pair	8.7 tok/s
#27	QKV triple (Qwen3-4B: Q4_K + Q4_K + Q6_K)	8.9 tok/s
#28	default-on (`JLLM_Q4K_UINT32_LOADS`)	9.1 tok/s

`JLLM_Q4K_UINT32_LOADS=0` opts back into the byte path. Same env-var grammar as `JLLM_BATCHED_PREFILL` (unset = enabled; `=0` = disabled).

Also folded in (direct-to-main between alpha.3 and Path C)

• Device-resident layer weight arena — weights copied into a per-layer device allocation at load time instead of streaming from mmap'd host memory.
• Decode GEMV residual-add fusion — the residual `vec_add` is now inside `gemv_quant_add_typed_kernel`. One fewer launch per layer per token.
• `gemv_rows_per_block` default lowered from 8 to 4 for the decode shapes.

Closed without merge (negative results, documented inline)

• #23 — CUDA Graphs decode capture. Launches were already overlapping with GPU execution on the LPDDR5-bound workload; eliminating host-side overhead bought ≤0%.
• #24 — Split-K Q4_K GEMV. SPLIT_K=4 grew block size 4× and register pressure dropped blocks-per-SM 4×, so warps-per-SM was unchanged. Bottleneck is arithmetic intensity per byte loaded, not per-warp MSHR depth.
• #29 — Q6_K uint32 weight loads. `block_q6_K` is 210 B (not divisible by 4); consecutive blocks alternate between 4-aligned and 2-aligned positions, half the uint32 loads fault with `cudaErrorMisalignedAddress`.

Full diagnoses on the closed PRs and on the Path C umbrella issue (#19).

Known limits

• Q6_K decode kernels (W_down ~8% of decode, output logits ~7%) are still on the byte-by-byte path. The 210-byte block size rules out uint32 vectorization without breaking GGUF compatibility. `uint16` would yield ~half the Q4_K win on ~half the decode share — deferred until a fresh `nsys` profile justifies the work.
• Decode `cudaStreamSynchronize` calls, sampling cost, and attention/RoPE overhead haven't been re-profiled since Path C landed. The pre-Path-C profile showing "93% of decode in five K-quant GEMV kernels" is now stale (four of those kernels are vectorized).

Docs

CHANGELOG.md · README.md · docs/performance.md · docs/kernels.md