Repository navigation
v0.2.0
v0.2.0 - 2026-05-25
Minor release for the GGUF runtime path and W7900 benchmark refresh. GGUF is a
meaningful new model-loading surface rather than a patch-level fix, so this
supersedes the previously planned v0.1.2 patch.
Added
- Added Qwen3.6 35B MoE GGUF support for
Q4_K_MandQ4_K_Smodel files,
including resident GGUF loading, bulk prefill, graph-replay decode,
decode-repacked T16 layouts, and WMMA/GEMV fast-path controls used by the
W7900 benchmark profile. - Added
docs/ENVS.mdas the canonical environment-variable reference, including
TheRock ROCm process setup, cached-build profiling guidance, and safe GGUF
benchmark profiles. - Added a persistent README sweep harness that loads each hipEngine model once
and runs repeated in-session workload measurements, matching llama-bench-style
repetition without multiplying model load/decode-repack time by every shape.
Changed
- Refreshed W7900 README performance tables with 5-run persistent-session medians
for packed PARO and GGUF Q4_K_S while keeping the existing llama.cpp HIP/Vulkan
comparison rows unchanged. - Documented the current GGUF tradeoffs: higher one-time load cost and resident
memory from decode-repack, Q4_K_S preferred for tighter VRAM budgets, and
performance still behind PARO on some shapes while already competitive in the
broader W7900 comparison.
Fixed
- Fixed the PARO resident prefill workspace-overlap regression that shipped in
v0.1.1: short and mid prompts now keep prefill workspaces resident through
32K tokens, restoring 512/128-class prefill throughput while retaining the
long-context memory-saving path for prompts above 32K when active chunking
splits the prompt. - Fixed GGUF non-split full-attention decode in max-context persistent sessions
by launching the context kernel with the active decode context instead of the
session's maximum allocation length.
Known limitations
- GGUF support remains alpha: production correctness and performance coverage is
strongest for the documented Qwen3.6 35B MoE Q4_K_M/Q4_K_S paths on gfx1100,
and other GGUF quants/models require local validation. - GGUF model load is slower than packed PARO on the same host because current
decode-repack happens on load and is not yet cached on disk.