Skip to content

v0.1.7-alpha.3 — Qwen3.6-A3B fix + Phase 2 KV + coherence gate

Choose a tag to compare

View all tags
@Kaden-Schutt Kaden-Schutt released this 19 Apr 04:48
· 4121 commits to master since this release
v0.1.7-alpha.3
a60eda1
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Bundles 100 commits since v0.1.7-alpha.2. Points at merge commit `a60eda1` (PR #27).

Highlights

Qwen3.6-A3B no longer spirals — final-norm convention fix

Commit `1e01c0b`. The final `output_norm` on `qwen3_5_moe` (A3B, arch_id=6) is stored as a raw RMSNorm scale (mean ~+1.6), NOT as deviation-from-0 like the per-block norms. Our `load_norm_weight` unconditionally added 1.0, over-amplifying pre-lm_head activations by ~60% and tipping 3.6-MQ4 into infinite `` spirals on reasoning prompts. 3.5-A3B tolerated the over-amplification but was also subtly wrong.

Fix gated on `config.num_experts > 0`, so dense Qwen3.5 0.8B/4B/9B are untouched and byte-exact compatible with prior baselines.

Validated end-to-end: hermes-agent → SSH tunnel → MI300X serve → 3.6-A3B thinking:on closes `` and delivers answers instead of spiraling.

Qwen3.6:35b-a3b support (`cf3031f`)

Drop-in — same arch_id=6 as 3.5-A3B. Added REGISTRY entry + `qwen3.6` / `qwen3.6:a3b` aliases.

Phase 2 KV: physical_cap decoupled from max_seq

Commits `422fbf6`, `8f9c972`, `63ad050`. The advertised context window (`max_seq`) no longer forces VRAM allocation at `max_seq` × KV-bytes — the physical buffer is sized to `cask_budget + β + 256` and rolled via eviction. A3B can now advertise 4M+ tokens of context at ~22 GB VRAM on 24 GB cards.

CASK + DFlash eviction integration

Commits `1fbdcfa`, `adfe836`. DFlash speculative decoding now composes with TriAttention KV eviction. Downgrade guard: `cask=true` + `draft` logs a warning and falls back to plain TriAttention (m-fold + draft spec currently interacts pathologically).

rocBLAS MFMA path (task #130, CDNA3)

MFMA-accelerated prefill GEMMs on MI300X gfx942 (4.46× vs hand-rolled GEMV on A3B prefill). Kill-switch `HIPFIRE_ROCBLAS_OFF=1` for A/B benching or workaround. `HIPFIRE_ROCBLAS_ALL_ARCHS=1` opens the path to RDNA3 for smoke testing. Default-gated to CDNA3 arches.

Note: there's a separate rocBLAS + eviction prompt-KV corruption bug on MI300X (27B panics with `rocblas_gemm_ex hipError=6`, A3B produces `"The The The"` garbage under CASK). Worked around with `HIPFIRE_ROCBLAS_OFF=1` baked into the MI300X serve launcher. Real fix pending.

Coherence battery replaces byte-exact quality gate

New `scripts/coherence-gate.sh` runs a small (model × prompt) matrix through the daemon and writes a markdown report. Fails only on hard errors (daemon panic / zero tokens / timeout). Byte-exact `quality-gate.sh` (still available for manual use) was blocking legitimate numerical-correctness fixes (like this release's final-norm fix); the pre-commit hook now uses coherence. Committer (human or agent) reads the report and confirms coherence before the commit lands.

Smaller fixes

  • `19f2c7b` — default A3B DFlash off (regression protection for reasoning tasks).
  • `30022f0` — config TUI crash on missing meta for new 0.1.7 keys.

Upgrade

`hipfire update` pulls `origin/master` directly — no release-tag tracking — so this release is live for any user who runs it. Release tags are for GitHub discoverability.

Known issues (not fixed by this release)

  • rocBLAS + eviction corruption on MI300X — workaround `HIPFIRE_ROCBLAS_OFF=1`.
  • Local phrase repetition inside `` on 3.6-A3B (engine spiral is fixed; model still sometimes repeats a sentence 3–4× inside think block, which can demote the correct answer token via repeat_penalty → off-by-one final answer). Requires porting `apply_ngram_block` into the GPU `sample_top_p` kernel with scope reset at ``.

Cargo version note

`Cargo.toml` still reads `0.1.7-alpha` in this release's tree. Version string bump to match `0.1.7-alpha.3` deferred to the next commit. Does not affect runtime behavior or `hipfire update`.

🤖 Generated with Claude Code

Assets 2
Loading