Skip to content

v0.1.7-alpha: FlashTriAttn + CASK + A3B family + MI300X wave64

Choose a tag to compare

View all tags
@Kaden-Schutt Kaden-Schutt released this 18 Apr 03:59
· 4121 commits to master since this release
v0.1.7-alpha
1e5ada6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

v0.1.7-alpha: FlashTriAttn + CASK + A3B family + MI300X wave64

Pre-release tag. Gated to full v0.1.7 on the outcome of the Hermes-agent
stack validation currently running on MI300X (task #125).

Highlights

  • FlashTriAttn long-ctx wins shipped. DFlash speculative decode + TriAttention
    KV eviction composes cleanly. Measured on 7900 XTX / 9B MQ4 / ~1500-token
    prompt / 200-token decode / --cask-budget 512 --cask-beta 128:

    config tok/s τ
    DFlash baseline (full KV) 150 5.31
    FlashTriAttn (new 1M sidecar) 214 (+42%) 5.36 (−0.0)
    FlashCASK (m-folding, CPU merge) 124 2.83

    Earlier sidecars (small corpora) lost ~27% τ — that's gone now that cals are
    1M+ tokens. The FlashCASK τ drop is CPU-merge smoothing; GPU merge kernel
    (task #82) lands in 0.1.7 stable.

  • Qwen3.5-35B-A3B and Qwen3.6-35B-A3B MoE end-to-end in DFlash. Batched
    MoE prefill + fused sigmoid+residual GEMV + indexed expert dispatch.

  • MI300X (gfx942) wave64 port. 10 hot HFQ4 kernels re-written for
    block=[64,1,1] 2-rows-per-block. A3B decode 48.6 → 96 tok/s on MI300X
    (matches 7900 XTX baseline despite the 4× BW gap).

  • Batched-prefill TriAttention tap — 4.5–5× faster sidecar calibrations.
    What made 1M-token cals across 5 targets feasible on one MI300X overnight.

  • DFlash tape-replay rollback for multi-turn state recovery after verify
    mismatches without a full target re-run.

Bench snapshot (7900 XTX, MQ4)

DFlash τ + tok/s per prompt class (ctx=4K, no CASK):

model short code math
4B 53 tok/s τ=1.27 92 tok/s τ=2.49 148 tok/s τ=6.0
9B 112 tok/s τ=1.52 461 tok/s τ=9.95 288 tok/s τ=5.77
27B 20 tok/s τ=2.21 41 tok/s τ=5.66 42 tok/s τ=6.14

5-model sidecar calibration (1M tokens wikitext):

model mean r̄ notes
4B 0.564 dense
9B 0.629 dense, best generalization (tested 0.672 on code prompt)
27B 0.542 dense
3.5-A3B 0.552 MoE
3.6-A3B 0.552 MoE

Paper Figure 3 target is r̄ ≈ 0.5 — all five above it.

CLI + daemon config

Per-model config via hipfire config set <key> <value> or
~/.hipfire/per_model_config.json:

dflash_adaptive_b   boolean   default true
cask_sidecar        string    default ""        # path to .triattn.bin
cask                boolean   default false     # m-folding on top of sidecar
cask_budget         int       default 512
cask_beta           int       default 128
cask_core_frac      float     default 0.5
cask_fold_m         int       default 2

Daemon load message accepts these in params. cask_sidecar is accepted
and logged in alpha; the serve-time generate-loop integration lands in 0.1.7
stable. For now, use dflash_spec_demo directly for the --cask-sidecar path.

Pending for v0.1.7 stable

  • Wire cask_sidecar + adaptive-B through the daemon generate loop.
  • Hermes agent validation outcome (task #125) — gates stable release.
  • GPU-side CASK merge kernel (task #82) for net-positive FlashCASK.
  • DDTree integration into CLI/daemon (currently τ-positive, not yet tok/s-positive
    without hipGraph coverage).

Test plan

  • Workspace cargo check clean on dflash branch
  • cargo build --release --features deltanet --example daemon succeeds
  • Local 7900 XTX bench sweep (scripts/dflash_branch_bench.sh) — all
    targets complete, τ + tok/s recorded
  • 5 sidecars calibrated + rsynced back from MI300X
  • triattn_validate --load-sidecar validates all 5 sidecars
  • FlashTriAttn +42% speedup reproduced on 9B long-ctx
  • Hermes agent + hipfire-daemon stack validation on MI300X (task #125)
  • Quality gate sweep — NOTE: 0.1.6 baselines diverged from dflash output
    (known since branch start); user flagged as "fossil, ignore" 2026-04-17

🤖 Generated with Claude Code

Assets 2
Loading