Skip to content

v0.1.20 — engine modularization

Choose a tag to compare

@Kaden-Schutt Kaden-Schutt released this 05 May 05:38
· 4121 commits to master since this release

Major architectural refactor. The monolithic `engine` crate has split
into a `hipfire-runtime` crate plus per-arch crates. New `Architecture`
trait makes adding model archs a clean bring-up workflow. Performance
preserved within ±2% across all hosts; decode is bit-identical at 9B
on gfx1151 (Strix Halo).

Discord: https://discord.gg/F3BaywB8Rs for real-time chat,
contributor coordination, rebase help.

Highlights

  • engine crate → hipfire-runtime (new name, same role + four new
    modules)
  • Per-arch crates: `hipfire-arch-qwen35`, `hipfire-arch-qwen35-vl`,
    `hipfire-arch-llama`, `hipfire-arch-toy` (template for new arches)
  • `Architecture` trait scaffold in
    `crates/hipfire-runtime/src/arch.rs` — bring-up spec for new model arches
  • Generation guards extracted into reusable modules: `loop_guard`
    (n-gram detector), `sampler` (top-p / repeat-penalty / blocked-tokens),
    `prompt_frame` (ChatML), `eos_filter` (output stream)
  • Bare examples gain `--guards on/off` flag — `infer_qwen35` /
    `infer_qwen3` can now opt into production-quality generation hygiene
    while preserving bare-engine probe semantics by default
  • Contributor onboarding: CONTRIBUTING.md crate-topology section,
    CHANGELOG migration map, PR template, `hipfire-arch-toy` reference
    crate as a copy-paste starting point for new arches
  • Rebase helper: `scripts/rebase-onto-modular.sh` +
    `.skills/rebase-onto-modular/SKILL.md` mechanically port pre-modular
    branches to the new topology

Breaking — for contributors with in-flight branches

If you have an open PR or local branch authored against pre-0.1.20
master, you'll see import / path errors after rebase. The new tooling
handles ~80% mechanically:

```bash
git checkout your-feature-branch
./scripts/rebase-onto-modular.sh
```

The script creates a backup tag, rebases onto master, applies the
path-rename + import-rewrite map, and reports any remaining manual
fixes. See issue #155
for the full migration guide and common-conflict table.

Quick old → new path map

Old New
`crates/engine/src/qwen35.rs` `crates/hipfire-arch-qwen35/src/qwen35.rs`
`crates/engine/src/qwen35_vl.rs` `crates/hipfire-arch-qwen35-vl/src/qwen35_vl.rs`
`crates/engine/src/llama.rs` `crates/hipfire-runtime/src/llama.rs` (facade)
`crates/engine/src/speculative.rs` `crates/hipfire-arch-qwen35/src/speculative.rs`
`crates/engine/src/pflash.rs` `crates/hipfire-arch-qwen35/src/pflash.rs`
`use engine::*` `use hipfire_runtime::*` (or arch crate)
`cargo build -p engine` `cargo build -p hipfire-runtime`

Full map in CHANGELOG.md.

What didn't change

  • Kernel files (`kernels/src/*.hip`) — unchanged
  • `rdna-compute/dispatch.rs` per-RDNA-arch routing — unchanged
  • `hip-bridge` HIP/ROCm FFI — unchanged
  • `hipfire-quantize` CLI — unchanged
  • Daemon JSON-line API surface — unchanged
  • All 45 `HIPFIRE_*` environment variables — preserved bit-for-bit
  • Locked speed-gate baselines — perf parity verified on gfx1100 + gfx1151

Performance verification (canonical speed-gate config)

`HIPFIRE_KV_MODE=asym3 HIPFIRE_DPM_WARMUP_SECS=3 HIPFIRE_GRAPH=1` against
`qwen3.5-9b.mq4` on freshly purged + recompiled kernel cache:

Host Metric Master Integration Δ
gfx1100 (7900 XTX) gen_tok_s 123.1 124.5 +1.1%
gfx1100 (7900 XTX) prefill_tok_s 1809 1821 +0.6%
gfx1151 (Strix Halo) gen_tok_s 45.5 45.5 0.0%
gfx1151 (Strix Halo) bw_gib_s 225.1 225.3 +0.1%

Kernel hash sum (48 .hash files, gfx1151, fresh recompile): byte-identical
at `09bc045fe3db6aa81f87abce03cd1be6ec528e9666b08e457d2f7585979646e6`
across master and integration daemons.

What's still TODO post-0.1.20

  • Transformer-extraction PR — pulls cross-arch primitives
    (`weight_gemv`, `KvCache`, dequant helpers, RoPE) out of `llama.rs`
    into `hipfire_runtime::transformer::*`. Unblocks the physical
    llama-arch crate split (currently a facade).
  • Gemma branch forward-port — `gemma4` lives on its own branch with
    pre-modular daemon hooks. Forward-port happens after transformer
    extraction lands.
  • `docs/architecture-ids.md` — arch_id registry to prevent
    collisions across community contributions.

Adding a new model architecture

`crates/hipfire-arch-toy/` is a minimal stub arch (~150 lines, 4 doc-
heavy files) demonstrating `Architecture` trait impl. Copy as starting
point for a new arch; see CONTRIBUTING.md "Crate topology" + decision
tree. RDNA GPU arch ports remain a separate concern handled by
`.skills/hipfire-arch-port/` (kernels live in `kernels/src/`,
dispatch in `rdna-compute`).

Issues / PRs

  • Pinned migration guide: #155
  • PRs in flight that may need rebase: #147 (UMA loader/MQ6),
    #148 (OpenAI streaming stats), #153 (weight-pager v0.2)

@Kaden-Schutt