Skip to content

Ferrite 0.4.0

Choose a tag to compare

@BHCC2025 BHCC2025 released this 11 Nov 19:44
· 7 commits to main since this release

Ferrite 0.4.0 lets the Intel Arc Pro B70 engine serve up to four requests at once.

In this release

  • Up to four requests at once on the Arc Pro B70 (FERRITE_SLOTS=4 with scripts/serve-xe.sh, from source). Each request gets a lane with its own conversation state, and /health reports max_batch_size 4.
  • One verify window for all lanes. Lanes that decode together share a window of up to 16 rows, so the weights are read once for all of them: four lanes total about 1.4x the speed of one request (126 vs 90 tokens per second with a 2K shared prefix, 85 vs 46 with 16K).
  • Shared prompt start reused. A prompt that starts like another lane's reuses that part in whole 4096-token prompt windows.
  • Same text as alone. Every lane's output is what the same request gives when sent alone, with greedy decoding and with seeded sampling.
  • The default is unchanged: one request at a time with 128K context. In lanes mode the context is split into one lane of 81,920 tokens and three of 28,672.

Details and the full table: docs/arc-b70.md. Full list: CHANGELOG.md.

Install on a DGX Spark (one line):

curl -fsSL https://raw.githubusercontent.com/BHCC2025/ferrite/main/scripts/install.sh | bash

Assets

  • ferrite-0.4.0-dgx-spark-aarch64.tar.gz: prebuilt binaries for DGX Spark (aarch64, CUDA 13, kernels for sm_121a) plus scripts and docs. No Rust toolchain needed. The DGX Spark engine itself is unchanged from 0.3.0. The RTX 5080 and Arc Pro B70 targets are built from source.
  • ferrite-0.4.0-dgx-spark-aarch64.tar.gz.sha256: its checksum (sha256sum -c).

Credits: CREDITS.md.