Repository navigation
Ferrite 0.4.0
Ferrite 0.4.0 lets the Intel Arc Pro B70 engine serve up to four requests at once.
In this release
- Up to four requests at once on the Arc Pro B70 (
FERRITE_SLOTS=4withscripts/serve-xe.sh, from source). Each request gets a lane with its own conversation state, and/healthreportsmax_batch_size4. - One verify window for all lanes. Lanes that decode together share a window of up to 16 rows, so the weights are read once for all of them: four lanes total about 1.4x the speed of one request (126 vs 90 tokens per second with a 2K shared prefix, 85 vs 46 with 16K).
- Shared prompt start reused. A prompt that starts like another lane's reuses that part in whole 4096-token prompt windows.
- Same text as alone. Every lane's output is what the same request gives when sent alone, with greedy decoding and with seeded sampling.
- The default is unchanged: one request at a time with 128K context. In lanes mode the context is split into one lane of 81,920 tokens and three of 28,672.
Details and the full table: docs/arc-b70.md. Full list: CHANGELOG.md.
Install on a DGX Spark (one line):
curl -fsSL https://raw.githubusercontent.com/BHCC2025/ferrite/main/scripts/install.sh | bashAssets
ferrite-0.4.0-dgx-spark-aarch64.tar.gz: prebuilt binaries for DGX Spark (aarch64, CUDA 13, kernels for sm_121a) plus scripts and docs. No Rust toolchain needed. The DGX Spark engine itself is unchanged from 0.3.0. The RTX 5080 and Arc Pro B70 targets are built from source.ferrite-0.4.0-dgx-spark-aarch64.tar.gz.sha256: its checksum (sha256sum -c).
Credits: CREDITS.md.