Skip to content

Ferrite 0.4.1

Choose a tag to compare

@BHCC2025 BHCC2025 released this 12 Nov 01:04
· 6 commits to main since this release

Ferrite 0.4.1 gives the model a name that makes sense on every machine and fixes the 16 GB preset when the NVIDIA prompt checkpoint is not there.

In this release

  • The default model id is qwen3.8-27b on every target. It was spark on the CUDA engine and b70 on the Intel Arc Pro B70 engine. /v1/models, /health and the replies report the new id. Requests may still name any model, so existing clients keep working; FERRITE_MODEL_ID sets another id (for example FERRITE_MODEL_ID=spark to keep the old one).
  • RTX 5080 preset without the NVIDIA checkpoint. PROMPT=bf16 (also the fallback when the checkpoint is missing) ran out of device memory while loading. On small-memory devices the prompt now goes through the EXL3 weights on the decode kernels.
  • RTX 5060 Ti 16 GB results. The RTX 5080 build runs on an RTX 5060 Ti 16 GB: about 140 tokens per second on code and 344 on a 500-token code edit, with the prompt chunk size that suits its 8-lane link.

Details: docs/rtx-5080.md. Full list: CHANGELOG.md.

Install on a DGX Spark (one line):

curl -fsSL https://raw.githubusercontent.com/BHCC2025/ferrite/main/scripts/install.sh | bash

Assets

  • ferrite-0.4.1-dgx-spark-aarch64.tar.gz: prebuilt binaries for DGX Spark (aarch64, CUDA 13, kernels for sm_121a) plus scripts and docs. No Rust toolchain needed. The RTX 5080 and Arc Pro B70 targets are built from source.
  • ferrite-0.4.1-dgx-spark-aarch64.tar.gz.sha256: its checksum (sha256sum -c).