Repository navigation
Ferrite 0.4.1
Ferrite 0.4.1 gives the model a name that makes sense on every machine and fixes the 16 GB preset when the NVIDIA prompt checkpoint is not there.
In this release
- The default model id is
qwen3.8-27bon every target. It wassparkon the CUDA engine andb70on the Intel Arc Pro B70 engine./v1/models,/healthand the replies report the new id. Requests may still name any model, so existing clients keep working;FERRITE_MODEL_IDsets another id (for exampleFERRITE_MODEL_ID=sparkto keep the old one). - RTX 5080 preset without the NVIDIA checkpoint.
PROMPT=bf16(also the fallback when the checkpoint is missing) ran out of device memory while loading. On small-memory devices the prompt now goes through the EXL3 weights on the decode kernels. - RTX 5060 Ti 16 GB results. The RTX 5080 build runs on an RTX 5060 Ti 16 GB: about 140 tokens per second on code and 344 on a 500-token code edit, with the prompt chunk size that suits its 8-lane link.
Details: docs/rtx-5080.md. Full list: CHANGELOG.md.
Install on a DGX Spark (one line):
curl -fsSL https://raw.githubusercontent.com/BHCC2025/ferrite/main/scripts/install.sh | bashAssets
ferrite-0.4.1-dgx-spark-aarch64.tar.gz: prebuilt binaries for DGX Spark (aarch64, CUDA 13, kernels for sm_121a) plus scripts and docs. No Rust toolchain needed. The RTX 5080 and Arc Pro B70 targets are built from source.ferrite-0.4.1-dgx-spark-aarch64.tar.gz.sha256: its checksum (sha256sum -c).