Skip to content

Valen custom GGUF CPU runtime 0.2.0

Choose a tag to compare

@Liuziyu77 Liuziyu77 released this 10 Nov 13:22

Prebuilt Valen-patched llama-server 0.2.0 for Linux x86-64 CPU, Ubuntu 22.04+/glibc 2.35+, AVX2/FMA/F16C/BMI2. Includes native libraries, the exact patched source, base pins, patch hashes and MIT/Apache-2.0 notices. Model weights are downloaded separately; no compiler is required.

Runtime 0.2.0 serves Valen-Decision-1.5: block attention, noul-rubric folding, text and image input, and bbox questions on image requests through /v1/systemone and /v1/decisions. Valen-Decision-1.0 GGUF files released for runtime 0.1.0 load and serve with this runtime too. Valen-Decision-1.5 GGUF files (F32, Q8_0 and the vision file) are in the Valen-Team/Valen-Decision-1.5-<size>-GGUF repositories on Hugging Face.

Measured on Valen-Decision-1.5-0.8B, Linux x86-64 CPU, against the native model: F32 probabilities within 0.00005 on the text and image requests and boxes within 0.005; Q8_0 probabilities within 0.021 on the text request. validation.json in the archive lists every request. The final archive passed its internal SHA256 checks after extraction and served a Valen-Decision-1.5-0.8B Q8_0 text request (repeated answers matched), a bbox request on an image, and a Valen-0.8B Q8_0 text request; extracted-validation.json records the answers.

The 0.1.0 to 0.2.0 source change is source/valen-decision-1.5-runtime.patch in the archive (17 files, SHA256 857d670488e0c796b4346b893353cc0d7a5b588cf0d5393c88d58d939f843bb8); source/llama-source.tar.gz is the complete patched tree.

Download and verify SHA256SUMS before extraction. English: https://valen-team.github.io/docs/valen/gguf/ ; 中文: https://valen-team.github.io/zh-Hans/docs/valen/gguf/