Skip to content

VelociQ8 1.2

Choose a tag to compare

View all tags
@dhaern dhaern released this 22 Oct 20:14
· 6 commits to main since this release
v1.2
d2eda87

VelociQ8 1.2 is based on llama.cpp b11738, the release that Ollama 0.40.8 uses.

Changes

  • Ollama support. The velociq8-1.2-ollama-0.40.8-* archives are a portable Ollama 0.40.8 for the CPU: the official ollama executable with the VelociQ8 server. Extract and run ollama serve; nothing is installed and an Ollama already on the machine is not touched. It loads Q8A64 files and converts BF16, F16 and F32 models while loading. Steps are in docs/usage.md. Tested on Linux on Arm. The Windows archive passes the static checks but has not been run on Windows yet.
  • Q8A64 files of embedding models now carry their pooling type and context length in the file header, which Ollama needs to detect them. Older files still load in every version. The files on Hugging Face have been updated.
  • The archives include llama, the single command that the Llama app for Windows runs, and llama-tts for text-to-speech models.
  • Docker image: ghcr.io/dhaern/velociq8.
  • Q8A64 output is the same as in 1.1, checked bit for bit on the control models.

Downloads

File For
velociq8-1.2-linux-arm64.tar.gz Arm servers with dotprod: Ampere Altra, AWS Graviton2 and later, Neoverse N1 and later
velociq8-1.2-linux-arm64-multi.tar.gz Any 64-bit Arm CPU. It picks the kernels for the CPU at startup, including i8mm
velociq8-1.2-linux-x64.tar.gz x86-64 with AVX2 and FMA: Intel Haswell, AMD Zen and later
velociq8-1.2-windows-x64.zip Windows on the same x86-64 CPUs
velociq8-1.2-ollama-0.40.8-linux-arm64.tar.gz Portable Ollama 0.40.8 for the CPU, Linux on Arm
velociq8-1.2-ollama-0.40.8-linux-x64.tar.gz Portable Ollama 0.40.8 for the CPU, Linux on x86-64 with AVX2 and FMA
velociq8-1.2-ollama-0.40.8-windows-x64.zip Portable Ollama 0.40.8 for the CPU, Windows on x86-64 with AVX2 and FMA

The Linux archives need glibc 2.28 or newer. Check each download against SHA256SUMS:

sha256sum -c --ignore-missing SHA256SUMS

The binaries are free to use under the VelociQ8 license. The llama.cpp MIT license and the other third-party notices are in licenses/ inside each archive.

Assets 10
Loading