Repository navigation
v0.8.0
EULLM Engine EuLLM-v0.8.0
LLM inference engine in Rust: one binary, any GGUF model, on CPU or GPU. Continuous batching, quantized KV cache, zero telemetry, and an AI Act-ready audit trail.
This release: llama.cpp b11925, and other programs' model folders used as they are
llama.cpp moves to b11925, 443 commits newer. On an RTX 5070 Ti, reading a prompt is 5-16% faster and writing level or up to 4% faster, on Qwen3.6-35B-A3B, Qwen3.8-Flash-Next and Qwen3.5-9B; the expert cache finds as many experts in VRAM as before. Eight more architectures load. With --mtp, Qwen3.5-9B keeps about 5 points fewer of its drafts and writes 2-3.5% slower.
Model libraries. EULLM_MODEL_LIBRARIES (in the environment or .env, folders separated by ;) names other programs' model folders, such as LM Studio's or a Hugging Face cache: every .gguf in them, through every subfolder, is listed and runs by name, and nothing there is moved, renamed or deleted. Projectors go with their model instead of being listed as models. eullm import no longer moves files.
Quick install
# Linux / macOS — one line, picks the right build, installs to ~/.local/bin
curl -fsSL https://raw.githubusercontent.com/eullm/eullm/main/installer/install.sh | sh
# Linux x64 (CPU only)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64 -o eullm
chmod +x eullm
# Linux x64 with NVIDIA GPU (CUDA 13.1 — supports RTX 3000/4000/5000 series)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-cuda-13.1 -o eullm
chmod +x eullm
# Linux x64 with an NVIDIA DATA-CENTER GPU (A100, H100) — a different
# build, on CUDA 12.4, because these cards are a different compute
# capability and their drivers are usually older. The consumer build
# above does not run on them.
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-cuda-12.4-datacenter -o eullm
chmod +x eullm
# Linux x64 with an AMD Radeon (ROCm 7.2 — RX 7000/9000, Ryzen AI iGPUs)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-rocm-consumer -o eullm
chmod +x eullm
# Linux x64 with an AMD Instinct MI250X/MI210 (ROCm 6.3)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-rocm-gfx90a -o eullm
chmod +x eullm
# macOS Apple Silicon
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-macos-arm64 -o eullm
chmod +x eullm# Windows — one line, picks CPU, CUDA or Vulkan, adds eullm to PATH
irm https://raw.githubusercontent.com/eullm/eullm/main/installer/install.ps1 | iex
# Windows x64 (CPU only) — ZIP: eullm.exe + the Visual C++ runtime,
# so it also runs on a Windows without the VC++ Redistributable
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64.zip -OutFile eullm.zip
Expand-Archive eullm.zip -DestinationPath eullm
.\eullm\eullm.exe run model.gguf
# Windows x64 with NVIDIA GPU (CUDA 13.1 — supports RTX 3000/4000/5000)
# This is a ZIP: eullm.exe + the CUDA and Visual C++ runtime DLLs it needs.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-cuda-13.1.zip -OutFile eullm-cuda.zip
Expand-Archive eullm-cuda.zip -DestinationPath eullm-cuda
# Run eullm.exe from inside the extracted folder (the DLLs must
# stay next to it).
.\eullm-cuda\eullm.exe run model.gguf
# Windows x64 with an AMD or Intel GPU, or an NVIDIA card the CUDA
# build does not cover (Vulkan). A ZIP: eullm.exe + the Visual C++
# runtime; the Vulkan loader comes with the graphics driver.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-vulkan.zip -OutFile eullm-vulkan.zip
Expand-Archive eullm-vulkan.zip -DestinationPath eullm-vulkan
.\eullm-vulkan\eullm.exe run model.gguf
# Windows x64 with an AMD Radeon RX 7900 XTX/XT, RX 9070 XT/9070 or
# Radeon AI PRO R9700, through ROCm 10.1. EXPERIMENTAL: not yet run
# on a Radeon. Needs the "AMD Software: Adrenalin Edition for ROCm
# 10.1" driver. ROCm's own layout: eullm.exe is in bin\, and the
# .kpack\ folder must stay beside it.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-rocm.zip -OutFile eullm-rocm.zip
Expand-Archive eullm-rocm.zip -DestinationPath eullm-rocm
.\eullm-rocm\bin\eullm.exe run model.ggufWhich binary to download?
| Binary | GPU Support | Requirements |
|---|---|---|
eullm-linux-x64 |
CPU only | glibc 2.28+ — built on Rocky Linux 8, so it runs on RHEL/CentOS/Rocky/Alma/Oracle 8+ as well as current Ubuntu/Fedora/Arch |
eullm-linux-x64-vulkan |
Any GPU with a Vulkan driver (AMD, Intel, NVIDIA) | libvulkan.so.1 and a working driver on the machine — mesa RADV, amdvlk, NVIDIA or Intel ANV. Nothing is bundled |
eullm-linux-x64-cuda-13.1 |
NVIDIA consumer GPU (RTX 3000/4000/5000, sm_86/89/120) | NVIDIA driver 580+. Does not run on A100/H100 — those need the data-center build below |
eullm-linux-x64-cuda-12.4-datacenter |
NVIDIA data-center GPU (A100 sm_80, H100 sm_90) | NVIDIA driver 550+. Built on CUDA 12.4, not 13.1, on purpose: these cards live in HPC centres and enterprise fleets that freeze drivers for years, and CUDA 13 needs an August-2025 driver. Does not run on RTX cards |
eullm-linux-x64-rocm-consumer |
AMD Radeon: RDNA 3 (RX 7900/7800/7700/7600), RDNA 3.5 (Ryzen AI integrated Radeons), RDNA 4 (RX 9000) | ROCm 7.2.x installed on the machine — nothing is bundled, the binary links the system's own libamdhip64/rocblas/hipblas. RX 6000 (RDNA 2) is not included: use the Vulkan binary there |
eullm-linux-x64-rocm-gfx90a |
AMD Instinct MI250X / MI210 (CDNA 2) | ROCm 6.3.x installed on the machine, the version EuroHPC sites run. Check hipconfig --version before assuming it fits yours |
eullm-linux-arm64 |
CPU only | ARM64 Linux — generic baseline, runs on Raspberry Pi, Graviton, Jetson, any ARM64 SoC |
eullm-linux-arm64-cuda-13.1 |
NVIDIA GPU on an ARM64 host (sm_86/89/120) | ARM64 Linux + NVIDIA driver 580+. For a discrete NVIDIA card in an ARM board's PCIe slot |
eullm-linux-arm64-cix-p1 |
CPU only | Armv9.2-A with SVE2, BF16, I8MM and DotProd only — a tuned CPU profile for the CIX P1 (Radxa Orion O6). These instructions are compiled in, so it crashes with SIGILL on any ARM64 host without them, Raspberry Pi included. If you are not on a CIX P1, take eullm-linux-arm64 |
eullm-macos-x64 |
CPU only | macOS Intel |
eullm-macos-arm64 |
Metal (Apple GPU) | macOS Apple Silicon |
eullm-windows-x64.zip |
CPU only | Windows 10/11 x64. ZIP bundles eullm.exe + the Visual C++ runtime DLLs — run from the extracted folder |
eullm-windows-x64.exe |
CPU only | Windows 10/11 x64 with the Visual C++ Redistributable already installed (otherwise it fails with "MSVCP140.dll was not found" — take the ZIP) |
eullm-windows-x64-cuda-13.1.zip |
NVIDIA GPU (RTX 3000/4000/5000) | NVIDIA driver 580+ for Windows. ZIP bundles eullm.exe + CUDA and Visual C++ runtime DLLs — run from the extracted folder |
eullm-windows-x64-vulkan.zip |
Any GPU with a Vulkan driver: AMD Radeon (cards and Ryzen integrated graphics), Intel Arc, and NVIDIA cards the CUDA build does not cover (GTX 1000, RTX 2000) | Windows 10/11 x64 and a current graphics driver, which installs the Vulkan loader. ZIP bundles eullm.exe + the Visual C++ runtime DLLs — run from the extracted folder |
eullm-windows-x64-rocm.zip |
AMD Radeon through ROCm 10.1: RX 7900 XTX/XT (gfx1100), RX 9070 XT/9070 and Radeon AI PRO R9700 (gfx1201). Experimental: not yet run on a Radeon | Windows 11 x64 and the AMD Software: Adrenalin Edition for ROCm 10.1 driver (26.10.41.05) or newer. ZIP bundles the HIP runtime, rocBLAS and hipBLASLt with the kernels of those cards, and the Visual C++ runtime, in ROCm's own layout: run bin\eullm.exe and keep .kpack\ beside bin\. Not installed by install.ps1 yet; eullm update updates it from 0.7.92 on |
eullm-windows-x64-store.msix |
CPU + NVIDIA GPU | Not for direct install: the unsigned package submitted to the Microsoft Store, which signs it. Use the ZIPs or install.ps1 instead |
Pick a model interactively
# Run with no arguments to open the model picker (local models + catalog)
./eullmWeb browsing
# Enable transparent web fetch (URLs in messages are fetched and injected)
./eullm run model.gguf --webUsage
./eullm run ./model.gguf # Run any GGUF model
./eullm run ./model.gguf --batch-size 16 # Continuous batching for RAG
./eullm run ./model.gguf --web # Enable web browsingVerify checksums
sha256sum -c checksums.txtWhat changed
See CHANGELOG.md for what was added, fixed and made faster in this version.
License
EULLM is AGPL-3.0-or-later. Running an unmodified binary, or modifying it for internal or local use, carries no additional obligation. If you modify EULLM and let others use it over a network, including as a hosted service, you must offer them the source of your modified version. Versions up to and including 0.7.4 were released under Apache 2.0 and keep those terms. The models EULLM runs have their own separate licenses.
What's Changed
- Feat/model libraries by @primoco in #1172
- Feat/engine finetune by @primoco in #1173
- feat(jupiter): MTP on two engine builds side by side, one GPU each by @primoco in #1174
Full Changelog: EuLLM-v0.7.95...EuLLM-v0.8.0