Skip to content

v0.8.0

Choose a tag to compare

@primoco primoco released this 28 Oct 18:07
· 514 commits to main since this release
d2d57a9

EULLM Engine EuLLM-v0.8.0

LLM inference engine in Rust: one binary, any GGUF model, on CPU or GPU. Continuous batching, quantized KV cache, zero telemetry, and an AI Act-ready audit trail.

This release: llama.cpp b11925, and other programs' model folders used as they are

llama.cpp moves to b11925, 443 commits newer. On an RTX 5070 Ti, reading a prompt is 5-16% faster and writing level or up to 4% faster, on Qwen3.6-35B-A3B, Qwen3.8-Flash-Next and Qwen3.5-9B; the expert cache finds as many experts in VRAM as before. Eight more architectures load. With --mtp, Qwen3.5-9B keeps about 5 points fewer of its drafts and writes 2-3.5% slower.

Model libraries. EULLM_MODEL_LIBRARIES (in the environment or .env, folders separated by ;) names other programs' model folders, such as LM Studio's or a Hugging Face cache: every .gguf in them, through every subfolder, is listed and runs by name, and nothing there is moved, renamed or deleted. Projectors go with their model instead of being listed as models. eullm import no longer moves files.

Quick install

# Linux / macOS — one line, picks the right build, installs to ~/.local/bin
curl -fsSL https://raw.githubusercontent.com/eullm/eullm/main/installer/install.sh | sh

# Linux x64 (CPU only)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64 -o eullm
chmod +x eullm

# Linux x64 with NVIDIA GPU (CUDA 13.1 — supports RTX 3000/4000/5000 series)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-cuda-13.1 -o eullm
chmod +x eullm

# Linux x64 with an NVIDIA DATA-CENTER GPU (A100, H100) — a different
# build, on CUDA 12.4, because these cards are a different compute
# capability and their drivers are usually older. The consumer build
# above does not run on them.
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-cuda-12.4-datacenter -o eullm
chmod +x eullm

# Linux x64 with an AMD Radeon (ROCm 7.2 — RX 7000/9000, Ryzen AI iGPUs)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-rocm-consumer -o eullm
chmod +x eullm

# Linux x64 with an AMD Instinct MI250X/MI210 (ROCm 6.3)
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-linux-x64-rocm-gfx90a -o eullm
chmod +x eullm

# macOS Apple Silicon
curl -L https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-macos-arm64 -o eullm
chmod +x eullm
# Windows — one line, picks CPU, CUDA or Vulkan, adds eullm to PATH
irm https://raw.githubusercontent.com/eullm/eullm/main/installer/install.ps1 | iex

# Windows x64 (CPU only) — ZIP: eullm.exe + the Visual C++ runtime,
# so it also runs on a Windows without the VC++ Redistributable
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64.zip -OutFile eullm.zip
Expand-Archive eullm.zip -DestinationPath eullm
.\eullm\eullm.exe run model.gguf

# Windows x64 with NVIDIA GPU (CUDA 13.1 — supports RTX 3000/4000/5000)
# This is a ZIP: eullm.exe + the CUDA and Visual C++ runtime DLLs it needs.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-cuda-13.1.zip -OutFile eullm-cuda.zip
Expand-Archive eullm-cuda.zip -DestinationPath eullm-cuda
# Run eullm.exe from inside the extracted folder (the DLLs must
# stay next to it).
.\eullm-cuda\eullm.exe run model.gguf

# Windows x64 with an AMD or Intel GPU, or an NVIDIA card the CUDA
# build does not cover (Vulkan). A ZIP: eullm.exe + the Visual C++
# runtime; the Vulkan loader comes with the graphics driver.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-vulkan.zip -OutFile eullm-vulkan.zip
Expand-Archive eullm-vulkan.zip -DestinationPath eullm-vulkan
.\eullm-vulkan\eullm.exe run model.gguf

# Windows x64 with an AMD Radeon RX 7900 XTX/XT, RX 9070 XT/9070 or
# Radeon AI PRO R9700, through ROCm 10.1. EXPERIMENTAL: not yet run
# on a Radeon. Needs the "AMD Software: Adrenalin Edition for ROCm
# 10.1" driver. ROCm's own layout: eullm.exe is in bin\, and the
# .kpack\ folder must stay beside it.
Invoke-WebRequest -Uri https://github.com/eullm/eullm/releases/download/EuLLM-v0.8.0/eullm-windows-x64-rocm.zip -OutFile eullm-rocm.zip
Expand-Archive eullm-rocm.zip -DestinationPath eullm-rocm
.\eullm-rocm\bin\eullm.exe run model.gguf

Which binary to download?

Binary GPU Support Requirements
eullm-linux-x64 CPU only glibc 2.28+ — built on Rocky Linux 8, so it runs on RHEL/CentOS/Rocky/Alma/Oracle 8+ as well as current Ubuntu/Fedora/Arch
eullm-linux-x64-vulkan Any GPU with a Vulkan driver (AMD, Intel, NVIDIA) libvulkan.so.1 and a working driver on the machine — mesa RADV, amdvlk, NVIDIA or Intel ANV. Nothing is bundled
eullm-linux-x64-cuda-13.1 NVIDIA consumer GPU (RTX 3000/4000/5000, sm_86/89/120) NVIDIA driver 580+. Does not run on A100/H100 — those need the data-center build below
eullm-linux-x64-cuda-12.4-datacenter NVIDIA data-center GPU (A100 sm_80, H100 sm_90) NVIDIA driver 550+. Built on CUDA 12.4, not 13.1, on purpose: these cards live in HPC centres and enterprise fleets that freeze drivers for years, and CUDA 13 needs an August-2025 driver. Does not run on RTX cards
eullm-linux-x64-rocm-consumer AMD Radeon: RDNA 3 (RX 7900/7800/7700/7600), RDNA 3.5 (Ryzen AI integrated Radeons), RDNA 4 (RX 9000) ROCm 7.2.x installed on the machine — nothing is bundled, the binary links the system's own libamdhip64/rocblas/hipblas. RX 6000 (RDNA 2) is not included: use the Vulkan binary there
eullm-linux-x64-rocm-gfx90a AMD Instinct MI250X / MI210 (CDNA 2) ROCm 6.3.x installed on the machine, the version EuroHPC sites run. Check hipconfig --version before assuming it fits yours
eullm-linux-arm64 CPU only ARM64 Linux — generic baseline, runs on Raspberry Pi, Graviton, Jetson, any ARM64 SoC
eullm-linux-arm64-cuda-13.1 NVIDIA GPU on an ARM64 host (sm_86/89/120) ARM64 Linux + NVIDIA driver 580+. For a discrete NVIDIA card in an ARM board's PCIe slot
eullm-linux-arm64-cix-p1 CPU only Armv9.2-A with SVE2, BF16, I8MM and DotProd only — a tuned CPU profile for the CIX P1 (Radxa Orion O6). These instructions are compiled in, so it crashes with SIGILL on any ARM64 host without them, Raspberry Pi included. If you are not on a CIX P1, take eullm-linux-arm64
eullm-macos-x64 CPU only macOS Intel
eullm-macos-arm64 Metal (Apple GPU) macOS Apple Silicon
eullm-windows-x64.zip CPU only Windows 10/11 x64. ZIP bundles eullm.exe + the Visual C++ runtime DLLs — run from the extracted folder
eullm-windows-x64.exe CPU only Windows 10/11 x64 with the Visual C++ Redistributable already installed (otherwise it fails with "MSVCP140.dll was not found" — take the ZIP)
eullm-windows-x64-cuda-13.1.zip NVIDIA GPU (RTX 3000/4000/5000) NVIDIA driver 580+ for Windows. ZIP bundles eullm.exe + CUDA and Visual C++ runtime DLLs — run from the extracted folder
eullm-windows-x64-vulkan.zip Any GPU with a Vulkan driver: AMD Radeon (cards and Ryzen integrated graphics), Intel Arc, and NVIDIA cards the CUDA build does not cover (GTX 1000, RTX 2000) Windows 10/11 x64 and a current graphics driver, which installs the Vulkan loader. ZIP bundles eullm.exe + the Visual C++ runtime DLLs — run from the extracted folder
eullm-windows-x64-rocm.zip AMD Radeon through ROCm 10.1: RX 7900 XTX/XT (gfx1100), RX 9070 XT/9070 and Radeon AI PRO R9700 (gfx1201). Experimental: not yet run on a Radeon Windows 11 x64 and the AMD Software: Adrenalin Edition for ROCm 10.1 driver (26.10.41.05) or newer. ZIP bundles the HIP runtime, rocBLAS and hipBLASLt with the kernels of those cards, and the Visual C++ runtime, in ROCm's own layout: run bin\eullm.exe and keep .kpack\ beside bin\. Not installed by install.ps1 yet; eullm update updates it from 0.7.92 on
eullm-windows-x64-store.msix CPU + NVIDIA GPU Not for direct install: the unsigned package submitted to the Microsoft Store, which signs it. Use the ZIPs or install.ps1 instead

Pick a model interactively

# Run with no arguments to open the model picker (local models + catalog)
./eullm

Web browsing

# Enable transparent web fetch (URLs in messages are fetched and injected)
./eullm run model.gguf --web

Usage

./eullm run ./model.gguf                    # Run any GGUF model
./eullm run ./model.gguf --batch-size 16    # Continuous batching for RAG
./eullm run ./model.gguf --web              # Enable web browsing

Verify checksums

sha256sum -c checksums.txt

What changed

See CHANGELOG.md for what was added, fixed and made faster in this version.

License

EULLM is AGPL-3.0-or-later. Running an unmodified binary, or modifying it for internal or local use, carries no additional obligation. If you modify EULLM and let others use it over a network, including as a hosted service, you must offer them the source of your modified version. Versions up to and including 0.7.4 were released under Apache 2.0 and keep those terms. The models EULLM runs have their own separate licenses.

What's Changed

Full Changelog: EuLLM-v0.7.95...EuLLM-v0.8.0