v0.9.0 — Local LLM support via llama.cpp

What's new

llamacpp provider — Run the full Aleph RLM loop on-device with any GGUF model. Zero API cost.

Setup

brew install llama.cpp                          # Mac
llama-server -m model.gguf -c 16384 -ngl 99    # Start server

export ALEPH_PROVIDER=llamacpp
export ALEPH_LLAMACPP_URL=http://127.0.0.1:8080
export ALEPH_MODEL=local
aleph

Or let Aleph auto-start the server:

export ALEPH_PROVIDER=llamacpp
export ALEPH_LLAMACPP_MODEL=/path/to/model.gguf
export ALEPH_MODEL=local
aleph

Details

• Auto-starts llama-server with health checking, configurable context size and GPU layers
• Handles reasoning models (Qwen 3.5, QwQ) that use reasoning_content
• ALEPH_BASE_URL env var for overriding any provider's endpoint
• 28 new tests
• Cross-platform: Mac, Windows, Linux

See CONFIGURATION.md for all ALEPH_LLAMACPP_* variables.

Full changelog: https://github.com/Hmbown/aleph/blob/main/CHANGELOG.md