Repository navigation
v0.2.2
- Legacy backend (default, no changes needed)
Run exactly as before, no special flags:
llama-server -m model.gguf --host 0.0.0.0 --port 8080 --warmup -t 8
- New backend (ggml-spacemit, opt-in)
Both conditions are required: the environment variable AND the device flag.
export GGML_SPACEMIT_BACKEND=1
llama-server -m model.gguf --host 0.0.0.0 --port 8080 --warmup --device SPACEMIT0
Multimodal models (vision / audio):
GGML_SPACEMIT_BACKEND=1 llama-server -m model-text.gguf
--media-backend smt --smt-config-dir <config_dir>
--host 0.0.0.0 --port 8080 --warmup --device SPACEMIT0
Benchmarking (llama-bench):
GGML_SPACEMIT_BACKEND=1 llama-bench -m model.gguf -p 128 -n 128 -dev SPACEMIT0
To confirm the new backend is active: the startup log should show assigned to device SPit lines.