Skip to content

v0.2.2

Choose a tag to compare

@github-actions github-actions released this 13 Oct 05:52
51c81f9
  1. Legacy backend (default, no changes needed)

Run exactly as before, no special flags:

llama-server -m model.gguf --host 0.0.0.0 --port 8080 --warmup -t 8

  1. New backend (ggml-spacemit, opt-in)

Both conditions are required: the environment variable AND the device flag.

export GGML_SPACEMIT_BACKEND=1
llama-server -m model.gguf --host 0.0.0.0 --port 8080 --warmup --device SPACEMIT0

Multimodal models (vision / audio):

GGML_SPACEMIT_BACKEND=1 llama-server -m model-text.gguf
--media-backend smt --smt-config-dir <config_dir>
--host 0.0.0.0 --port 8080 --warmup --device SPACEMIT0

Benchmarking (llama-bench):

GGML_SPACEMIT_BACKEND=1 llama-bench -m model.gguf -p 128 -n 128 -dev SPACEMIT0

To confirm the new backend is active: the startup log should show assigned to device SPit lines. ⚠️ Setting the environment variable alone does NOT activate the new backend —without --device SPACEMIT0 you are still running the legacy backend.