-
Notifications
You must be signed in to change notification settings - Fork 0
FAQ
Does this need internet or an API key? No. Everything runs on-device with llama.cpp. Once the GGUF model file is on the phone, the library never makes a network call.
Is my users' data private? Yes — prompts, replies, and embeddings stay in the app's process. Nothing is sent anywhere.
What model formats are supported?
GGUF only (the current llama.cpp format). Any quantization — Q4_0, Q4_K_M,
Q5_K_M, Q8_0, etc. See Choosing a Model.
Why isn't a model bundled? LLM weights are 400 MB–8 GB, far larger than an AAR or a reasonable APK. You download or ship the GGUF separately — see Choosing a Model.
Which ABIs / devices?
Free targets arm64-v8a (every modern Android phone), minSdk 24. The build is
16 KB-page aligned for Android 15. x86_64 (emulators/Chromebooks) is a Pro
build option.
Can I run it on the GPU?
The Free tier is CPU/NEON only (gpuLayers = 0). Vulkan GPU offload is a
Pro feature — see Free vs Pro.
Can I stream tokens as they generate?
The Free complete returns the full text at once. Streaming (a Kotlin Flow of
tokens) is a Pro feature.
Can I keep a conversation going (multi-turn)?
Free complete is single-turn: build the history into your prompt yourself. Managed
multi-turn sessions with a persistent KV cache are a Pro feature.
Can I force JSON / a schema out of the model? Grammar / JSON-constrained decoding is a Pro feature. In Free, prompt for JSON and parse defensively.
Is it thread-safe?
A single LlamaModel is not thread-safe — serialize calls. Multiple concurrent
sessions (each with its own KV cache) are a Pro feature.
How much RAM do I need? Roughly model file size + KV cache + ~20%. On a 6 GB phone stay at ≤ 3B (Q4). See the RAM table in Choosing a Model.
Why is generation slow?
On CPU, speed depends on model size, quantization, and phone. Use a smaller/more
quantized model, more threads, and a modern device. See Troubleshooting.
What license? The wrapper is MIT. llama.cpp is MIT (© the ggml authors). You can ship it in commercial apps.
Which llama.cpp version?
Build b9878, pinned as a git submodule. The version is listed in the
README.
Do you offer a paid tier? Yes — Pro adds Vulkan GPU, streaming, JSON/grammar, vision, context save/restore, multi-session, LoRA, and function calling, distributed via jokobee.com. See Free vs Pro.