Troubleshooting

Troubleshooting

UnsatisfiedLinkError / native library not found

The AAR bundles libggml-base.so, libggml-cpu.so, libggml.so, libllama.so, and libllama_jni.so for arm64-v8a. If you see an UnsatisfiedLinkError:

• You're on an unsupported ABI. Run on an arm64-v8a device or emulator. The Free build has no x86_64 (that's a Pro option), so an x86_64 emulator will fail — use an arm64 emulator image or a real phone.
• You stripped the .so files. Don't exclude **/lib/** in packaging rules, and don't set an abiFilters that omits arm64-v8a.
• Verify at runtime: Log.i("llama", Llama.getSystemInfo()) right after startup. If that returns the CPU string, the libraries loaded fine.

LlamaException.ModelLoadFailed

loadModel failed. Check, in order:

1. Path is correct and the file exists. Use an absolute path, e.g. File(getExternalFilesDir("models"), "model.gguf").absolutePath.
2. It's a valid GGUF file (not a .bin, not a partial/corrupt download). Re-download and compare sizes.
3. Enough RAM. A model needs ~(file size + 20%) free. If the OS can't allocate it, load fails. Use a smaller / more-quantized model or lower contextSize. See Choosing a Model.
4. Storage permission — getExternalFilesDir() needs no permission; a path under shared storage might.

Generation is very slow

On-device CPU inference speed depends on model size, quantization, and the phone.

• Use a smaller / more quantized model — Q4_K_M at 0.5B–3B. A 7B model on CPU is slow on most phones.
• Increase threads in LlamaConfig (try the number of performance cores, often 4–6). More isn't always better — measure.
• Lower maxTokens — you pay per generated token.
• Lower contextSize — a huge context slows the prompt phase and eats RAM.
• GPU: Vulkan offload (much faster on capable phones) is a Pro feature — see Free vs Pro.

Building the library yourself? Always benchmark a release (-O3) build. A debug native build is 10–50× slower and will mislead you.

App crashes / out-of-memory when loading

You're over the RAM budget. Drop to a smaller model or smaller contextSize, and make sure you releaseModel any previously loaded model before loading another — native memory isn't freed by the garbage collector.

The reply is empty or cut off

• Empty: the model may have emitted an end-of-generation token immediately — check you're using an Instruct model and a sensible prompt/system prompt.
• Cut off: raise maxTokens. The reply stops at maxTokens or at the model's end-of-generation token, whichever comes first.

Output is different every run

That's sampling with temperature = 0.7 and seed = -1 (random). For deterministic output set temperature = 0f (greedy) or pin seed to a fixed value.

ANR / UI freeze

You're calling inference on the main thread. All heavy calls are suspend functions — run them from a coroutine (lifecycleScope.launch { … }), never directly on the UI thread.

Two calls at once corrupt output / crash

A single LlamaModel is not thread-safe. Serialize complete calls on one model. Concurrent per-session KV caches are a Pro feature — see Free vs Pro.

Still stuck? Open an issue with your device model, Android version, the GGUF you used, and the logcat output.