v0.7.0 — measuring the decode, and a session that lets go of its RAM

Pre-release. The APK builds, installs and generates on the target device, but the changes below have not had their on-device A/B yet — in particular the session teardown fix, which is a lifecycle change only the app can exercise. Promoted to a stable release once that measurement is in.

Measuring the decode instead of inferring it

--compute-trace PATH and --io-trace PATH. Until now compute_ms was a residual — whatever was left after the parts that were measured — so a large figure could equally mean genuine matmul, a dense-weight fault storm on a >RAM model, or a throttled CPU. Indistinguishable.

The compute trace returns true for every node from the eval callback, which forces ggml to isolate and synchronise each one. The wall delta between callbacks is then that node's real compute, and the major-fault delta attributes a >RAM stall to the node that actually paid for it. The I/O trace emits one row per expert read (latency, aligned window, layer/expert/projection/lane). Read both with scripts/decode-analyze.py.

Both are diagnostics that perturb the run they measure, so only shares are meaningful — never compare their tok/s with the benchmark tables. Done entirely from outside llama.cpp through the public cb_eval: no patch, no fork.

A session that lets go of its RAM before the next one loads

Changing the model or a setting started a new engine while the old process still held its model and its expert cache. The replacement then sized its cache against a MemAvailable still deflated by the dying one — so the app was triggering "two engines at once" on itself at every settings change, silently starving the very cache being retuned, and the combined footprint could be OOM-killed. The new session now reaps the old process off the main thread before probing memory.

Also fixed: the delayed force-kill is now cancellable, so an unload followed quickly by a new prompt can no longer let a stale kill land on the fresh process (exited 137).

Smaller cache rungs, to find where the cache stops paying

Settings now offer 500 and 1000 MiB expert-cache budgets. They were unavailable because the engine rejects a fixed budget under its 1500 MiB floor — a floor that says a cache smaller than one token's routed working set can only thrash. Sound, but it was measured on models whose cache pays for itself. On gpt-oss-120b at top-2 (~886 MB routed per token, an 8–13% hit from a 2000–3000 MiB budget covering ~5% of a 56.8 GB expert bank) the question is live, so the small rungs route through the floor's own escape hatch.

Also in this release

• Markdown-rendered answers in the Android chat, and a scroll that no longer fights the user.
• --route-trace: which experts every token actually routed to, per layer, for offline analysis (scripts/route-analyze.py, scripts/route-viewer.py).
• docs/android-memory.md — what reclaims a >RAM engine's memory on a phone, which levers exist (almost none), and why the cache hit rate is the signal the kernel judges you by: the LRU promotes a page only on a second reference, and a cache hit is that reference. Records the watermarks, the vendor's swappiness-160 override, and the 65536-byte RLIMIT_MEMLOCK ceiling that makes mlock unusable here.
• docs/benchmarks.md split into the Android matrix and docs/benchmarks-gpt-oss.md.

Performance

Unchanged, and not re-measured for this release. No throughput claim here is new — the tables in docs/benchmarks.md still stand as measured for v0.6.0. Everything added here is either a diagnostic (off by default), an app-lifecycle fix, or a new Settings rung.

Install

app-dev-debug.apk is the sideload (dev) build for arm64-v8a, built from 0f1d503. It reads models adb-pushed to /sdcard/Download or /data/local/tmp; installing it does not touch already-downloaded models.

adb install -r app-dev-debug.apk