v0.15.0: Cache-aware expert dropping
Cache-aware expert dropping: --drop-cold-experts F skips a routed expert only when it is a cache miss and the router weighted it below F × (1/top-k) — quality is spent only where it buys a flash read. An expert already resident always runs, however small its weight.
Measured on device (Qwen3.6-35B-A3B, top-k 8 of 256, cache 3000): 2.55 tok/s off → 3.94 at F = 0.75 (+55%) → 4.70 at F = 1.0 (+84%), with per-token bootstrap intervals disjoint. A 15-question GSM8K check found no quality loss at any threshold — a sample that size rules out a collapse, not a subtle cost. Raw data: docs/bench-data/2026-07-22-drop-cold-experts/ and .../2026-07-22-drop-quality/.
Unlike turbo top-k the output is not reproducible — what gets dropped depends on what the cache held — so it carries no rows in the README's deterministic benchmark tables. Full design and caveats: docs/expert-dropping.md.
In the app: Speed / quality → Drop cold experts (off / 50% / 75% / 100%), defaulting to 75% — the conservative end of the measured curve. The CLI keeps defaulting to off: the byte-identity gates need a deterministic default.
Also in this release
- New gates G8a/G8a'/G8b/G8c pin the machinery: an inert threshold is byte-identical to the undropped stream, and dropping is a proven no-op at top-k 1.
validate()now rejects a NaN--drop-cold-expertsthreshold instead of silently arming nothing (found by a pre-release audit).- README: the two quality-trading knobs (turbo top-k and cache-aware dropping) are now a single section, Trading quality for speed.
- Route traces gain a
droppedcolumn; the metrics summary reportsexperts_routed/experts_dropped.
Full details in the CHANGELOG.
The attached app-dev-release.apk (versionCode 28) is the sideload build; it installs over previous releases.