Skip to content

v0.8.1 — anon dense weights by default

Latest

Choose a tag to compare

@Helldez Helldez released this 17 Jul 16:33

Flips the default dense-weight policy to anon (O_DIRECT).

On a >RAM model, anon reads the dense (non-expert) weights via O_DIRECT into private buffers, so a memory reclaim swaps them to zram (fast) instead of refaulting them from flash (slow) — and the expert cache finally has room to earn its budget. Measured 0.998 vs 0.711 tok/s on gpt-oss-120b with a 2000 MiB cache.

Warm stays one tap away in Settings for models that fit in RAM, where its page-cache prefetch is the better trade. The fresh-install path now defers to a single source of truth for the default.

Note on reproducing the published numbers: those were measured with warm. To match them, set dense weights to Warm at load in Settings.

Pre-release, like v0.8.0: the session-teardown A/B is still owed, and this default flip's cross-model A/B (Qwen, Gemma) has not been run yet. Promote with gh release edit v0.8.1 --prerelease=false once measured.

Install

BigMoeOnEdge-v0.8.1-dev.apk — dev flavor, arm64. adb install -r to keep any models already on the device.