Skip to content

v0.1.0

Choose a tag to compare

View all tags
@Helldez Helldez released this 11 Jul 06:52
v0.1.0
db1d971

First tagged release of BigMoeOnEdge — a ports-and-adapters engine that streams MoE experts from flash so models larger than device RAM run on-device, built on top of llama.cpp's public API with no fork.

Highlights

  • MoE expert-selective streaming for qwen3moe (Qwen3-30B-A3B and siblings), qwen2moe, and llada-moe: only the routed experts per token are read from flash, with an optional LRU cache and a parallel read pool.
  • Lossless — byte-identical to a full in-memory run, proven by the synthetic gates and confirmed on a real 64-expert 4 GiB model.
  • Zero-fork llama.cpp: streaming rides entirely on the public eval-callback and gguf accessors; third_party/llama.cpp stays a stock upstream submodule.
  • bmoe-cli host tool with machine telemetry and a CSV sink.
  • Android example app with a live telemetry panel (tok/s, compute-vs-flash-I/O split, cache hit rate).

Assets

The attached app-debug.apk (arm64-v8a) is built by CI and debug-signed for on-device validation, not Play distribution. Push a .gguf to the app's files dir and pick it in the model picker. See examples/android/README.md.

Full details in CHANGELOG.md.

Assets 3
Loading