v0.3-preview

Why this exists: v0.3 — the repack without the second copy — turned out to be a longer
road than one release cycle, so this is a checkpoint published as a pre-release. It shares
where v0.3 stands today, in a form you can actually run. It is not the finished v0.3 and
does not claim to be.

What changed

• The repacker can now build a virtual repack: instead of writing a second packed copy
  of the experts (which costs your disk the model's size again), it writes a small manifest
  and the engine serves experts straight out of your original GGUF file. Space cost is
  exactly 1.0x and no data is moved.
• The launcher can serve a virtual repack as an opt-in. The packed path you already use
  is untouched and stays the default. The launcher's model menu now offers a repack mode
  toggle (packed | virtual); when the menu is skipped it asks once before identifying the
  model.
• Prefetch is a property of the catalog row, not of the virtual path. Two rows ship with
  it on by default (static K=8/N=4): qwen35-122b-nonextn and kimi-k2.6-ram-447gb. The other
  four rows ship off (qwen35-397b is additionally on promotion hold), and the launcher
  leaves them off. In a preregistered A/B on the 122B test model, prefetch made virtual-mode
  decode about 14% faster at under 2% extra bytes read.
• Engine and repacker binaries are updated to carry the virtual path. The packed pipeline
  in them is unchanged in behavior.
• The bundled catalog models.json moves to schema version 2.

Good to know

• The virtual path is slower than the packed path today. Reading in place means the
  experts sit scattered through the original file, so fetching them costs more. Prefetch
  claws part of that back — and note the 14% above is virtual-with-prefetch versus
  virtual-without, not virtual versus packed. That 14% is a PROBE-grade measurement,
  not a gate run, and it was taken on a working tree that predates the release binary
  rather than on the binary in this zip. We have not published a formal packed-vs-
  virtual comparison yet; that measurement comes with the finished release. If your disk
  fits the packed copy, keep using it.
• This pre-release is a status report, not the v0.3 the roadmap promises. The polished
  in-place release — call it v0.3.1 — follows once more of the gap is closed, and that
  work is under way.
• The virtual path has been exercised end to end on the 122B test model. Treat other
  models as untested in this preview.
• Known issue: on a virtual run the launcher's status screen still reports the packed
  path's rows. The launcher does not yet tell the two modes apart on that screen, so
  copy integrity, serving validation, the performance gate row and the reference
  numbers all show what the packed path would show. A virtual repack copies nothing, so
  the copy integrity row does not apply to it; the row simply inherits a true it was
  handed. The virtual path is verified separately, by an 8-item virtual plan gate in the
  launcher and the engine's own re-derivation of the manifest, and that row reflects none
  of it. The speed figures on that screen are packed measurements, and the virtual path
  is not under the performance gate at all:
  a -Repro or -Smoke run can go as far as printing performance gate : PASS on a
  virtual serve, and that verdict belongs to the packed criteria, not to what you are
  running. The screen is fixed in v0.3.1.

Verify the download

moe-direct-v0.3-preview-win-x64.zip  (541,608,407 bytes)
SHA-256: 5cffcec0732c0f673222eee58c6a4bb361f0d7130991df16bac4dc0a09517ba0

SHA256SUMS.txt (shipped alongside the zip):

5cffcec0732c0f673222eee58c6a4bb361f0d7130991df16bac4dc0a09517ba0  moe-direct-v0.3-preview-win-x64.zip

v0.2.3-corrected - same release, correct source tag

Why this exists: the original v0.2.3 tag accidentally pointed at source still carrying the v0.2.2 launcher - the zip below was always the real v0.2.3 build. This re-release pins the matching source; same binaries, same SHA-256, nothing to re-download. The notes below are unchanged from v0.2.3.

---

A convenience patch. Same engine and repacker binaries as v0.2.2; no new measurements.

What changed

• Arch-template is on by default - an unlisted GGUF of a known architecture now serves without any switch, labelled experimental. Turn off with -ArchTemplate off.
• Warm-up is on by default - the server warms its serving path once, right after ready. Restore the old behaviour with -Warmup off.
• The model menu labels each entry - [catalog], [template: <arch>], [unsupported] or [identify pending], read from the file's own header, so you can see the likely route before you pick.
• A warmed start says so - the status screen reports the performance gate as [unmeasured] on a default start, because every published number is a cold-cache run. -Repro and -Smoke force warm-up off so benchmark runs stay comparable.

Good to know

• Warm-up warms the serving path, not your prompt - a large system prompt still pays its full prefill on the first question. -Warmup file:<path> precomputes a real prompt; a graceful stop still saves the session's KV for the next start.
• The menu labels are a good guess, not a verdict - the real decision at start also weighs shards, sizes and the source pin.
• No integrity check moved: architectures without a template are still refused, and the repack still waits for your y/N. If your automation assumed the old defaults, pass -Warmup off or -ArchTemplate off.

Details: README · docs · TECHNICAL.md

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH:

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2.3-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

If you skip it, the launcher still checks extracted files against the bundle's sealed manifest on every start, but it does not compare the downloaded zip with the checksum published here. The checksum detects download corruption; because releases are unsigned, publisher trust rests on the GitHub account and the HTTPS distribution path.

v0.2.3 - both defaults on, and the model list tells you more

A convenience patch. Same engine and repacker binaries as v0.2.2; no new measurements.

What changed

• Arch-template is on by default - an unlisted GGUF of a known architecture now serves without any switch, labelled experimental. Turn off with -ArchTemplate off.
• Warm-up is on by default - the server warms its serving path once, right after ready. Restore the old behaviour with -Warmup off.
• The model menu labels each entry - [catalog], [template: <arch>], [unsupported] or [identify pending], read from the file's own header, so you can see the likely route before you pick.
• A warmed start says so - the status screen reports the performance gate as [unmeasured] on a default start, because every published number is a cold-cache run. -Repro and -Smoke force warm-up off so benchmark runs stay comparable.

Good to know

• Warm-up warms the serving path, not your prompt - a large system prompt still pays its full prefill on the first question. -Warmup file:<path> precomputes a real prompt; a graceful stop still saves the session's KV for the next start.
• The menu labels are a good guess, not a verdict - the real decision at start also weighs shards, sizes and the source pin.
• No integrity check moved: architectures without a template are still refused, and the repack still waits for your y/N. If your automation assumed the old defaults, pass -Warmup off or -ArchTemplate off.

Details: README · docs · TECHNICAL.md

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH:

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2.3-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

If you skip it, you are still protected: the launcher re-verifies every file inside the extracted bundle against its sealed manifest on every start, fail-closed. Only this paste establishes that the zip itself is the released one.

---

Erratum (2026-08-14): this tag's source snapshot accidentally carries the v0.2.2 launcher; the zip above was always the correct v0.2.3 build (SHA-256 unchanged), so downloads are unaffected. For source that matches the zip, use v0.2.3-corrected.

MoE-Direct v0.2.2

Two additions, both usable today. The engine delta is the arch-template acceptance path and catalog bookkeeping - nothing else moved.

What changed

• Warm-up file precompute - warmup now takes file:<path>: point it at the system prompt your client sends and that prefix is computed once, right after the server comes up, instead of on your first question. Measured once on the reference machine: 282 tokens precomputed in 12.0 s, 93 % reused by the first real request.
• Running an unlisted model (experimental) - start with -ExperimentalArchTemplate and a GGUF of a known architecture the catalog does not pin is derived, repacked, verified and served, labelled experimental. Templates exist for gpt-oss, qwen35moe and deepseek2; the end-to-end evidence is one model of one of them (gpt-oss-20b, 25 frozen gates, five greedy prompts token-identical against plain mmap).
• All nine expectation files now ship in the zip - the repacker's --selftest passes 65/65 from the zip itself, closing the 63/64 known issue from v0.2.1.
• The engine delta is published as patches/moedirect-v0.2.2-b10057.patch - applying it to the pinned upstream commit reproduces the exact source tree the shipped binaries were built from, also browsable as fork branch moe-direct-v0.2.2.

Good to know

• The headline paired measurement (2.3x over plain mmap) was taken on the v0.2.1 engine binary and has not been re-run on this zip's binary.
• No published number covers a template-derived model, and prefetch stays off on one - the experimental label is the honest state, not a formality.
• Still unsigned, so SmartScreen will warn; the MSVC runtime remains a prerequisite.

Details: README · TECHNICAL.md

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH:

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2.2-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

If you skip it, you are still protected: the launcher re-verifies every file inside the extracted bundle against its sealed manifest on every start, fail-closed. Only this paste establishes that the zip itself is the released one.

MoE-Direct v0.2.1

Download moe-direct-v0.2.1-win-x64.zip, not GitHub's auto-generated "Source code" archive - that one contains no binaries.

A maintenance and feature release on top of v0.2. Still a hands-on preview - Windows only, unsigned - and your weights are treated exactly as before: source GGUF opened read-only, every repacked record verified, consumed fail-closed.

What changed

• Warm start - a clean stop now saves the session state and the next start restores it. Time to first token on a strict same-prompt pair: 26,148 ms cold to 3,238 ms restored (8.1x), output character-identical. Restores are identity-guarded by a sidecar (model shards, manifest, bundle, config all hashed); any mismatch is a cold start with the mismatching field named.
• Autosave - the server also saves about every five minutes while idle, alternating two generations, so a crash or power cut costs at most the save being written. Roughly 190 MB per save at a 12k context, 257-280 ms per save including verification.
• Cache budget autotune - the RAM cache is sized from installed RAM and the model's slot geometry (clamped 4,096-12,288 MiB) instead of one fixed number, and the choice is announced with its reasons. Your override still wins.
• DeepSeek-V4-Flash-0731 joins the catalog - repack verified 33,024 of 33,024 records, smoke run 3.13-3.28 tok/s; prefetch ships disabled for this family and says so.
• BF16 router allowance for the deepseek4 family - that family's router tensors only; everything else stays F32-only and the compute path is byte-identical to the previous build.
• Kimi K2.6 prefetch promoted to validated - the pending A-B-B-A pair was run: both adjacent pairs favoured ON (x1.068, x1.056) and all four arms produced byte-identical output. A prefetch-relative gain only; K2.6's performance gate stays unpassed.

Numbers

• Qwen3.5-122B decode on this zip's binary: 5.65-6.26 tok/s per probe (arm average 5.96) against the same bundle's plain-mmap path at 2.13-3.21 (average 2.66) - 2.3x, both arms at ctx 12288, differing only in the read path.

Good to know

• Warm-start reuse needs an exact prefix - a request that diverges from the restored prompt is reprocessed in full on hybrid-attention models, and the expert cache still starts empty.
• Saved state lives under %LOCALAPPDATA%\MoE-Direct\kv\<profile>\, holds the session's tokens verbatim, is never uploaded, and is kept for at most four profiles.
• Known in this build: the repacker's --selftest reports 63/64 from the zip - one expectation file did not ship; serving is unaffected and a repo checkout passes 64/64. On non-English Windows a few warning lines can render garbled glyphs - cosmetic only, the operation itself behaves correctly.
• Prefetch across the six profiles: two validated, one reference-only, three disabled; an override on a non-validated profile is refused by launcher and engine both.

Details: README · TECHNICAL.md

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH (the expected hash is a0b78d0d22add05da42cc2af8bed0a739147268c22e3dadd700f9444cf01ee7a):

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2.1-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

Then unblock the zip (right-click > Properties > Unblock) before extracting with Windows "Extract All" into a new, empty folder - unblocking after extraction does not clean up files already extracted. If you skip the paste, you are still protected: the launcher re-verifies every file inside the extracted bundle against its sealed manifest on every start, fail-closed. Only the paste establishes that the zip itself is the released one.

MoE-Direct v0.2

MoE-Direct v0.2 - first public build

Download moe-direct-v0.2-win-x64.zip, not GitHub's auto-generated "Source code" archive - that one contains no binaries.

The first build you can actually run: a public preview for hands-on users, Windows only, unsigned. It serves Mixture-of-Experts models far larger than system RAM by keeping routed expert weights on the NVMe SSD and reading, per token, only the experts that token routes to.

What is in it

• Direct-read serving - expert weights stay on disk in an aligned store and are read on demand into a fixed, model-aware RAM cache (unbuffered overlapped I/O, no OS page cache). The math, the routing, the weights and the sampling are untouched - only the storage path underneath the expert tensors changed.
• A repacker that proves itself - the one-time repack rewrites expert tensors and SHA-256 compares every record against the source bytes, no sampling; launcher and engine both refuse to serve a repack that did not fully verify.
• A launcher that shows its work - twelve steps from bundle check to clean shutdown; the repack plan is shown and confirmed before anything is written; every run ends in one machine-readable status= line, and failures add a plain-language hint.
• A startup queue-depth sweep - the launcher measures your drive at QD 1/2/4/8 instead of assuming ours; your override always wins over the measurement.
• Two independent gates - format (the repack is byte-exact and verified) and performance (a run passed the frozen release gate) are recorded separately, and neither implies the other.
• Five model profiles with exact repos and revisions: Qwen3.5-122B (start here), gpt-oss-120b, Qwen3.5-35B, Qwen3.5-397B, Kimi K2.6 (1T-class). Weights are never redistributed.
• Local by default - the server binds to loopback; no telemetry, no automatic uploads, no background updater. The README enumerates every file this project writes.

Numbers

Reference machine: Ryzen 7 7800X3D, 32 GB DDR5, RTX 5080, PCIe Gen5 NVMe, Windows 11.

• Qwen3.5-122B sustained decode 5.59-5.69 tok/s, frozen release gate passed; the shipped binary re-checked in the anchor band at 6.32 tok/s average in a vicinity run.
• gpt-oss-120b integrity anchor: 12 paired responses, token IDs identical between direct-read off and on.
• Observations (measured, no gate): Kimi K2.6 at 1.03 tok/s with coherent output; a matched K2.6 direct-read vs plain-mmap pair at 3.95x; Qwen3.5-397B 1.99 tok/s; Qwen3.5-35B 4.40 tok/s CPU-only on a 16 GB laptop.

Good to know

• "Read speed divided by miss bytes per token" is an I/O ceiling, not an expected tok/s - the launcher prints the ceiling and a calibrated estimate separately, and marks the fixed term [unmeasured] off the reference machine.
• K2.6 at 1.03 tok/s is a demonstration of reach, not an interactive experience; its performance gate is unpassed.
• The first-run repack costs minutes to roughly 18 minutes on the recorded machine and roughly the model's size again on disk, with no resume; the output is never deleted automatically.
• One request at a time; an OpenAI-compatible subset; text only; MoE only; prefetch serves only on validated profiles, and overrides elsewhere are refused.

Details, full tables and every limitation: README. Security issues go through SECURITY.md, not public issues.

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH:

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

Then unblock the zip (right-click > Properties > Unblock) before extracting with Windows "Extract All" into a new, empty folder - unblocking after extraction does not clean up files already extracted. If you skip the paste, you are still protected: the launcher re-verifies every file inside the extracted bundle against its sealed manifest on every start, fail-closed. Only the paste establishes that the zip itself is the released one.

---

Built on llama.cpp (MIT), base commit 0bd0ec6 (b10057); upstream copyright, license and notices preserved. Built in an AI-assisted, measurement-first workflow; full credits in the README.