Skip to content

MoE-Direct v0.2.1

Choose a tag to compare

View all tags
@tmxkzm1925-max tmxkzm1925-max released this 03 Aug 02:44
· 56 commits to main since this release
v0.2.1
116285a

Download moe-direct-v0.2.1-win-x64.zip, not GitHub's auto-generated "Source code" archive - that one contains no binaries.

A maintenance and feature release on top of v0.2. Still a hands-on preview - Windows only, unsigned - and your weights are treated exactly as before: source GGUF opened read-only, every repacked record verified, consumed fail-closed.

What changed

  • Warm start - a clean stop now saves the session state and the next start restores it. Time to first token on a strict same-prompt pair: 26,148 ms cold to 3,238 ms restored (8.1x), output character-identical. Restores are identity-guarded by a sidecar (model shards, manifest, bundle, config all hashed); any mismatch is a cold start with the mismatching field named.
  • Autosave - the server also saves about every five minutes while idle, alternating two generations, so a crash or power cut costs at most the save being written. Roughly 190 MB per save at a 12k context, 257-280 ms per save including verification.
  • Cache budget autotune - the RAM cache is sized from installed RAM and the model's slot geometry (clamped 4,096-12,288 MiB) instead of one fixed number, and the choice is announced with its reasons. Your override still wins.
  • DeepSeek-V4-Flash-0731 joins the catalog - repack verified 33,024 of 33,024 records, smoke run 3.13-3.28 tok/s; prefetch ships disabled for this family and says so.
  • BF16 router allowance for the deepseek4 family - that family's router tensors only; everything else stays F32-only and the compute path is byte-identical to the previous build.
  • Kimi K2.6 prefetch promoted to validated - the pending A-B-B-A pair was run: both adjacent pairs favoured ON (x1.068, x1.056) and all four arms produced byte-identical output. A prefetch-relative gain only; K2.6's performance gate stays unpassed.

Numbers

  • Qwen3.5-122B decode on this zip's binary: 5.65-6.26 tok/s per probe (arm average 5.96) against the same bundle's plain-mmap path at 2.13-3.21 (average 2.66) - 2.3x, both arms at ctx 12288, differing only in the read path.

Good to know

  • Warm-start reuse needs an exact prefix - a request that diverges from the restored prompt is reprocessed in full on hybrid-attention models, and the expert cache still starts empty.
  • Saved state lives under %LOCALAPPDATA%\MoE-Direct\kv\<profile>\, holds the session's tokens verbatim, is never uploaded, and is kept for at most four profiles.
  • Known in this build: the repacker's --selftest reports 63/64 from the zip - one expectation file did not ship; serving is unaffected and a repo checkout passes 64/64. On non-English Windows a few warning lines can render garbled glyphs - cosmetic only, the operation itself behaves correctly.
  • Prefetch across the six profiles: two validated, one reference-only, three disabled; an override on a non-validated profile is refused by launcher and engine both.

Details: README · TECHNICAL.md

Verify the download

One paste, nothing to compare by eye - in Explorer, open the folder that holds both downloaded files, right-click empty space > Open in Terminal (PowerShell), then paste; it prints OK or MISMATCH (the expected hash is a0b78d0d22add05da42cc2af8bed0a739147268c22e3dadd700f9444cf01ee7a):

$e=((Get-Content .\SHA256SUMS.txt -Raw).Trim() -split '\s+')[0];$a=(Get-FileHash .\moe-direct-v0.2.1-win-x64.zip -Algorithm SHA256).Hash;if($a -eq $e){'OK: hash matches'}else{'MISMATCH: download again'}

Then unblock the zip (right-click > Properties > Unblock) before extracting with Windows "Extract All" into a new, empty folder - unblocking after extraction does not clean up files already extracted. If you skip the paste, you are still protected: the launcher re-verifies every file inside the extracted bundle against its sealed manifest on every start, fail-closed. Only the paste establishes that the zip itself is the released one.

Assets 4
Loading