Sparsa 0.1.3 experimental (MiniMax-M3.1-Flash runtime)

Sparsa 0.1.3 experimental

EXPERIMENTAL. The runtime for MiniMax-M3.1-Flash (427B MoE, 60 layers, 57 x 128 experts top-4)
on consumer NVIDIA cards plus system RAM and the SSD. 0.1.3 adds image input and the engine work
from feat/minimax-engine since 0.1.2. Still slow, still experimental; see README.md.

What 0.1.3 changes

• Vision (new). A 1.73 GB mmproj-MiniMax-M3.1-Flash-F16.gguf adds pictures: OpenAI image_url
  parts and Anthropic image blocks, as a URL, a base64 data URL or a local path. The engine loads the
  mmproj, runs the 32-layer ViT on the CPU in fp32 and splices the image rows into the prompt. Parity
  against transformers: cosine >= 0.99999. A full-model check on a red circle answered "The object is
  a red circle."
• python sparsa.py download mmproj --dir <model folder> fetches the vision file, and serve finds a
  mmproj-*.gguf beside the parts. --mmproj PATH names another one; --no-vision stays text only.
• Decode reads (#25, #26). The SSD prefetch's corruption and 2-GPU fault are fixed (the prefetch
  stays off by default), tools/minimax/io_probe.cpp measures the drive's slow windows, the prompt
  workspace can borrow the decode expert slots (--minimax-prefill-vram swap, opt-in), and vram_fill
  copies from the pinned RAM tier.
• Small tiers (#14). Recipes and the publishing tool for the 94.8-124.5 GB tiers; those files are
  on the Hub.
• sparsa.py download knows all ten tiers, and the README is rewritten: the ten tiers with size, bpw,
  loader and top-1, the measured speed, vision, and the limits.

Known limits

• Decode is a few tokens a second and drive-bound; a chat message waits about half a minute for its
  first token.
• The vision encoder is CPU fp32 only: 64-80 s per 640x480 picture, one picture at a time.
  Images, no video.
• Long context works to 32 K; above that expect it to be slow or to fail.
• Tool calls arrive whole, not streamed as fragments.

Assets

Built from release/sparsa-0.1.3 with python tools/minimax/package_release.py; each archive was
unpacked into an empty folder and bin/strata-minimax --minimax-inspect run there (exit 0).

• sparsa-v0.1.3-windows-x64-cuda.zip - Windows x64 engine, bin/strata-minimax.exe with the CUDA
  and VC++ libraries it needs. CUDA architectures sm_75 / sm_80 / sm_86 / sm_89 / sm_120. Needs an
  NVIDIA driver that supports CUDA 13 (580 or newer).
• sparsa-v0.1.3-linux-x64-cuda.tar.gz - Linux x64 engine, bin/strata-minimax, same
  architectures, with the CUDA runtime libraries. Built on Ubuntu with CUDA 13.2.
• SHA256SUMS.txt - SHA-256 of both archives.

Unzip the engine into a checkout of this repository so bin/strata-minimax.exe sits there, then
python -m pip install -r requirements.txt and python sparsa.py serve --model <folder>.

Licences

Sparsa is MIT. THIRD_PARTY_NOTICES.md lists ggml/llama.cpp, ik_llama.cpp, agentionai/llama.cpp,
stb_image and upstream Strata. The model files and the mmproj are under the MiniMax Community License
and are not part of these assets.

Sparsa 0.1.2 experimental (MiniMax-M3.1-Flash runtime)

Sparsa 0.1.2 experimental

EXPERIMENTAL. The runtime for MiniMax-M3.1-Flash (427B MoE, 60 layers, 57 x 128 experts top-4)
on consumer NVIDIA cards plus system RAM and the SSD. 0.1.2 brings the engine work from
feat/minimax-engine into main and makes chat read its prompt the fast way. Still slow, long context
still work in progress; see README.md.

What 0.1.2 changes

• The batched prompt path is the chat default. sparsa.py serve asks for one 4096-token chunk
  (--prefill-batch auto) and the engine shrinks it to what each card's free VRAM holds. A 411-token
  message is read in 30.5 s instead of 124 s (13.5 against 3.3 tok/s) on the measured box: the
  per-token path re-reads every routed expert once per token.
• Long context and decode from #15, #18, #19, #20: 32 K-256 K KV planning, the indexer and MSA at
  long context, the RAM/SSD tier read path, direct reads and the one-layer-early prefetch.
• --prefill-expert-gemm mmvq|auto|tc exposes the tensor-core expert GEMM (P11). It is not the
  default: the full model's G3 gate rejects it (below). It stays 1.33x on a 4 K chunk for anyone who
  wants it.

Measured (full IQ2_XS-D3, 2x RTX 5070, the box idle)

what	result
decode, 753-token prompt, 64 greedy tokens	4.17 tok/s (240 ms/token), 891 MB/s drive
prefill, 411 tokens	30.5 s = 13.5 tok/s
prefill, 753 tokens	33.8 s = 22.3 tok/s
prefill, 4,686 tokens	112.9 s = 41.5 tok/s
prefill, 32 K tokens (measured in P11)	739.6 s = 44.3 tok/s
the same 411-token prompt on the token path	124 s = 3.3 tok/s

Everything here is drive-bound and the NVMe on that box swings between 0.12 and 5.6 GB/s: the same
753-token prompt took 33.8 s once and 59.9 s later with nothing else running.

Quality (G3, six short prompts, 679 positions, vs the fp32 reference)

arm	KL64 mean	full-vocab KL	top-1
this release (experts on the card)	1.2346	1.4409	60.82 %
llama.cpp, same file	1.3446	-	59.20 %
--prefill-expert-gemm tc	1.2998	1.4886	58.17 %

tc also drops code_lru from 38.6 % to 30.7 % top-1, so it is opt-in. The 15-layer slice gate passed
(KL 1.34e-4) with a Q2_0 down projection; this tier's down projection is IQ3_XXS, which is where to
look first.

Assets

Built from release/sparsa-0.1.2 (PR #22) with python tools/minimax/package_release.py; each archive
was unpacked into an empty folder and the engine started there (--minimax-inspect, exit 0).

• sparsa-v0.1.2-windows-x64-cuda.zip (399 MB) - Windows x64 engine, bin/strata-minimax.exe with
  cublas64_13, cublasLt64_13 and the VC++ runtime. CUDA architectures sm_75 / sm_80 / sm_86 / sm_89 /
  sm_120. Needs an NVIDIA driver that supports CUDA 13 (580 or newer).
• sparsa-v0.1.2-linux-x64-cuda.tar.gz (471 MB) - Linux x64 engine, bin/strata-minimax with the CUDA
  runtime sonames, same architectures, built on Ubuntu with CUDA 13.2.
• SHA256SUMS.txt - SHA-256 of both archives.

Licences

Sparsa is MIT. THIRD_PARTY_NOTICES.md lists ggml/llama.cpp, ik_llama.cpp, agentionai/llama.cpp and
upstream Strata. The model files are under the MiniMax Community License and are not part of these
assets.

🤖 Generated with Claude Code

Sparsa 0.1.1 experimental (MiniMax-M3.1-Flash runtime)

Sparsa 0.1.1 experimental

EXPERIMENTAL. A bugfix release for the 0.1.0 runtime that runs MiniMax-M3.1-Flash (427B MoE,
60 layers, 57 x 128 experts top-4) on consumer NVIDIA cards plus system RAM and the SSD. It is still
slow and long context is work in progress; see the 0.1.0 notes and README.md for what the runtime
does and does not do.

What 0.1.1 fixes

The published IQ2_KL, IQ3_K and IQ4_KS files could not run a prompt on the 0.1.0 binaries: the
batched prefill of a prompt longer than a token stopped with

strata-minimax: blk.3.ffn_gate_exps.weight: ggml-cpu has no vec_dot for type 164

(type 164 is IQ2_KL). Every prompt goes through that path, so those three tiers were unusable with
the release, whatever the hardware. Two engine fixes:

• The batched CPU expert projection now handles the ik_llama.cpp formats. It quantizes the
  chunk's activations to q8_K once and dots each weight row with the engine's own codec, the way the
  single-token path already did. IQ2_KL now returns a complete logits dump where 0.1.0 exited 1.
• Linux builds compile that codec for AVX2. iqk_quants.cpp gets -mavx2 -mfma -mf16c on
  GCC/Clang, matching the MSVC /arch:AVX2 it already had. Before, a Linux build ran the codec
  scalar: one 547-token prompt was still running after 12 minutes; it now takes 484 s including the
  load. The older-CPU builds (STRATA_ISA_FLOOR) keep the scalar codec.

Also: --minimax-prefill-experts auto (the default) now falls back to CPU experts when the expert
tensors are one of those formats, instead of stopping at iq_mmvq: type 164 is not supported. An
explicit --minimax-prefill-experts gpu still asks for the device. And the README now carries the G3
quality numbers for the three tiers.

Everything else, including the default path for other models, is unchanged from 0.1.0.

Quality (G3, six short prompts, 679 positions, vs the fp32 reference)

tier	KL64 mean	full-vocab KL	top-1 agreement
MM-IQ2_XS-D3	1.211	1.472	59.4 %
MM-IQ2_KL	1.184	1.387	59.8 %
MM-IQ3_K	0.847	0.982	69.8 %
MM-IQ4_KS	0.760	0.905	74.4 %

Assets

Built from release/sparsa-0.1.1 (PR #17). Each archive was unpacked into an empty folder and the
engine started there (--minimax-inspect on a small test file, exit 0).

• sparsa-v0.1.1-windows-x64-cuda.zip - Windows x64 engine, bin/strata-minimax.exe with the CUDA
  and VC++ libraries it needs. CUDA architectures sm_75 / sm_80 / sm_86 / sm_89 / sm_120 (Turing
  through Blackwell). Needs an NVIDIA driver that supports CUDA 13 (580 or newer).
• sparsa-v0.1.1-linux-x64-cuda.tar.gz - Linux x64 engine, bin/strata-minimax, same
  architectures, with the CUDA runtime libraries. Built on Ubuntu with CUDA 13.2.
• SHA256SUMS.txt - SHA-256 of both archives.

Unzip the Windows asset into a checkout of the repository so bin/strata-minimax.exe sits there; the
Linux tarball unpacks the same way. Then:

python -m pip install -r requirements.txt
python sparsa.py serve --model <folder with the GGUF parts>

The GGUF files are on
LyJonathan/MiniMax-M3.1-Flash-GGUF;
IQ2_XS, IQ2_XS-D3, IQ3_K, IQ4_KS and IQ2_KL load in this engine. IQ2_XS and IQ2_XS-D3 also load in a
patched llama.cpp.

Licences

Sparsa is MIT. THIRD_PARTY_NOTICES.md in the repository lists ggml/llama.cpp, ik_llama.cpp,
agentionai/llama.cpp and upstream Strata. The model files are under the MiniMax Community License and
are not part of these assets.

🤖 Generated with Claude Code

Sparsa 0.1.0 experimental (MiniMax-M3.1-Flash runtime)

Sparsa 0.1.0 experimental

EXPERIMENTAL. A working preview of the runtime that runs MiniMax-M3.1-Flash (427B MoE, 60
layers, 57 x 128 experts top-4) on consumer NVIDIA cards plus system RAM and the SSD. It loads and
answers; it is slow, long context is work in progress, and vision is not implemented.

Sparsa is a fork of Strata. The engine is strata-minimax; the
Python server is Strata's (OpenAI /v1/chat/completions, Anthropic /v1/messages, streaming, tool
calls, web page).

Assets

• sparsa-v0.1.0-windows-x64-cuda.zip - Windows x64 engine, bin/strata-minimax.exe with the CUDA
  and VC++ libraries it needs. CUDA architectures sm_75 / sm_80 / sm_86 / sm_89 / sm_120 (Turing
  through Blackwell). Needs an NVIDIA driver that supports CUDA 13 (580 or newer).
• sparsa-v0.1.0-linux-x64-cuda.tar.gz - Linux x64 engine, bin/strata-minimax, same
  architectures, with the CUDA runtime libraries. Built on Ubuntu with CUDA 13.2.

Unzip the Windows asset into a checkout of the repository so bin/strata-minimax.exe sits there; the
Linux tarball unpacks the same way. Then:

python -m pip install -r requirements.txt
python sparsa.py serve --model <folder with the GGUF parts>

It listens on 127.0.0.1:8080 only. To use it from another machine on your network, add
--access lan: Sparsa binds every address, makes an sk-strata- API key (kept in
sparsa-minimax.json), prints the LAN URL and the firewall note, and the server refuses requests
without the key. --show-key prints the key again; --no-api-key serves without one, with a warning.
The same key works as Authorization: Bearer (OpenAI) or x-api-key (Anthropic).

README.md in the repository has the model downloads (each tier's hf download line), the measured
numbers, the hardware notes and the limits. The GGUF files are on
LyJonathan/MiniMax-M3.1-Flash-GGUF.

Which quant tiers these binaries load

All of them: IQ2_XS, IQ2_XS-D3, IQ3_K, IQ4_KS, IQ2_KL and NVFP4. IQ2_XS and IQ2_XS-D3 also load in a
patched llama.cpp; the ik_llama-style expert types (IQ3_K, IQ4_KS, IQ2_KL) and the lossless NVFP4
need this engine.

Measured on this build (2x RTX 5070 12 GB, Ryzen 9 5900X, 128 GB DDR4, Windows)

Full MiniMax-M3.1-Flash-IQ2_XS-D3 (146.1 GB), real 512-token text, 32 greedy tokens: 2.88 decode
tok/s (347 ms/token) against 2.64 on the build before the P7 decode work, i.e. about 2.9 tokens a
second; 7.8 experts and 135 MB per token came off the SSD. Experimental, and speed work is ongoing.
Prefill on the earlier build was 16.5-17.7 tok/s for 512 tokens (first token after about half a
minute). Quality against an fp32 reference on six prompts: top-1 agreement 59.35 % (llama.cpp on the
same file 59.20 %), KL64 mean 1.2110 (llama.cpp 1.3446).

Licences

Sparsa is MIT. THIRD_PARTY_NOTICES.md in the repository lists ggml/llama.cpp, ik_llama.cpp,
agentionai/llama.cpp and upstream Strata. The model files are under the MiniMax Community License and
are not part of these assets.

🤖 Generated with Claude Code