Skip to content

ThunderMLX v0.2.0

Choose a tag to compare

View all tags
@jonathan308 jonathan308 released this 19 Jul 09:34
v0.2.0
c9f189a

Long-form thinking, without channel loss

ThunderMLX v0.2.0 fixes the remaining MiniMax-M3 Q4 failure where a long standalone artifact could begin inside native reasoning, never emit </mm:think>, and leave clients with reasoning but no final answer.

  • Detects only complete, tool-free standalone HTML requests and the model's own explicit planning-to-implementation declaration.
  • Injects the native thinking-close token through rank 0's existing sampled-token synchronization, so both pipeline ranks remain in lockstep and generation continues in the content channel.
  • Stops at the first complete </body></html> boundary, suppressing a duplicated second document while synchronized EOS propagates.
  • Uses MiniMax's published thinking defaults when clients omit sampling: temperature 1.0, top-p 0.95, no repetition penalty.
  • Raises the portable default output budget and hard ceiling to 32,768 tokens.

Loop containment

The existing exact-span decode guard is now documented and configurable as an AntiDoom-inspired runtime safeguard. It scans the rolling tail every 12 tokens and ends only confirmed byte-identical semantic loops through synchronized EOS. It does not alter normal logits and is explicitly not upstream AntiDoom FTPO/LoRA training. See docs/ANTIDOOM.md.

Operations and visibility

  • Adds advanced dashboard settings for thinking sampling, artifact transition, and loop-scan cadence.
  • Adds repeatable exact-prompt and offline transition probes.
  • Keeps the native mlx-vlm 0.6.5 tool path, split 38,22, MSA profile, cache policy, networking, and model gateway unchanged.

Qualification results

Reference hardware: M3 Ultra 256GB + M5 Max 128GB over Thunderbolt/JACCL, MiniMax-M3-4bit.

  • Exact Shadow Syntax prompt: 2,158 reasoning characters followed by one complete 52,020-character HTML document, 26.43 tok/s decode.
  • 200.5K context: 309.14 prompt tok/s cold and 24.52 tok/s cached decode, reusing 200,474/200,518 prompt tokens.
  • Short post-restart tool decode: 32.15-32.29 tok/s; thinking began streaming at 0.64s on the hot turn.
  • Non-stream standalone artifact: one complete document with reasoning/content separation at 30.68 tok/s.
  • True stop/start released wired memory to 4GB/3GB; a 34-tool agent session then restored 20,645/20,682 tokens from SSD.
  • Passed OpenAI tool calls, Anthropic Messages, Codex Responses, Claude Code extended loops, real ZCode/OpenCode workflows, images, cancellation, multi-session isolation, hot cache, SSD cache, and OpenWebUI-shaped stress with no new server failures.

Upgrade

git pull origin main
open ./M3_Stop.command
open ./M3_Start.command

Existing private machine configuration remains in .env.local and is not part of the release.

Assets 2
Loading