Production setup: MiniMax M2.5 as OpenClaw local brain with prefix caching + tool calling

[littlemac51]

We've been running local LLMs as an OpenClaw sub-brain since February — started with Qwen3-32B, moved through Qwen3-Coder-Next, then a 235B MoE on llama.cpp. Each iteration taught us something. Today we landed on what feels like the right setup: MiniMax M2.5-4bit on Rapid-MLX with working prefix caching and streaming tool calls.

The Stack

• Hardware: Mac Studio M3 Ultra, 256GB unified memory
• Model: mlx-community/MiniMax-M2.5-4bit (~129GB, 229B params, 10B active)
• Server: Rapid-MLX v0.4.2
• Framework: OpenClaw with openai-completions API provider

rapid-mlx serve mlx-community/MiniMax-M2.5-4bit \
  --port 8080 --host 0.0.0.0 \
  --enable-prefix-cache --cache-memory-percent 0.40 \
  --kv-cache-quantization --kv-cache-quantization-bits 8 \
  --enable-auto-tool-choice --tool-call-parser minimax \
  --served-model-name qwen3

What It Handles

• Heartbeats every 30 min — IMAP email check, channel status, memory indexing
• Morning/evening summary emails — multi-step tool chains (read files → IMAP → compose → SMTP send)
• Weather lookups, file operations, system checks — full tool calling through OpenClaw
• Identity grounding — personality files as system prompt, model stays in character across sessions

Performance

Metric	Value
Decode speed	~45 tok/s
TTFT (cold, 3.5K prompt)	~8s
TTFT (cached)	~1s
Prefix cache savings	33K+ tokens on repeated prompts
Morning email job	~5 min (multi-round tool calling)
Heartbeat cycle	~50s

Why MiniMax M2.5

We tried several models along the way. The big learning was that model architecture matters more than parameters for caching on MLX.

MiniMax M2.5 is pure full-attention MoE — no Mamba layers, no sliding window. That's why prefix caching works. Qwen 3.5 models (all sizes) use Mamba/SSM hybrid layers, which means the recurrent state can't be trimmed at arbitrary token boundaries. Every request rebuilds from scratch. See mlx-lm issue #980 for the details.

Quick reference:

• Pure full attention (incl. MoE) → prefix caching works (MiniMax M2.5, Qwen3, Llama)
• Attention + Mamba/SSM hybrid → caching broken (Qwen 3.5 all sizes)
• Sliding window + full attention → caching broken (GPT-OSS, Gemma 3)

For agentic workloads with repeated system prompts (heartbeats, cron jobs), prefix caching is the difference between 8s and 1s per request. That 8x improvement compounds when you're running 48 heartbeats a day.

Things We Learned

Use Rapid-MLX, not upstream vllm-mlx. The PyPI vllm-mlx (waybarrios) doesn't parse tool calls in streaming mode — they come back as raw <tool_call> text in the content field. Rapid-MLX fixes this.

Use the right tool parser. --tool-call-parser minimax for MiniMax. Not hermes, not qwen. Each model family has its own format.

Set explicit timeouts for network operations. If your model runs SMTP/IMAP via exec, set timeout: 60 on the exec call and socket.setdefaulttimeout(30) in Python scripts. Otherwise the framework may kill the process before the network operation completes.

cache-memory-percent 0.40 not 0.60 — MiniMax is 129GB, so you need to leave headroom on a 256GB machine.

Happy to answer questions. The setup handles real production work — not just benchmarks.

[raullenchai]

This is exactly the kind of real-world deployment story we love to see—thank you for sharing it in such detail. Running MiniMax M2.5 as a local OpenClaw brain for multi-step tool use and automated workflows is genuinely impressive.

Rapid-MLX has evolved quite a bit since v0.4.2. Notably, recent releases introduced opt-in prefix-extension reuse for hybrid Mamba/SSM and sliding-window models via --hybrid-cache-entries. While they still can't trim and reuse KV state as freely as full-attention models, they no longer have to rebuild stable prefixes from scratch every time.

If you upgrade and revisit Qwen 3.5 or another hybrid model, we’d love to hear how it impacts your heartbeat workload.

Please feel free to DM me as well. If you have specific requirements, bottlenecks, or ideas around running OpenClaw locally, I’d be very happy to learn more and see how Rapid-MLX can better support what you’re doing.

Thanks again for sharing such a thoughtful production setup!

— Raullen