Skip to content

MFQ Studio 0.3.2 - DeepSeek V4.1 Prefill at 471 tok/s

Latest

Choose a tag to compare

@xrj-com xrj-com released this 12 Sep 12:52
· 309 commits to master since this release

MFQ Studio 0.3.2

This release focuses on DeepSeek V4.1 Flash raw-HF prefill performance on Apple M3 Ultra, while preserving the 0.3.1 token-generation and MTP paths.

Highlights

  • Reaches up to 471.1 tok/s warmed prefill for DeepSeek V4.1 Flash raw-HF on a 512 GB Mac Studio M3 Ultra.
  • Adds an exact FP32 Metal top-k path inspired by DeepSelect for DeepSeek V4.1 sparse attention.
  • Accelerates large-M MoE prefill with the native sorted-MXFP4 path.
  • Auto-selects a 5,440-token maximum prefill chunk for the resident M3 Ultra geometry, keeping routed expert rows below the MLX 32,768-row boundary.
  • Fuses window-KV RMSNorm, partial RoPE, and activation fake quantization.
  • Keeps the backbone and MoE experts resident in unified memory while offloading only Engram storage to the Mac's internal SSD.
  • Refreshes the Tylogi AI Lab identity used in the repository documentation.

Measured DeepSeek V4.1 performance

Measured locally at batch size 1, 32,768-token context, temperature 0, and warmed execution on a 512 GB Apple M3 Ultra:

  • Model load: 38.6 s.
  • Prefill: 390.2 / 458.6 / 471.1 tok/s for representative 509 / 2,009 / 15,969-token deterministic repeated raw-completion prompts.
  • Token generation with MTP off: 18.29 tok/s.
  • High-acceptance MTP: 33.46 tok/s (+83.0%), with 132/132 drafted tokens accepted and output identical to non-MTP decoding.
  • Steady-state process RSS after load: approximately 287 GiB.

MTP gains depend on workload predictability and remain disabled by default.

Validation

  • 9 Metal component/source tests passed for the DeepSelect, MXFP4, chunk-autotuning, and fused KV preparation paths.
  • Native AppleClang Release build completed on an Apple M3 Ultra.
  • Metal runtime self-test, packaged CLI command checks, TypeScript build, Rust checks, Tauri release build, deep code-signature verification, and hdiutil verify all passed.
  • The packaged app and CLI report version 0.3.2; native executables are arm64.

Download

  • MFQ.Studio_0.3.2_aarch64.dmg: self-contained Apple Silicon macOS app.
  • SHA256SUMS.txt: release checksum.

Build information

  • Source tag: v0.3.2
  • Source commit: fdfb12852a226b6219db0320f8afca3d75da591d
  • DMG size: 703,880,572 bytes
  • DMG SHA-256: cb2459650a74c5eeb3616fbb2884bbd10641f54eae062173ccacdc84b22599ee

This macOS build is ad-hoc signed and has not been Apple-notarized.