Qwen3.8-Flash-Next on one laptop CPU: native C, automatic 8 GB path

[shyringo]

I open-sourced qwen3.8-flash-next-in-c, a native C inference engine purpose-built for Qwen3.8-Flash-Next. It runs the 125B-A6B main model plus 51B PLE locally on a single laptop CPU.

The engine does not invoke another inference runtime. It directly implements the Qwen4Exp graph, including four-stream Hyper-Connections, Gated DeltaNet, QSA, PLE and 512-expert MoE. It supports terminal chat and a resident OpenAI-compatible local API with SSE and function tools.

Measured on an Intel Core i5-1340P / 32 GB Windows 11 + WSL2 laptop:

• resident single-conversation chat: 5.03 token/s, TPOT 0.199 s/token
• exact batch-4 verification: 9.89 positions/s
• automatic 8 GB RAM path: 1.64 token/s, 596 MiB peak RSS for a 19-token prompt plus 16-token output at context 2,048

The near-10 result is exact-batch aggregate throughput, not single-conversation TPOT. The batch benchmark computes a token-major reference first and reports speed only when all four greedy token IDs and the complete final-logit buffer match byte for byte. The 8 GB path preserves the same complete 248,320-logit SHA.

The Quick Start downloads and verifies the pinned Unsloth UD-IQ1_S GGUF from ModelScope, then opens chat:

https://github.com/shyringo/qwen3.8-flash-next-in-c

Reproducible results from other x86, ARM64, Linux, WSL2 and macOS laptop CPUs are welcome.