dsx

DeepSeek-V4.1-Flash-1017 (552 B + 196 B Engram) on your old gaming PC
two RTX 3090s and 128 GB of RAM · 40+ tokens/s decode · 1,000+ tokens/s prefill measured · full routing, no expert pruning · OpenAI- and Anthropic-compatible API · chat in the browser, pictures, thinking levels

dsx runs DeepSeek-V4.1-Flash-1017 locally on two RTX 3090s and 128 GB of RAM. It is a Rust and CUDA inference engine with a T16 quantized model, a browser chat UI, image input, and OpenAI- and Anthropic-compatible APIs for coding agents.

The current model is a single 252 GB GGUF with the experts, Engram memory and tokenizer included. The download also supplies the expert ranking and a separate vision tower. T16 uses a trellis format designed for both desktop CPU and GPU execution.

The reference machine has two 24 GB RTX 3090s, an i7-12700K, 128 GB of RAM and an NVMe drive with about 255 GB free. You need Rust, CUDA 13 and an x86 CPU with AVX2; AVX-VNNI or AVX-512 VNNI is preferred.

Download the model and build the NVIDIA runner:

hf download lxe/DeepSeek-V4.1-Flash-1017-dsx-t16 --local-dir dsx-t16
CUDA_ARCHS=86 cargo build --release -p dsx --features cuda

Use CUDA_ARCHS=89 for an RTX 4090 or 120 for an RTX 5090. Start the server:

./target/release/dsx-server \
  --model dsx-t16/DeepSeek-V4.1-Flash-1017-dsx-t16.gguf \
  --rank dsx-t16/expert-rank.json \
  --ctx 131072 \
  --arena-gb 110 \
  --defer-rank 3 \
  --host 0.0.0.0 --port 8090

Open http://localhost:8090/ for the chat UI. The server loads vision.safetensors beside the model for image input. Use --no-vision for text only.

--defer-rank 3 lets lower-ranked CPU experts finish one layer late to reduce GPU waiting. It changes the model’s computation; omit the flag to disable that approximation. See server configuration for all options and expert placement for other memory and GPU configurations.

Point an OpenAI-compatible client at http://localhost:8090/v1, or an Anthropic-compatible client at http://localhost:8090. The model ID is deepseek-v4.1-flash.

For Pi, follow the provider setup, then run:

pi --provider dsx --model deepseek-v4.1-flash:off

Thinking is on at high effort by default. Requests can turn it off or choose another effort. For example:

curl http://localhost:8090/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "model": "deepseek-v4.1-flash",
    "stream": true,
    "reasoning_effort": "none",
    "messages": [
      {"role": "user", "content": "Write a Python function that merges sorted iterators."}
    ]
  }'

The default sampling is temperature 1 and top-p 1. Thinking controls cover both APIs. Use --api-key to require authentication.

On the reference two-3090 machine, a T16 free-generation run measured 41.0 tokens/s with --defer-rank 3 and optional FP8 Engram projections (DSX_ENGRAM_FP8=1). FP8 projections are off by default. This was a 120-token run after 300 tokens; throughput depends on context length, expert routing and cache warm-up.

The October 25 prefill checkpoint measured 1,008 tokens/s on an 8,000-token prompt with an 8K chunk, 256K context allocation and the same T16 weights. The prefill profile and receipts cover the opt-in kernels, buffer reuse, warm-up and expert placement. This is a separate measurement from the 41 tokens/s generation result above; smaller prompt updates have different throughput.

The measurement record gives build identities, quality measurements and open checks.

The model selects six of its 384 experts in each layer. dsx keeps frequently used experts on the GPUs and computes the remaining selected experts on the CPU from RAM. The CPU and GPUs work in parallel, and the main GPU combines their results. An adaptive cache follows the experts used by the current conversation.

At startup, dsx measures each GPU’s memory and host-link speed to assign its role. During prefill, experts stream from RAM over the available fast links. Follow-up requests reuse the cached conversation prefix so only new text needs prefill. DESIGN.md describes the execution and memory layout.

MIT license. See credits for upstream projects, model sources and attribution.