0.93

Delta Inference Engine 0.93

New: a model picker at first start, and a Setup tab. The first start now asks which model you want and offers the one that suits your RAM, so nobody has to read a table before a 300 GB download. The Setup tab in the chat page changes the model and the settings without a batch file or a text editor.

New: the default from 128 GB of RAM up is Unsloth's UD-IQ2_XXS. At 192 GB it writes about as fast as the ds4 Q2 GGUF, and at 128 GB about 5% slower (34.0 vs 35.9 tok/s). It agrees with DeepSeek's own model more often and gets more of the hard questions right, from a 65 GB smaller download. The first start offers it on 120 GB or more. ds4 Q2 stays one answer away.

More models, all measured the same way. Unsloth's UD-IQ1_S, UD-IQ2_XXS, UD-Q2_K_XL and UD-IQ3_XXS each run at full speed for their size. Each one needed its own work: faster CPU kernels for the 1-bit files, the Q5_K Engram tables read as they are, and memory settings per model.

Writing speed in tok/s on an RTX 3090, greedy, and agreement with DeepSeek's own model at the settings that ship:

Model	Size	192 GB	128 GB	96 GB	64 GB	General	Code + maths	Agentic
Unsloth UD-IQ2_XXS	301 GB	41	34.0	29.3	21.0	91.1%	94.5%	95.9%
ds4 Q2	366 GB	40.7	35.9	27	20	89.9%	93.1%	95.1%
Unsloth UD-IQ1_S	233 GB	41.0	40.6	36.5	30.6	89.1%	91.5%	95.3%
Unsloth UD-Q2_K_XL	335 GB	31.3	26.6	22.9	15.2	92.5%	95.3%	97.0%
Unsloth UD-IQ3_XXS	395 GB + the default	28.2	22.8	18.3	11.7	92.7%	95.9%	97.4%

At the default sampling (temperature 1, top-p 0.95) UD-IQ2_XXS writes 39.7 tok/s and ds4 Q2 40.3.

On 99 hard questions, greedy: UD-IQ2_XXS 57, ds4 Q2 52, DeepSeek's API 59, Flash-Next 44. The method is in BENCHMARKS.md.

New: 64 and 96 GB of RAM are supported. Under about 110 GB the first start offers UD-IQ1_S. The launcher's memory reserve, SSD read-ahead and prompt reading were all tuned for these PCs. New prompts take longer there: about 4.3 s for 180 tokens at 96 GB and 8.3 s at 64 GB.

Sampling is faster. The drafter's guesses are now sampled instead of taken as its top pick, which cut the slowdown at the default sampling from about 12% to about 1% on ds4 Q2 and 3% on UD-IQ2_XXS. The answers' distribution is unchanged.

New: a second GPU. A Multi-GPU switch on the Setup tab runs a helper on a second card of 8 GB or more. With an RTX 3060 beside the 3090 it adds about 7% writing speed. It does not split the model across cards, so for two or more big cards use Project Maya.

New: EXL3 2.5 bpw as an option. It is in Choose Model.bat, for 192 GB of RAM and an internal NVMe, and its GPU kernel is about twice as fast as the first version. UD-Q2_K_XL agrees more and writes faster, so prefer that one.

The route bias is 0.28 on every PC. It was tested at 0.10, which costs about 18% of writing speed at 192 GB and more on smaller PCs. The agreement numbers above are at 0.28.

Reading long prompts. On UD-IQ2_XXS, long prompts read at up to 740 tok/s at 192 GB and 600 at 128 GB (at ~64K tokens).

The engine is now built in public. The release zips come from .github/workflows/release.yml on GitHub's Windows runners, so anyone can read how they were made. Each zip has a build attestation:

gh attestation verify build-ds41-portable.zip --repo musicurgy/delta-inference-engine

Fixed and changed:

• Tool calls that a low-bit model spells oddly are now read as calls, not text.
• A turn cut off inside a tool call no longer sends raw tool-call text.
• A </think> the model writes in prose no longer ends the reasoning.
• Long contexts past 65,535 tokens are padded correctly.
• A failed image encoder no longer stops the model.
• A blank replayed reasoning no longer breaks the cached prompt.
• Streamed replies are sent straight away instead of waiting in the network buffer.
• The server no longer misses a queued request's wake-up.
• The engine survives a long pause between turns (a CUDA graph fix from llama.cpp).
• UD-IQ3_XXS no longer runs out of memory on its first prompt.
• Short prompts read about 40% faster on 192 GB PCs, and turns of 64 or more new tokens take the faster path.
• The README now shows a Windows Defender exclusion for the model folder and the engine: about 20% faster random reads from the model drive, which helps most on 64 and 96 GB PCs.

If you already have UD-IQ1_S: Unsloth re-made these files on 10-28, and the new ones agree 1.3-1.6 points more at the same size. Your older files keep working. To get the new ones, delete files 1 to 4 in models\unsloth\UD-IQ1_S with their .ok files, then pick the download in Choose Model.bat. File 5 is the same in both, so keep it.

Upgrading from 0.92: git pull in the folder (or download the new ZIP and unpack it over the old one), then double-click Start DeepSeek.bat. It offers this release's engine by itself. Your model files stay as they are.

SHA-256 checksums are in SHA256SUMS.txt. The zips include NVIDIA CUDA runtime libraries (cudart, cublas, cublasLt), redistributed under the NVIDIA CUDA Toolkit EULA.

Which zip? Almost every PC wants build-ds41-portable.zip (any CPU with AVX2: Intel 2013+, every AMD Ryzen). build-ds41-compat-avx.zip is for Sandy / Ivy Bridge, build-ds41-compat.zip for anything older. You only need one if you set up by hand: download the source code (or git pull) and double-click Start DeepSeek.bat, and it fetches the right engine for your CPU by itself.