QwFNfer v0.3.0

The first release with a native Windows build, next to the Linux one.

Downloads

• qwfnfer-windows-x86_64-cuda.zip: Windows 10/11 x64. Unzip it anywhere and run qwfnfer.cmd. It carries the engine, the console, ggml (every CPU variant; CUDA for Turing to Blackwell), the CUDA 13.4 runtime and the Visual C++ runtime, so there is nothing to install but the NVIDIA driver (580 or newer) and Python 3.
• qwfnfer-linux-x86_64-cuda.zip: as before; scripts/install.sh installs it. glibc 2.35 or newer.
• SHA256SUMS for both.

On Windows, for full speed (README, FAQ, Windows?): set the NVIDIA Control Panel's CUDA - Sysmem Fallback Policy to Prefer No Sysmem Fallback for bin\qwfn-server.exe, turn on hardware-accelerated GPU scheduling, and keep the model files off a BitLocker-encrypted volume.

Changes since v0.2.3

• Windows support (#2 by @HyperRamzey, brought onto master in #16): one I/O worker pool for both systems, over unbuffered overlapped reads on Windows; the VRAM expert tier is sized from the free memory the driver reports, so it never spills silently into shared system memory; the engine opts out of Windows 11 power throttling.
• The engine never runs more CPU threads than the machine has. The default of 8 on a machine with fewer logical CPUs made every CPU graph wait on descheduled threads: 50x slower decode on the CPU in the worst case measured.
• #10: stall-probe deadlock, prefill-aware stall watchdog, reset after a failed generation, equal-prompt reuse, direct-I/O alignment. #12: the sparse attention's prefill block selection. #13: llama.cpp pinned by commit. #14: io_uring reaping and request accounting (with a further fix for a rejected request counted on the thread-pool backend).
• The server reads text files, and zip archives of them, attached to a chat (OpenAI file parts, Anthropic document blocks).
• The console's tune names a GPU whose PCIe link runs below its width or generation.
• CI on Linux and Windows checks the engine against llama.cpp, token for token, on a tiny random model with every tensor of the architecture; each release zip is unpacked and run on a machine with no GPU before it is published.

What's Changed

• README: pin llama.cpp by commit; the b10798-mix-659e406 tag was moved by @cakescats in #13
• QSA: fix prefill block selection by @ccabrerah in #12
• Correctness fixes by @ccabrerah in #10
• io: io_uring reap could wait forever, rejected requests broke the count, prefill ring depth by @cakescats in #14
• Windows support: #2 merged onto master, one io backend for both platforms, Windows performance fixes and packaging by @Apolog1ze-Dev in #16
• Add Windows support: unbuffered-overlapped io backend, platform layer, console port by @HyperRamzey in #2
• Release fixes: a manual run with a tag publishes; the engine builds with Ubuntu 22.04's liburing by @Apolog1ze-Dev in #17
• release.yml: the Linux bundle ships Ubuntu 22.04's runtime, not noble's by @Apolog1ze-Dev in #18

New Contributors

• @cakescats made their first contribution in #13
• @ccabrerah made their first contribution in #12
• @Apolog1ze-Dev made their first contribution in #16
• @HyperRamzey made their first contribution in #2

Full Changelog: v0.2.3...v0.3.0