QwFNfer v0.3.0

The first release with a native Windows build, next to the Linux one.

Downloads

• qwfnfer-windows-x86_64-cuda.zip: Windows 10/11 x64. Unzip it anywhere and run qwfnfer.cmd. It carries the engine, the console, ggml (every CPU variant; CUDA for Turing to Blackwell), the CUDA 13.4 runtime and the Visual C++ runtime, so there is nothing to install but the NVIDIA driver (580 or newer) and Python 3.
• qwfnfer-linux-x86_64-cuda.zip: as before; scripts/install.sh installs it. glibc 2.35 or newer.
• SHA256SUMS for both.

On Windows, for full speed (README, FAQ, Windows?): set the NVIDIA Control Panel's CUDA - Sysmem Fallback Policy to Prefer No Sysmem Fallback for bin\qwfn-server.exe, turn on hardware-accelerated GPU scheduling, and keep the model files off a BitLocker-encrypted volume.

Changes since v0.2.3

• Windows support (#2 by @HyperRamzey, brought onto master in #16): one I/O worker pool for both systems, over unbuffered overlapped reads on Windows; the VRAM expert tier is sized from the free memory the driver reports, so it never spills silently into shared system memory; the engine opts out of Windows 11 power throttling.
• The engine never runs more CPU threads than the machine has. The default of 8 on a machine with fewer logical CPUs made every CPU graph wait on descheduled threads: 50x slower decode on the CPU in the worst case measured.
• #10: stall-probe deadlock, prefill-aware stall watchdog, reset after a failed generation, equal-prompt reuse, direct-I/O alignment. #12: the sparse attention's prefill block selection. #13: llama.cpp pinned by commit. #14: io_uring reaping and request accounting (with a further fix for a rejected request counted on the thread-pool backend).
• The server reads text files, and zip archives of them, attached to a chat (OpenAI file parts, Anthropic document blocks).
• The console's tune names a GPU whose PCIe link runs below its width or generation.
• CI on Linux and Windows checks the engine against llama.cpp, token for token, on a tiny random model with every tensor of the architecture; each release zip is unpacked and run on a machine with no GPU before it is published.

What's Changed

• README: pin llama.cpp by commit; the b10798-mix-659e406 tag was moved by @cakescats in #13
• QSA: fix prefill block selection by @ccabrerah in #12
• Correctness fixes by @ccabrerah in #10
• io: io_uring reap could wait forever, rejected requests broke the count, prefill ring depth by @cakescats in #14
• Windows support: #2 merged onto master, one io backend for both platforms, Windows performance fixes and packaging by @Apolog1ze-Dev in #16
• Add Windows support: unbuffered-overlapped io backend, platform layer, console port by @HyperRamzey in #2
• Release fixes: a manual run with a tag publishes; the engine builds with Ubuntu 22.04's liburing by @Apolog1ze-Dev in #17
• release.yml: the Linux bundle ships Ubuntu 22.04's runtime, not noble's by @Apolog1ze-Dev in #18

New Contributors

• @cakescats made their first contribution in #13
• @ccabrerah made their first contribution in #12
• @Apolog1ze-Dev made their first contribution in #16
• @HyperRamzey made their first contribution in #2

Full Changelog: v0.2.3...v0.3.0

QwFNfer v0.2.3

v0.2.2 and earlier were built in a way that stamped the engine as requiring AVX-512, from the build machine's libraries rather than from anything the engine does, so they would not start on Intel 12th–14th gen consumer parts or Zen 1–3. The bundle is now built x86-64-v3 with its runtime libraries taken from Ubuntu rather than the build host.

QwFNfer v0.2.2

Fixed console not checking properly for HF cache

QwFNfer v0.2.0

All around fixes to the engine including fully moving vision to CPU and RAM, fixing MTP head not working beyond a certain token count.
Added several more options available through the console.

QwFNfer v0.1.0

Initial release of QwFNfer
Linux only