v0.1.0 Research Preview

ds4-8gb-cpu v0.1.0 — Research Preview

Experimental CPU/NVMe inference support for running a quantized DeepSeek V4 Flash GGUF on memory-constrained Linux systems.

What this release contains

• Port of AProjQ4 CPU dense-attention support to the current upstream ds4 base.
• Opt-in DS4_CPU_NO_PREFETCH=1 demand-paging path for memory-constrained CPU inference.
• Hardware, methodology, benchmark, and reproducibility documentation.
• Evidence register covering E-001 through E-012.
• Sanitized V16.1 Linux swappiness benchmark summary.

Tested configuration

• Intel Core i7-8550U.
• CPU-only execution; no GPU used for the documented 8GB validation.
• 7.7 GiB physical RAM.
• NVMe-backed mmap/demand paging.
• DeepSeek V4 Flash AProjQ4 GGUF: 84,420,584,032 bytes (78.62 GiB).
• 284.33B logical model parameters.

Evidence highlights

• The current public candidate completed the controlled cold whole-model first-token diagnostic in 5.33 seconds with 6,204,632 KiB maximum RSS and zero process swaps.
• Historical V16.1 testing found swappiness 100 versus 10 reduced pooled decode latency by 9.11% across four of four paired runs on the research branch. This result is explicitly classified as STRONG-HISTORICAL and has not yet been reproduced on the current public candidate.
• The historical V4 expert-major prefetch effect did not reproduce reliably against the latest upstream base and is excluded from the v0.1 default.

Scope and limitations

• This is quantized inference, not full-precision model execution.
• The model does not fit in physical RAM; weights remain mmap-backed and are faulted from storage as required.
• Model weights are not included in this repository or release.
• The 5.33-second result is a controlled first-token diagnostic, not a sustained production-generation throughput claim.
• Performance is hardware, storage, kernel, and workload dependent.
• Historical experimental results are clearly separated from current-public validation in the evidence register.

See docs/evidence-register.md, docs/methodology.md, docs/hardware.md, and docs/benchmarks.md for the complete evidence boundaries.

Credits

This project is an evidence-driven fork of antirez/ds4.

AProjQ4 support originates from the work by oppo99 (Giorgio Oppo); the v0.1 branch ports the CPU dense-attention support to the selected current upstream base.