·
0 commits
to optimize/adaptive-q8-sparse-gather
since this release
IMPORTANT: FP8 KV warning
Do not use FP8 KV cache when planning to exceed 100K context. Use F16 KV for long-context operation.
Validated Strix Halo Vulkan release for the custom DeepSeek V4 TM64 target and selective-Q4 DSpark drafter.
Model
- Target GGUF: erayyapagci/DeepSeek-V4-Flash-IQ2XXS-TM64-Q4KAttention-v1
- Models are not bundled in this release.
Build provenance
- Source commit prepared for this fork:
4fc4f230f6cc7735dc05cd403a8eb73ccd64813c - Local source base:
b54f16e18ba3bd6e34b87a1ef827ca985307cb5b - Accepted patch SHA-256:
3cacee17358f178f4c2a6c02c95c61426fec52e9f04db2b05a2d1de28c4cd623 - Runtime archive SHA-256:
2bd5eb87c7a855959bbbc6d652378a025a5e6c9bd38a6b9837ceaf963cae46da - Source archive SHA-256:
28e0964545eef6b515b1ba60cc371356245df17e779da3cc83d2994a58988bf1 - Build: Release, Vulkan/RADV, Linux x86-64, AMD Strix Halo/gfx1151
The complete patched source tree is provided as a release asset. The runtime archive also includes the accepted patch, exact diff, CMake configuration, provenance, checksums, and validation logs.
Included
llama-server,llama-cli,llama-bench, quantizer, tests, and common tools- Bundled validated RADV runtime
- Tuned
bin/run-server.shreproducing the production configuration - NP1/NP2/NP3 support with state isolation
- F16 KV and F16 lightning-indexer defaults
- FP8-over-100K launcher safety gate
- Optional slot persistence and systemd example
- Complete patched source snapshot
See the packaged README.md for tuned options, setup, safety constraints, and rollback controls.
Validated no-speculation benchmark snapshot
| Metric | 0K | 64K |
|---|---|---|
| 2,048-token prefill | 260.90 tok/s | 199.04 tok/s |
| 128-token generation | 20.62 tok/s | 16.82 tok/s |