Repository navigation
QFN WV TokenWing Plugin v1.1 (Linux x86_64 .so + Windows x86_64 .dll)
Prebuilt GGML_QUANT_PLUGIN_ABI_VERSION = 1 runtime & converter plugins (--quant-plugin) for running WV GGUFs (WV-v5-2.26bpw and WV-v5-3.5bpw EXT0..EXT5) in TokenWing on both GPU (sm_75, sm_80, sm_86, sm_89, sm_90, sm_120a + compute_120 PTX, static cudart) and CPU (Scalar + AVX2+FMA + AVX-512BW+VL+VNNI runtime dispatch, on-load slot repacking --moe-stream-cpu-repack 1, and 4-row vec_dot_x4 --moe-stream-cpu-x4 2).
Highlights in wv-plugin-v1.1
- 6 External Quant Types (
EXT0..EXT5): Addswv_swz3_g16(3.59 bpw),wv_vq4_u8(2.26 bpwcold MoE + nativedown_exps),wv_swz4_r12(4.50 bpwrotated spine), andwv_swz6_r12(6.06 bpwrotated LM head / critical spine). - On-Load I/O-Thread VNNI Repack +
vec_dot_x4: Background NVMe reader threads pre-swizzle streamed2.26 bpwslices once on arrival (0.1418 ms/matmul10-expert /0.0999 ms/matmul7-expert 8-thread AVX-512 VNNI). - Zero-Bank-Conflict SM120 CUDA Kernels & Speculative MTP:
71.42 tok/sfull-VRAM GPU decode with--spec-type draft-mtp(55.74 tok/sB=1,103.34 tok/spp512), and27.77 tok/s(tech) /26.50 tok/s(code) under<11 GiBVRAM +6 GiBRAM coldO_DIRECTNVMe streaming (82.35 MiB/tokwithH=96hybrid VRAM experts). - Portability & Symbol Hygiene: Linux
.sobuilt onubuntu-22.04(GLIBC <= 2.34), Windows.dllbuilt withMSVC 19.44 + CUDA 12.8, staticcudart, singleggml_quant_plugin_get_apiC-ABI export.
SHA-256 Checksums
2285d62f00595e22de9d44e3e03ba0ef0bd07efd66ff90aab83119eae3fb30c9 libqfn_wv_plugin.so
6a62a3193019ba4e9e4e8c7cacc2d9d2e8f1dfca75f1985d6651d977ffcb0f80 qfn_wv_plugin.dll