Repository navigation
v0.4.6 — Tensor splitting with quantized KV cache
Fix the launcher blocking llama.cpp tensor splitting when K/V cache uses q8_0 or q4_0. Recent llama.cpp builds support this combination; Flash Attention on/auto remains required.
- Preserve selected cache types and GPU proportions in generated commands.
- Explain that NVFP4 model weights and KV-cache quantization are separate settings.
- Warn that older engines may need an update, layer splitting, or f16 cache.
- Add builder, advisor, and API regressions. Tensor placement still reports unknown fit.
Merged fix: PR #22. Package and runtime versions are now 0.4.6.
Validation: 430 tests passed on the release tree (one inherited deprecation warning). No real-model GPU inference or performance benchmark was run. Publishing this release does not restart or update the running GUI.