Skip to content

[FR] Make GGML_CUDA_FA_ALL_QUANTS the default or add runtime warning for missing quantized FA kernels #24485

Description

@mirkowodtke

Problem

When --flash-attn on is used with quantized KV cache types (e.g. --cache-type-k q8_0 --cache-type-v q4_0), and llama.cpp was built without -DGGML_CUDA_FA_ALL_QUANTS=ON, the server silently falls back to CPU-based attention. This results in:

  • Dramatically worse prefill speed (25-45x slower in our tests)
  • GPU shows 0% utilization during inference
  • No visible error or warning to the user

This was confirmed in #20866 and by our testing with Gemma 4 12B QAT + MTP (head_size=512, SWA, Blackwell RTX 5070 Ti).

Suggestion

Two options (either would be a big improvement):

Option A: Enable FA_ALL_QUANTS by default
Add -DGGML_CUDA_FA_ALL_QUANTS=ON to the default CMake configuration. The build time increase is a one-time cost that most users would accept for 25x performance with quantized KV cache.

Option B: Add a runtime warning
In llama-server, when --flash-attn on is set with a quantized KV cache type, check if the appropriate CUDA FA kernel was compiled. If not, emit a clear warning:

Warning: Flash attention with quantized KV cache requires building with -DGGML_CUDA_FA_ALL_QUANTS=ON. 
Using CPU fallback for attention which will be extremely slow. 
Consider using --cache-type-k f16 --cache-type-v f16 instead.

Our Test Environment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions