forked from ggml-org/llama.cpp
-
Notifications
You must be signed in to change notification settings - Fork 16
Benchmark Future Coverage
Rocco A edited this page Sep 14, 2026
·
9 revisions
MoE cache benchmarks indexes the current public results. The table below lists remaining test gaps without placeholder performance claims.
| Gap | Why it matters | Minimum useful test |
|---|---|---|
Qwen fused gate_up quant layouts |
Exercises a different physical bank layout | NP1 and NP4, stock/cache0/positive cache, close VRAM |
| Qwen NVFP4 with large block-scale banks | Tests slot-bound auxiliary banks | 32K or 64K prompt plus 512 decode |
| BF16 MoE weights | Separates quantized-kernel behavior from cache-control behavior | NP1 prefill/decode and NP4 aggregate decode |
| LFM Q5 positive-cache NP4 | Current build fails closed | Repeat only after grouped-layout support changes |
| Nemotron parallel grouped execution | NP1 ungated grouped execution now passes | NP4 stock/cache0/cache72 with close VRAM |
| Multi-GPU | Validates per-device cache ownership and transfer behavior | Same model and prompt on one GPU and two GPUs |
| Physical lower-VRAM GPUs | The 4/8/12 GB tiers are measured process ceilings on a 16 GB GPU | Repeat the published geometry on physical 4/8/12 GB cards and record free VRAM |
| MTP and other drafting modes | Target-only results do not measure draft acceptance or extra state | Target-only vs drafting with acceptance and peak VRAM |
New rows should always publish the exact server launch command, model link and hash, prompt/output geometry, sampling, raw TTFT/prefill/decode/wall numbers, and peak VRAM. A speed ratio should be highlighted only when the VRAM envelopes are close.
GenerelSchwerz llama.cpp
- Home
- Discord community
- Contributors
- Complete feature index
- Hardware setup guides
- Owner-verified evidence
- Notable runs
- Benchmark comparison showcase
- BeeLlama Main
- Llama Main
- Llama Dev
- MoE Cache
Feature groups
- Memory placement and workspace
- Validation and diagnostics
- CUDA MoE cache and helpers
- Grouped MoE drafting
Source branches