Skip to content

Benchmark Future Coverage

Rocco A edited this page Sep 14, 2026 · 9 revisions

Benchmark coverage still needed

MoE cache benchmarks indexes the current public results. The table below lists remaining test gaps without placeholder performance claims.

Gap Why it matters Minimum useful test
Qwen fused gate_up quant layouts Exercises a different physical bank layout NP1 and NP4, stock/cache0/positive cache, close VRAM
Qwen NVFP4 with large block-scale banks Tests slot-bound auxiliary banks 32K or 64K prompt plus 512 decode
BF16 MoE weights Separates quantized-kernel behavior from cache-control behavior NP1 prefill/decode and NP4 aggregate decode
LFM Q5 positive-cache NP4 Current build fails closed Repeat only after grouped-layout support changes
Nemotron parallel grouped execution NP1 ungated grouped execution now passes NP4 stock/cache0/cache72 with close VRAM
Multi-GPU Validates per-device cache ownership and transfer behavior Same model and prompt on one GPU and two GPUs
Physical lower-VRAM GPUs The 4/8/12 GB tiers are measured process ceilings on a 16 GB GPU Repeat the published geometry on physical 4/8/12 GB cards and record free VRAM
MTP and other drafting modes Target-only results do not measure draft acceptance or extra state Target-only vs drafting with acceptance and peak VRAM

New rows should always publish the exact server launch command, model link and hash, prompt/output geometry, sampling, raw TTFT/prefill/decode/wall numbers, and peak VRAM. A speed ratio should be highlighted only when the VRAM envelopes are close.

Clone this wiki locally