🚀 Cache-DiT v1.5.1 Release Notes
Release Date: 2026-09-01
Comparison: v1.5.0...v1.5.1 (45 commits, 43 PRs, 7 contributors)
📋 Overview
Cache-DiT v1.5.1 is a feature release spanning ~2.5 months (2026-06-16 ~ 2026-09-01), covering 43 PRs. The headline feature is the FFPA CUDA attention backends (fp16/fp8/fp4) for Blackwell GPUs with native zero-copy NHD layouts. The release also adds 7 new model integrations (boogu-image, ernie-image, Joy-Image-Edit, AnyFlow & AnyFlow-FAR, Krea-2, Bria-Fibo, Cosmos3 Omni), a Sage3 Blackwell attention backend, torchao NVFP4 dynamic quantization, an alternating SVD refinement option for SVDQuant, the hyper-parameter-free FoCa calibrator, Context-Parallelism communication optimization for boogu-image, and Ascend NPU profiling support.
✨ Core Highlights
1. ⚡ FFPA CUDA Attention Backends
New attention backends built on ffpa-attn (Flash Prefill Attention), delivering quantized CuTe TMA kernels for sm_120 (Blackwell) GPUs — forward inference only, Context Parallelism compatible:
• --attn ffpa: CuTe TMA fp16/bf16 kernels
• --attn ffpa_fp8: FP8 quantized attention, defaults to the highest-precision quant config (int8 QK MMA + fp16 PV accumulation, per-thread Q/K + per-channel V scales on consumer GPUs)
• --attn ffpa_fp4: NVFP4 quantized attention
• --attn ffpa_fp8_per_block: per-block quantized FP8 variant (fastest, lower precision)
Fine-grained CLI controls: --ffpa-hybrid (hybrid fp16 early-rows stage, default off for every backend), --ffpa-hybrid-n-early, --ffpa-fp8-hadamard / --ffpa-fp4-hadamard (Hadamard pre-rotation). The enable_fp8 + enable_fp4 combination is explicitly rejected with a clear error.
Native NHD layouts (#1104–#1107): all CuTe families (fp16/fp8/fp4) now consume NHD ([B, H, N, D]) inputs natively — both packed and strided — through the fp8 persist-D tensor_layout fast path, eliminating the BHSD↔NHD permute + .contiguous() materialization overhead on every forward.
Performance (FLUX.1-dev, 2048×2048, single RTX PRO 5000, 28 steps; E2E latency / PSNR / SSIM vs BF16 baseline, measured before the NHD fast path landed):
| Backend | Latency | Speedup | PSNR / SSIM |
|---|---|---|---|
| BF16 baseline | 92.46s | 1.00× | — |
| sage | 81.09s | 1.14× | 40.2 dB / 0.986 |
| sage3 | 80.73s | 1.15× | 31.4 dB / 0.869 |
| ffpa (fp16) | 93.24s | ~1.00× | 49.3 dB / 0.997 |
| ffpa_fp8 | 83.64s | 1.11× | 32.7 dB / 0.919 |
| ffpa_fp4 | 78.66s | 1.17× | 33.6 dB / 0.949 |
With 2×GPU Ulysses Context Parallelism: ffpa_fp8 48.42s, ffpa_fp4 45.89s. The fp16 row reflects the pre-NHD permute overhead, which the native NHD path (#1104–#1107) removes.
PRs: #1098, #1099, #1100, #1101, #1103, #1104, #1105, #1106, #1107
2. 🧩 New Model Integrations
Seven new pipelines gain Cache + Parallelism support (cache adapters, CP/TP wiring, and cache_dit.generate CLI examples):
• boogu-image (#1072) — plus a dedicated CP optimization: reduced KV all-2-all communication overhead for boogu (#1082) and per-model attention backend dispatch (#1083)
• ernie-image (#1073)
• Joy-Image-Edit (#1079)
• AnyFlow & AnyFlow-FAR (#1081)
• Krea-2 (#1074)
• Bria-Fibo (#1075)
• Cosmos3 (Omni) (#1088, community contribution by @johnnynunez) — new Cosmos3OmniPatchFunctor (functor_cosmos3.py) reorders the MoT dual-stream contract so DBCache tracks the actually-denoised gen stream (und is quasi-static and must not drive residual-diff decisions). Measured on Cosmos3-Nano T2V (RTX PRO 6000 Blackwell, 189 frames / 35 steps): 480p 138.59s → 76.75s (1.81×) with DBCache rdt=0.20, 2.04× with torch.compile; 256p 1.53×. Requires diffusers main for Cosmos3OmniPipeline.
3. 🔥 Sage3 Blackwell Attention Backend
• --attn sage3 (#1085): SageAttention-3 backend for sm_120 (Blackwell) GPUs
• Fixed the sage3 (sm_120) tensor layout (#1086)
• Made sage3-attn compatible with torch.compile (#1087)
4. 💎 Quantization Enhancements
• torchao NVFP4 dynamic quantization (#1090): new quant types nvfp4 and nvfp4_weight_only on the torchao backend for Blackwell GPUs. torchao>=0.17.0 is now required for the quantization extra.
• Alternating SVD refinement for SVDQuant (#1095, community contribution by @Rudin6): new opt-in svd_refine_iters knob. Each round refits the low-rank factors against smoothed_weight - dequant(quantize(residual)) so the low-rank branch absorbs the residual quantizer's error pattern. Default 0 is bitwise-identical to the previous one-shot behaviour (verified across 72 configurations). Measured on 14 PixArt-Sigma layers with real calibration activations: 5 rounds cut weight error by 7.8–8.9% and layer output error by 17.4–18.2% for both INT4 and NVFP4.
• SVDQ W4A4 pad-to-128 (#1084): pad SVDQ linear layers to a multiple of 128 so the W4A4 kernel covers shapes that are not tile-aligned.
• Fix: clamp torchao float8 zero scales to avoid 0/0 NaN (#1097) — previously, all-zero scale rows produced NaNs on dequantization.
• Cleaner quantization logging format (#1091)
5. 🔮 FoCa Calibrator
New FoCaCalibrator (#1066): forecasts cached tensors with a BDF2 predictor + Heun corrector. It is a drop-in alternative to TaylorSeerCalibrator and DMDCalibrator with zero hyper-parameters — step intervals are derived automatically from the step counters.
6. 📈 Parallelism & Profiling
• CP communication optimization (#1082): reduced boogu-image KV all-2-all communication overhead on the Context-Parallelism path.
• Ascend NPU profiling (#1096, with @ChangeTheWay and @ooooooye): cache_dit.profiler now supports torch_npu.profiler for Ascend NPUs, mirroring the torch.profiler API.
🔧 Other Enhancements
• CLI: support extra input kwargs (#1068); allow using local test data (#1092); new cache+dmd+svdq YAML example configs (#1069)
• Docs: Cache-DiT tech report added (#1062); DMD calibrator design docs (#1063, #1064, #1065); docs rendering fix (#1067); SVDQ NVFP4 kernel comments (#1071)
• Community: new community integration links in README (#1108)
• Dev experience: agent workflows + model-integration skill updates (#1074, #1075, #1076, #1080); skills relocated to .github/skills (#1109); example data updates (#1077, #1078)
🐛 Bug Fixes
• sage3 (sm_120) attention backend tensor layout (#1086)
• torchao float8 zero scales clamped to avoid 0/0 NaN (#1097)
• ffpa_fp8_per_block V quantization switched to per-channel for 2048-resolution precision (#1103)
⬆️ Dependencies
• torchao>=0.14.1 → torchao>=0.17.0 (quantization extra; required for NVFP4 dynamic quantization)
👥 Contributors
Thanks to everyone who contributed to this release:
@DefTruth, @Rudin6 (first contribution), @johnnynunez (first contribution), @changetheway, @ooooooye, Copilot
