Repository navigation
🚀 Cache-DiT v1.5.5 Release Notes
Release Date: 2026-11-18
Comparison: v1.5.3...v1.5.5 (187 commits)
Full Changelog: GitHub Releases
📋 Overview
Cache-DiT v1.5.5 spans a month (2026-10-15 ~ 2026-11-18) and is organized around five threads: model support (25 new pipeline families, 🤗Diffusers coverage up to 0.43.0), the kernel and quantization stack (SVDQuant CuTe/NVFP4 with fused MLP, native FP8 and FP4 on CUTLASS collectives, fused memory-bound block operators, one _C_ops extension with no backend-selection layer), the distributed refactor (communicate / parallel split), SVD-Cache, and the CLI / package layout cleanup (_utils → cli).
✨ Core Highlights
1. 🧩 Model Support: 🤗Diffusers up to 0.43.0 and Built-in Pipelines
Model coverage now spans 🤗Diffusers releases from 0.12.0 to 0.43.0 (see the pipeline matrix in the README), plus built-in pipelines for models that have no 🤗Diffusers implementation. Every entry was ported with the full capability matrix — Cache, CP/TP, quantization and offload — and validated end to end.
| Modality | Model | Notes |
|---|---|---|
| Video | MiniMax-H3 (T2VA / FL2VA / Ref2VA) | cache, CP, TP, AdaLN host residency |
| Video | Fast-H3 | few-step DMD2 students with VSA |
| Video | SANA-Video 2.0 (720p) | 5B |
| Video | SANA-Video 2B (T2V/I2V) | cache, CP, TP, quantization |
| Video | SANA-WM (I2V) | camera-controlled |
| Video | LingBot-Video Dense 1.3B | vendored 5-file pipeline |
| Video | LongLive 2.0 (5B) | cache, CP, TP, quantization |
| Video | LongCat-Video-Avatar 1.5 (A2V/AI2V) | cache, CP/TP, quantization |
| Video | MAGI-2 Preview | 114B MoE audio-video DiT, cache, CP/TP, offload, decoders |
| Video | MOVA 360p (T2VA/I2VA) | cache, CP, TP |
| Video | HY-Nomas | segment-AR camera-controlled, cache, CP, TP |
| Video | Wan-Animate-2 | base + distilled |
| Video | Pika H3 Flex (T2VA/FL2VA/REF2VA) | cache, CP, TP, fp8; plus a diffusers-native modular pipeline |
| Video | CausVid-Wan | Wan2.1-T2V-1.3B causal student |
| Video | LTX-2.5 | cache, TP, CP, fp8/SVDQuant, compile |
| Image | HiDream-O1-Image | dense CP mask, vision-conditioned |
| Image | Ming-Image 0.1 | design variants, MLLM condition extraction |
| Image | PRXPixel | cache, CP, TP, TE-P, quantization, compile |
| Image | FireRed-Image-Edit 1.1 | also fixes the Qwen-Image 1.x TP head_dim |
| Image | Boogu-Image | native, on the _C_ops -> cache_dit::models kernel family |
| Image | MiniMax-H3-Image (T2I/Edit) | exact AdaLN plan memoization |
| Image | P-H3-Image (T2I/Edit) | 8-step MiniMax-H3 based |
| Image | FLUX.3 | diffusers modular pipelines |
| Audio | Stable Audio 3 | text-to-audio |
| Audio | LongCat-AudioDiT | TTS, plus audio metrics |
Built-in pipelines (those beyond 🤗Diffusers) are listed in the README with their minimum cache-dit version.
2. ⚡ Kernel & Quantization Stack
💎 SVDQuant W4A4
- CuTe NVFP4 grew into a first-class path next to the native one: a unified runtime API (
runtime+lora_layout), split-K producer with rank/M dispatch, a simplified FP4 mainloop, and a tuned row-major LoRA schedule. - Fused MLP and SwiGLU (#1332): a fused epilogue (qout/oscales, both D layouts,
write_dense), producer-side SwiGLU on both producer families, androw_majoras a first-class fused path. - Chain-level tuning (#1343, #1344): vectorized and paired fused-quantize stores, pipelined B-operand loads in the D epilogue,
kFuseDownspecialization, and a geometry-gated large-K fc2 path. Full-M fc2 now reaches 0.948x / 0.965x of the native kernel at M=4096 / 8192. - FlashInfer chain, opt-in (#1308, #1310):
runtime_kernel="flashinfer+cute-dsl"or"flashinfer+cutlass-sm120"; ranks the chain cannot cover are rejected instead of silently downgraded. Replacing the exactamax(x * smooth)global scale with a safe upper bound cuts the per-layer cost from 127 µs to 48 µs. --svdq-lora-layoutis exposed in thegenerateCLI.
🎛️ FP8
- Native FP8 backend (#1302): CUDA quantize plus a warp-specialized GEMM with split row/tensor kernels, and activation quantize fused through
torch.compile. - CUTLASS SM120 collectives for every mode: per-row, per-tensor and per-block now share one mainloop with automatic padding/depad and a unified
fp8_quantize_gemm. Theuse_wsknob is gone — CUTLASS selects the mainloop and the warp specialization;use_native=Falsestill selects thetorch._scaled_mmreference for the shapes it supports.
🔢 FP4
- Native NVFP4 backend with
torch._scaled_mmas the default GEMM, an SM120 CuTe GEMM family and a native NVFP4 Linear. - Then CUTLASS 4.8.1 collectives only: the hand-written FP4 GEMM and its
use_wswere removed, entry points renamed tofp4_gemm_cutlass_2x (two stages quantize)/fp4_gemm_cutlass_1x (one stage quantize), with automatic padding for unsupported shapes.
🧬 Memory-bound block-operator fusion
Second-pass pipelines that keep the diffusers modules and state dict, and rebind only the block/attention seams to fused CUDA operators:
| Model | Measured (RTX PRO 5000, CUDA events) |
|---|---|
| FLUX.2-Klein (#1298) | eager 228.79 → 196.44 ms/step (−14.1%), compile −0.3%; fused+compile vs stock+compile 46.75 dB / 0.9970 against compile's own drift of 46.70 dB / 0.9971 |
| FLUX.3-Klein (#1299) | eager 290.5 → 241.1 ms (−17.0%), compile ±0; e2e 4.83 → 4.03 s, --cache 3.78 → 3.17 s; qk_norm_rope 28.0 µs at 83.5% of DRAM peak vs inductor's 75.2 µs at 71.8% |
| Qwen-Image 2.1 (#1274) | eager 403.0 → 355.3 ms/step (−11.8%), CP2 −7.3%; 43.74 dB / 0.9920 vs stock+compile |
Every fused-op family is controlled by one switch, CACHE_DIT_FUSED_OPS (#1301).
📦 One extension, one backend
- The four CUDA extensions were merged into a single
_C_opsmodule (#1288) with one pybind entry point and per-family registrars; the build has one source list, one arch predicate (>=sm89, excludingsm90) and the default arch set("89", "120a"). - The backend-selection layer is gone:
kernels/backend.py,kernels/cuda/,kernels/triton/,kernels/cutedsl/andkernels/ops.pywere removed. CUDA is the only implementation, and there is no runtime choice between Triton, CuTe DSL and CUDA. svdq_extension_is_availablewas shortened tosvdq_is_available.
3. 🔀 Distributed Module Refactor
distributed/core/was split by responsibility and deleted, with no compatibility shim (#1290):communicate/primitives.pyowns the all-to-all / Ring p2p collectives, head pad-unpad helpers, comm metadata and the FP8 comm kernel bindings;parallel/owns the CP config/plan types, the hook registry, the CP runtime,ulysses/ring/uspand the diffusers bridge. Call sites import fromdistributed.parallelordistributed.communicateby symbol ownership.async_ulyssesmoved underdistributed/transformers/, which is the only place it ever patched.- Ulysses communication: per-chunk 16-byte alignment for the FP8 and BF16 all-to-all paths, the FP8 O-permute fused with quantize/dequantize, K-FP8 turned into an explicit opt-in, the communication kernels reorganized under
csrc/kernels/ulyssesasulysses_fp8_*, read-only NUMA topology diagnostics, and Ring K/V receive buffers reused. - BF16 alignment padding is computed from the last dimension alone, so the common aligned case costs no list traversal.
4. 🧠 SVD-Cache Calibrator
A subspace-split calibrator (#1340): the cached block tensor is decomposed in the token × channel plane into a low-rank principal part and a high-frequency residual, the principal part is smoothed across steps and the residual is frozen. The basis is fitted online from the first full-compute feature of the current inference, so there is no offline calibration artifact (arXiv:2601.07396).
Two predictors are available: ema (the paper's per-step smoothing) and taylor (SVD plus Taylor extrapolation).
Measured status (FLUX.1-dev, 1024², 28 steps, seed 0, RTX PRO 5000) — treat SVD-Cache as reproduction/experimental support: on this harness neither predictor beats plain reuse often enough to recommend it.
| Configuration (rdt 0.12, 14/28 cached, ~1.9x) | latency | PSNR (dB) | SSIM |
|---|---|---|---|
| DBCache (plain reuse) | 8.85 s | 23.11 | 0.8848 |
| + TaylorSeer | 8.86 s | 24.21 | 0.9123 |
| + DMD | 8.91 s | 23.49 | 0.9074 |
+ SVD-Cache ema (β 0.9) |
9.48 s | 20.62 | 0.8051 |
+ SVD-Cache taylor (order 1) |
9.47 s | 23.38 | 0.8882 |
+ SVD-Cache taylor (order 1, τ 0.999) |
9.51 s | 23.81 | 0.8990 |
For production use --calibrator taylorseer (or dmd).
Calibrator selection is now unified: --calibrator {taylorseer,dmd,foca,svd} plus --calibrator-params "k=v,k=v". The legacy --taylorseer / --dmd / --foca flags remain as aliases, and conflicting selections fail instead of silently winning.
5. 🧰 CLI and Package Layout
src/cache_dit/_utilsis nowsrc/cache_dit/cli(#1291): argument parsing, example factories, example registration and the entrypoint. The old name hid the command-line surface behind a generic private name.- Module and package names lost their leading underscores (#1292, #1293) — for example
attention/_interface.py→attention/interface.py,attention/_backend_selector.py→attention/backend_selector.py. Class-level_{NAME}naming is untouched: it still marks "subclass of diffusers'NAME".
🔧 Other Enhancements
- Profiler: windowed Torch/Nsight collection plus NVTX stage scopes and module hooks, wired into the MiniMax-H3 steps (#1326).
- Metrics: PSNR inputs are cast to float64, so a uint8 tensor no longer wraps around (#1335). Audio metrics were added with LongCat-AudioDiT.
- Quantization: opt-in per-layer accuracy observers (#1216).
- Registry: owned adapters are scoped to cache-dit's own pipelines, so a foreign adapter cannot be claimed (#1320).
- Build: the redundant
[tool.setuptools_scm]table was removed frompyproject.toml(it silently lost tosetup.py) (#1317); the release tools share one workspace preparation step and one wheel inspector. - Docs and skills: a usage guide for the built-in pipelines, a model-integration runbook, the downstream-compatibility audit skill, the cache-algorithm integration skill, and an operator reference for the FP8/FP4 contracts.
⚠️ Upgrade Notes
| Area | Change | Action |
|---|---|---|
| Package | cache_dit._utils → cache_dit.cli |
update imports |
| Package | leading underscores dropped from module/package names | update imports (cache_dit.attention.interface, cache_dit.attention.backend_selector, …) |
| Extension | four CUDA extensions merged into one _C_ops |
rebuild or reinstall the extension; a stale .so is not compatible |
| Kernels | kernels/backend.py and the Triton / CuTe DSL backends were removed |
CUDA is the only implementation; drop backend selection |
| Kernels | svdq_extension_is_available → svdq_is_available |
rename |
| FP8 | fp8_gemm(..., use_ws=...) was removed |
drop the kwarg; CUTLASS selects the mainloop |
| FP4 | hand-written GEMM removed | use fp4_gemm_cutlass_2x / fp4_gemm_cutlass_1x |
| Distributed | distributed/core/ deleted |
import from distributed.communicate / distributed.parallel |
| Extension | the CUDA targets are now sm_89 + sm_120a, under one floor predicate (>=sm89, sm90 excluded) |
Ampere (sm_80/sm_86) and Hopper (sm_90) no longer load the CUDA operators; add sm_100 with CACHE_DIT_CUDA_ARCH_LIST when building |
| CLI | --taylorseer / --dmd / --foca still parse, but conflicting selections now fail |
migrate to --calibrator |
| Cache | SVD-Cache is experimental | prefer --calibrator taylorseer or dmd |
📦 Install
pip install -U cache-dit # PyPI, pure Python
pip install -U cache-dit --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/ # +cu13, with the CUDA extensionThe CUDA channel carries the local version +cu13 (cache-dit==1.5.5+cu13) for CPython 3.11–3.14, built for sm_89 and sm_120a. Those wheels are manylinux_2_34_x86_64 (the floor dropped from 2_38 because the extension no longer needs the newer GLIBC symbols).