v1.5.3

Release Notes: cache-dit v1.5.3

Release Date: 2026-10-15
Comparison: v1.5.2...v1.5.3 (10 commits)
Type: Feature + bugfix release

Summary

v1.5.3 fixes four correctness bugs and adds one feature. A silent GQA corruption in the
Ulysses KV replication path (replicate_kv_sequence) could turn outputs into pure noise for
GQA models on 4-way context parallelism; torchao int8 quantization was unusable on Ampere
GPUs and on torchao 0.18; the DMD calibrator silently reused the last snapshot on CPU; and
the DBCache continuous-cache streak counter could run past max_continuous_cached_steps. The
FFPA attention backend now accepts attn_mask and runs on sm_89 (Ada) GPUs.

This release also unifies the CUDA (SVDQuant) wheels with the PyPI release: they are now the
same distribution, told apart by the local version +cu13 and served from our own index, so a
single command installs either build.

Affected Components

Component	Path	Change
Distributed (Ulysses/GQA)	src/cache_dit/distributed/core/_distributed_primitives.py	Per-window KV head selection at world == group size (fix #1115)
Attention (backend registry)	src/cache_dit/attention/backends/register.py	Probe enable_gqa once at import instead of per dispatch
Attention (FFPA backend)	src/cache_dit/attention/backends/ffpa.py, src/cache_dit/_utils/utils.py	attn_mask support and sm_89 admission
Quantization (torchao)	src/cache_dit/quantization/torchao/quantize_ao.py	Import on pre-Ada GPUs; single granularity for int8 configs
Caching (contexts/manager)	src/cache_dit/caching/cache_contexts/cache_context.py, cache_manager.py	Continuous-cache streak accounting
DMD calibrator	src/cache_dit/caching/cache_contexts/calibrators/dmd.py	driver="gesvd" only on CUDA
Release tooling	tools/build_releases.sh, tools/release_workspace.py, tools/repair_wheel_platform_tag.py, tools/inspect_release_wheel.py, tools/make_index.py, tools/publish_index.sh	Unified CUDA channel, relocatable RPATH, artifact assertions, index generation and publish
Docs	docs/user_guide/INSTALL.md, docs/user_guide/ATTENTION.md, docs/user_guide/QUANTIZATION.md, docs/user_guide/LOAD_CONFIGS.md, README.md	Install command, FFPA matrix, release flow

Breaking Changes

None. The retired cache-dit-cu13 distribution installs the same cache_dit/ package as
cache-dit==1.5.3+cu13, so uninstall it before switching (see Packaging).

New Features

• FFPA attention backend: attn_mask and sm_89 support (#1119): the FFPA backend now
  accepts attn_mask -- 1-D and 2-D masks are expanded to [B, 1, 1, Nkv], 3-D and 4-D masks
  pass through, and a masked call forces the HND route (the NHD fast path is mask-free by
  design). sm_89 (Ada) is admitted for fp16 and fp8; fp4 and the NHD fast path remain
  sm_120-only. 70 test cases cover mask parity against SDPA and the arch matrix.

• Unified CUDA wheels: cache-dit==1.5.3+cu13 from our own index (#1120, release
  engineering): the CUDA build of this release is published under the cache-dit name with the
  local version +cu13, so installing it is one command and needs no second distribution name.
  See Packaging below.

Bug Fixes

• Fix silent GQA corruption in replicate_kv_sequence at world == group size (#1115):
  gather_replicated_kv_for_local_q skipped the per-window KV head selection whenever the
  global KV head count equaled the local Q head count. For GQA models this is not an identity:
  Boogu (28 Q / 7 KV) with Ulysses world=4 gives every rank 7 local Q heads, coincidentally
  equal to the 7 global KV heads, so every rank skipped the GQA mapping, attended with the wrong
  KV heads and produced pure-noise outputs. The shortcut now only applies to global MHA.
  Regression test: tests/parallelism/test_ulysses_gqa_multirank.py.

• Fix dmd_svd_precision="high" silently falling back on CPU (#1116):
  torch.linalg.svd(..., driver="gesvd") was passed on every device. PyTorch rejects driver=
  for non-CUDA inputs, and _dmd_fit_one swallowed the error and returned None, so on CPU the
  DMD calibrator reused the last snapshot without a word. The driver is now selected per device,
  matching svdquant/lowrank.py. Test: tests/api/test_dmd_svd_precision.py (all three
  precision levels on CPU).

• Fix the DBCache continuous-cache streak accounting (#1114): the streak counter was not
  restarted after a compute step, non-adjacent cache hits did not reset it, and the cap was reset
  in redundant places, so a run could exceed max_continuous_cached_steps; DBPrune's streak
  limits are now applied per timestep. Regression tests in tests/api/test_refresh_context.py
  cover interleaved warmup, dynamic computation masks, both CFG orders and repeated cache checks.

• Fix torchao quantization on Ampere GPUs and on torchao 0.18 (#1118): two defects.
  _basic_filter_fn / _fallback_filter_fn took quant_ctx: QuantizeAOContext = QuantizeAOContext(), a default built at import time whose quant_type defaults to
  float8_per_row -- so __post_init__ asserted an sm89+ GPU on import, and every torchao
  quant type, int8 and int4 included, failed on A100/A800 with "FP8 requires Ada or newer GPUs
  (>=sm89), but got (8, 0)"; on CPU-only hosts the import raised NotImplementedError. The
  context is now a required argument (every caller already passed it through partial), and
  requesting FP8 on such a GPU still fails with the same assertion. Second,
  int8_per_row/int8_per_tensor built Int8DynamicActivationInt8WeightConfig(granularity= (PerRow(), PerRow())), which torchao 0.18 rejects with "Invalid granularity type"; a single
  granularity applies to both activation and weight in 0.17 and 0.18. Test:
  tests/quantization/test_torchao_int8.py.

• Fix the CUDA wheels' RPATH (part of #1120): the extension carried the build environment's
  lib directory (.../envs/py312/lib), a path that means nothing on a user's machine and a wrong
  library if it happens to exist there. The release tooling now rewrites it to $ORIGIN and
  refuses to publish a wheel that is not relocatable. The wheels published as cache-dit-cu13
  carry the old path, so this removes an existing wart rather than fixing a regression.

Packaging and Installation

The CUDA (SVDQuant) wheels are now the same distribution as the PyPI release, distinguished by
the local version of the wheel (1.5.3+cu13) and served from a static PEP 503 index:

pip install -U cache-dit --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/
pip install "cache-dit==1.5.3+cu13" --extra-index-url https://vipshop.github.io/cache-dit/whl/cu13/  # pinned

• The former cache-dit-cu13 distribution (last release 1.5.2) is retired. Both names ship the
  same cache_dit/ package, so uninstall the old one before switching:
  pip uninstall cache-dit-cu13.
• Wheels cover CPython 3.10-3.14, manylinux_2_38_x86_64 (the floor follows the GLIBC symbols
  the extension needs) and the sm_80, sm_86, sm_89, sm_120a, sm_121a architectures.
• Add our index as an extra index, not as --index-url: it holds cache-dit alone, with no
  dependencies, so PyPI has to stay the index those come from. uv searches extra indexes before
  the default one, so the command above resolves the CUDA build there too; naming PyPI as the
  extra index instead leaves uv considering the PyPI release alone.
• Install torch first, from the PyTorch index you normally use. The wheels link against a CUDA 13
  torch and cover CUDA >= 13.0.

Performance Improvements

• torch.compile graph breaks from the enable_gqa probe (#1115):
  _supports_enable_gqa() ran inspect.signature on every attention dispatch, which dynamo
  cannot fold away (lru_cache wrappers, id()-based pseudo signatures for C builtins), so the
  probe stayed visible to the recorder. The answer cannot change within a process, so it is now
  resolved once at import: measured on fast-boogu (single card, B=1, flash-varlen backend),
  this removed the last source of graph breaks in the compile cell, 56 -> 0.

Documentation

• README.md: installation instructions, refreshed.
• docs/user_guide/ATTENTION.md: the FFPA mask and arch matrix.
• docs/user_guide/INSTALL.md: the CUDA install command, the migration off cache-dit-cu13, and
  a maintainer section covering the release flow (profiles, publish, verification).
• docs/user_guide/QUANTIZATION.md, docs/user_guide/LOAD_CONFIGS.md: install commands synced.

Dependencies

• No dependency version changes. torchao 0.17 and 0.18 are both supported.

Testing

• New regression tests: tests/parallelism/test_ulysses_gqa_multirank.py (multi-rank GQA
  replication), tests/api/test_dmd_svd_precision.py (SVD driver per precision), and expansions
  to tests/api/test_refresh_context.py (streak accounting) and tests/api/test_ffpa_backend.py
  (mask parity and arch matrix).
• Release verification: the full profile builds all five CPython wheels end to end
  (cp310-cp314, manylinux_2_38_x86_64, 13 cubins per architecture per arch set), the index
  publisher passes 23 synthetic cases (sha256 anchors, merge, re-release, refusals) and the
  resulting wheels install from a local index -- cache_dit.__version__ == 1.5.3+cu13,
  version_tuple == (1, 5, 3, 'cu13'), and import cache_dit._C_svdquant loads.

Contributors

Thank you to everyone who contributed to v1.5.3:

• DefTruth (@DefTruth)
• Windyskr (@Windyskr)
• chengyaoxue64-lab (@chengyaoxue64-lab)
• Keshav Chhabra (@kv-248)
• NgZiXin (@NgZiXin)
• copilot-swe-agent[bot]

Full Changelog

v1.5.2...v1.5.3

Pull requests: #1114, #1115, #1116, #1118, #1119, #1120