Release FlashQLA v0.1.3 · QwenLM/FlashQLA

🚀 Highlights

Dual-CP: inter- + intra-card context parallelism by @Erix025 — FlashQLA now unifies inter-card (cross-GPU) and intra-card CP into a two-level scheme. This extends CP to TP + long-sequence + small-head-count settings and further improves SM utilization.
AutoCP v2: a smarter partitioning strategy by @Erix025 — a redesigned auto-CP that decides whether to enable CP and which subsequence length to use, driven by a lightweight per-architecture kernel-latency model and a wave-aware partition search. Adds infer / train modes, SWA gate-decay support, per-arch fitted coefficients (SM90 / SM100), and dedicated documentation under tools/autocp.
SM120 / SM121 backward pass by @minatoyukinaa — completes Blackwell SM120/121 with a fused backward kernel (previously forward-only), stabilized for auto-CP and variable-length sequences with zero-padding fixes.

SM100 (Blackwell) performance — moved the prepare_h recurrence to tcgen05, improved KKT-solve occupancy and rebalanced backward register caps (@Butterfingrz); async-staged incomplete forward/state tiles and specialized pure full-tile batches (@SakiNana7qi); optimized the fused dQ TMEM path (@byshiue).
New architecture support — SM103 (Blackwell) by @hysmio; SM121 (NVIDIA GB10 / DGX Spark) via SM120 kernels by @zhachar14h-cell.
Host metadata fast path — reuse of prepared variable-length chunk descriptors to cut CPU overhead for varlen / CP metadata preparation.
CUDA-graph friendliness — added an sglang-style tensor cache for CUDA graph capture.
Correctness & stability fixes — improved dG accuracy via high-precision matrix transposition; fixed a race condition in cross-partition aggregation; added fence.proxy.async and extra syncs to SM90 / SM100 Blackwell kernels; fixed SM90 backward padding and the autograd interface.
Docs — fixed the low-level API cp_cache example (@eyesofish) and added AutoCP documentation.

Special thanks to @Erix025, @minatoyukinaa, @Butterfingrz, @SakiNana7qi, @byshiue, @hysmio, @zhachar14h-cell and @eyesofish for their contributions, and to all community members for their valuable feedback. We welcome your continued participation through issues and pull requests.