Skip to content

v0.6.0

Latest
Latest

Choose a tag to compare

View all tags
@yzhangcs yzhangcs released this 23 Oct 17:47
· 66 commits to main since this release
v0.6.0
413ad2c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • [Docs] Add fla-ascend-performance agent skill for Ascend NPU profiling and optimization by @sunyi0505 in #1068
  • [Fix] Zero-init Ascend chunk_scaled_dot_kkt output and add solve_tril to CI by @zheliuyu in #1070
  • [Perf] Refactor Ascend KDA WY backward: split dA finalize and precompute dw workspace by @sunyi0505 in #1074
  • [Perf] Rewrite Ascend chunk_delta_h fwd by @OsirisDuan in #1055
  • [Fix] Gate the recomputed output in layer_norm_gated backward by @truong-v in #1071
  • [Fix] Use the full token stride for DPLR safe-gate offsets by @morluto in #1085
  • [Fix] Fix int32 overflow in pointer arithmetic across Triton kernels by @zhiyuan1i in #1082
  • [Feature] Add Ascend NPU backend for KDA fused recurrent forward by @sunyi0505 in #1090
  • [Fix] Restore tuple-form legacy cache round trips by @morluto in #1089
  • [Fix] Correct GRPO loss with old log probabilities by @dongchany in #1076
  • [GDN] Enable FlashQLA forward dispatch on SM120 by @Akrao9 in #1094
  • [GDN] Fix invalid launch above 65535/HV sequences in the state kernels by @schmidt-jake in #1077
  • [Fix] Fix invalid launch above 65535 blocks in decode-path kernels by @zhiyuan1i in #1097
  • [GDN] Match SM120 exactly for FlashQLA dispatch by @Akrao9 in #1098
  • [Fix] Preserve explicit cu_seqlens with attention masks by @morluto in #1088
  • [CI] Enforce PR checklist and add PR template by @zhiyuan1i in #1101
  • [CI] Run Ascend A2 CI on NPU-related file changes by @zheliuyu in #1091
  • [Perf] Optimize KDA NPU recompute_w_u_fwd kernel launch and fused u/w loop- #1081 by @sunyi0505 in #1099
  • [Fix] Hoist in-kernel triton.next_power_of_2 to host launchers by @zhiyuan1i in #1108
  • [Fix] Mask out-of-bounds accesses in RWKV7 channel mixing kernels by @truong-v in #1103
  • [KDA] Fix illegal memory access in SM100 backward autotuning by @lbx154 in #1109
  • [Fix] Handle chunk-boundary-straddling key tiles in DeltaFormer forward by @truong-v in #1105
  • [Fix] Ascend NPU: guard against tl.dot left-operand clobber across triton_ascend kernels by @sunyi0505 in #1113
  • [Fix] Restructure Ascend chunk_gated_delta_rule_bwd_dhu to AICore task loop by @sunyi0505 in #1118
  • [Perf] Optimize Ascend causal_conv1d with 1D core-grid and constexpr DMA split by @sunyi0505 in #1126
  • [Rodimus] Fix TypeError from stale head_first argument in chunk path by @swayambshah in #1124
  • [GDN] Allow passing pre-computed chunk_indices to chunk_gated_delta_rule by @zhiyuan1i in #1132
  • [Perf] Optimize Ascend KDA intra backward with split kernels and 1D core-grid by @sunyi0505 in #1130
  • [Fix] Correct the final-state gradient for chunks shorter than W - 1 by @truong-v in #1141
  • [Fix] Honor the real layouts of y and dy in compute_dh0_kernel by @truong-v in #1139
  • [Fix] Mask the g load in fused_recurrent_kda_fwd_kernel by @truong-v in #1135
  • [Fix] Correct Ascend causal_conv1d short-chunk cache backward by @sunyi0505 in #1146
  • [Fix] Use native reverse scans in scalar cumsum by @piero2c in #1095
  • [Typo] Fix recurring spelling errors across docs and layers by @morluto in #1100
  • [Fix] Reject invalid context-parallel token partitions by @morluto in #1093
  • [Cleanup] Remove remaining head_first code paths by @zhiyuan1i in #1148
  • [Fix] Use native reverse scans in Ascend scalar cumsum by @zhiyuan1i in #1150
  • [Perf] Optimize Ascend KDA wy backward with 1D core-grid and T-contiguous loads by @sunyi0505 in #1149
  • [Ops] Rewrite NPU chunk_scaled_dot_kkt_fwd by @OsirisDuan in #1023
  • [Model] Add Preconditioned Gated DeltaNet (PGDN) and KDA (PKDA) by @ntumm120 in #950
  • [Perf] A Tilelang implementation of DPLR by @sablin39 in #1069
  • [Fix] Handle a partial last chunk in log-linear attention by @truong-v in #1143
  • [Perf] Optimize Ascend GDN fwd_h with dynamic tiling, oneslab path, and precomputed gates by @sunyi0505 in #1157
  • [Ops] Add triton-ascend backend for chunk_gla by @zheliuyu in #1121
  • [KDA] Make A_log optional in lower bound gate for KDA by @yzhangcs in #1107
  • [DPLR] Add exact tensor-core (I-L)^-1 inversion for safe gates, default chunk_size to 64 for tight lower_bound by @zhiyuan1i in #1162
  • [Fix] Return an integer from ShortConvolution.state_size by @taking-lying-flat in #1133
  • [Fix] Allow GVA in fused recurrent GDN2 by @taking-lying-flat in #1166
  • [Docs] Tighten contributor docs and make the PR checklist N/A-aware by @yzhangcs in #1168
  • [Perf] Optimize Ascend fused_attnres with N-tiling, fused bwd prep, and residual reuse by @sunyi0505 in #1167
  • [Fix] Pin NPU kernels for torch_npu=2.9.0.post6, triton-ascend=3.2.2 and cann=9.1.0 by @zheliuyu in #1161
  • [Docs] Add the fla-design-coverage skill by @zhiyuan1i in #1164
  • [Perf] Fold scale into the gated operand and autotune BK in Ascend GLA fwd_o by @qianjinqiu in #1170
  • [Fix] Drop unused NB from causal_conv1d autotune cache key by @AmirF194 in #1154
  • [Fix] Mask GDN2 recurrent gate loads by @taking-lying-flat in #1165
  • [Fix] Repeat Comba auxiliary keys for GVA by @taking-lying-flat in #1169
  • [Fix] Mask GDN gk/gv recurrent gate loads and key chunk autotuners on K/V by @zhiyuan1i in #1171
  • [Fix] Anchor loop-derived token offsets to int64 in chunk kernels by @zhiyuan1i in #1173
  • [Fix] Fold chunk-launch grids to fit the 65535-block limit by @zhiyuan1i in #1174
  • [Perf] Optimize Ascend chunk_scaled_dot_kkt_fwd with block_ptr loads and safer gating by @sunyi0505 in #1172
  • [Ops] Support native GVA in Comba chunk kernel by @yzhangcs in #1175
  • [Perf] Rewrite Ascend local_cumsum by @OsirisDuan in #1061
  • [Fix] generate() prefills the full prompt instead of only the last token by @Sunt-ing in #962
  • [Fix] Keep GDN eval gradients on chunk kernel by @taking-lying-flat in #1131
  • [Fix] Preserve full logits when computing CausalLM loss by @taking-lying-flat in #1185
  • [Fix] Handle empty Parallax sliding-window cache by @taking-lying-flat in #1183
  • [Perf] Add full-BT Ascend bwd paths for chunk_bwd_dqkwg and dv_local by @sunyi0505 in #1186
  • [Fix] Avoid rebinding chunk loop variables for int64 anchoring by @zhiyuan1i in #1188
  • [Fix] Correct ABC and LightNet state sizes by @taking-lying-flat in #1192
  • [Fix] Prevent PGDN naive packed state leakage by @taking-lying-flat in #1184
  • [Fix] Validate rotary dimensions and offset caches by @taking-lying-flat in #1177
  • [Fix] Use NUM_WARPS_AUTOTUNE in causal_conv1d_bwd_kernel autotune by @yurekami in #1190
  • [Fix] Guard GSA inference final state stores by @taking-lying-flat in #1191
  • [Fix] Reject invalid cache/mask/cu_seqlens combos and degenerate inputs across layers by @zhiyuan1i in #1189
  • [Fix] Repair LogLinearMamba2 Triton cached decoding by @taking-lying-flat in #1176
  • [Fix] Restore backend dispatch under torch.compiler.disable by @xy200303 in #1111
  • [Fix] Keep wall_attn decode's pre-rescaled keys in bf16 to avoid fp16 overflow by @zhiyuan1i in #1195
  • [Fix] Apply scale in naive recurrent gated delta product by @zhiyuan1i in #1201
  • [Fix] Prevent NaN failure in MesaNet by @Undermyth in #1152
  • [Fix] Validate attention dimensions and grouped-query heads by @taking-lying-flat in #1193
  • [Fix] Correct packed token shift decode detection by @taking-lying-flat in #1194
  • [Fix] Correct HGRN state size for expansion by @taking-lying-flat in #1196
  • [Fix] Handle grouped Mamba3 cached decode by @taking-lying-flat in #1197
  • [Fix] Stabilize DeltaNet sum normalization by @taking-lying-flat in #1198
  • [Perf] Avoid unused max sync in unpadding by @taking-lying-flat in #1200
  • [Perf] Cache Mamba3 SISO decode projection by @taking-lying-flat in #1199
  • [CP] Enable context parallelism for GDP by @Mellonta in #1022
  • [CP] Enable context parallelism for GDN-2 by @eric-ide in #1116
  • [Perf] Optimize Ascend prepare_wy_repr_bwd with larger kv tiles, bf16 mmad, and fused a2/dg by @sunyi0505 in #1209
  • [Fix] Store TTT dh/dhb in bf16 to avoid fp16 overflow by @zhiyuan1i in #1214
  • [Fix] Gate decode shortcuts and validate dims across layers by @zhiyuan1i in #1203
  • [CP] Fix causal conv residual and width-one handling by @taking-lying-flat in #1211
  • [Fix] Intel/XPU compatibility fixes for modules tests by @zhiyuan1i in #1206
  • [Fix] Fix NPU triton_ascend kernel recompilation on variable-length sequences by @ys2025-AI in #1207
  • [GDN] Key the fused GDN-2 WY/dqkg backward autotuner on K and V by @tianxiao-rgb in #1120
  • [Fix] Cast Ascend kernel indices to int64 before stride multiply by @sunyi0505 in #1216
  • [Feat] Support platform-graph capture and replay for KDA chunk by @zhiyuan1i in #1212
  • [Perf] Optimize NSA compression dKV query grouping by @heiheiha798 in #1159
  • [Test] Fix the broken GDN-2 autotune-key test and drop its cold-autotune cost by @tianxiaojiang4 in #1218
  • [Perf] Replace split Ascend KDA bwd_intra kernels with a single persistent kernel by @qianjinqiu in #1210
  • [Feat] Support zigzag layout for context parallelism by @zhiyuan1i in #1221
  • [Fix] Correct perplexity evaluation by @yzhangcs in #1222
  • [Models] Add MomentumDeltaNet implementation from arXiv:2605.05838 (ICML2026) by @bhaochen in #1208
  • [Model] Add CAT (Compress and Attend Transformer) by @bicycleman15 in #1017
  • [Perf] Remove host syncs from fused linear CE and KL divergence by @taking-lying-flat in #1220
  • [CE] Share loss kernels and support vocabulary parallelism by @yzhangcs in #1231
  • [GDN] Restrict Blackwell chunk_fwd_o to <= 4 warps by @jepio in #1229
  • [Perf] Optimize Ascend GDN recompute_w_u_fwd with block pointers and independent BK/BV tiling by @sunyi0505 in #1225
  • [Refactor] Align Based attention with FLA layer conventions by @yzhangcs in #1238
  • [Fix] Restore package imports with precompile caching by @arbi-dev in #980
  • [Fix] Reject odd K/V for 16-bit dtypes in chunk_bwd_dv by @zhiyuan1i in #1241
  • [Fix] Eliminate per-call reference cycle in launch_grid_chunked by @Ginray in #1235
  • [Ops] Add Triton-Ascend compatibility for GDN-2 by @hazelduan in #1213
  • [Ops] Enable TMA on AMD gfx1250 by @CRobeck in #1237
  • [Fix] Store dx only for the rows the program owns in layer_norm_bwd_kernel by @mark14wu in #1249
  • [Fix] Cast Comba inner decay and output correction back to projection dtype by @zhiyuan1i in #1267
  • [Perf] Optimize KDA fwd inter_solve_fused kernel for Ascend NPU by @qianjinqiu in #1252
  • [Fix] Avoid unit-scale specialization in Hopper state backward by @taking-lying-flat in #1262
  • [Fix] Default legacy cache offset when unspecified by @aniruddhaadak80 in #1280
  • [CI] Include nested imports in find_dependent_tests import scan by @aniruddhaadak80 in #1260
  • [Conv] Validate effective backend after FLA_CONV_BACKEND override in ShortConvolution by @aniruddhaadak80 in #1257
  • [Fix] Skip unneeded student gradients in fused KL loss by @taking-lying-flat in #1248
  • [Fix] Preserve negative inputs in DPFP feature map by @taking-lying-flat in #1276
  • [Fix] Correct PrepareModuleWeight parameter handling by @yzhangcs in #1287
  • [Refactor] Remove obsolete attention kwargs by @yzhangcs in #1288
  • [Refactor] Remove obsolete kwargs from operator entries by @yzhangcs in #1289
  • [Fix] Handle distinct value dimensions in attention reference by @yzhangcs in #1291
  • [Fix] Handle padding and packed sequences in linear attention by @Lexwzp0 in #1266
  • [Attn] Add sliding-window support to decoding by @yzhangcs in #1293
  • [Fix] Fold chunk state launch grids to fit the 65535-block limit by @xy200303 in #1239
  • [Misc] Align PackSequenceFunction backward returns with forward inputs by @aniruddhaadak80 in #1258
  • [Fix] Validate GQA heads and handle strided decoding inputs by @morluto in #1032
  • [Refactor] Align normalization kernels with FLA conventions by @yzhangcs in #1297
  • [Fix] Avoid unit-scale constexpr folding in Ascend chunk kernels by @TecJesh in #1298
  • [NPU] Add single-device NPUGraph support for KDA by @FightingZhen in #1245
  • [Fix] Build the L2Wrap logits gradient in the logits dtype by @aniruddhaadak80 in #1302
  • [Fix] Preserve KDA input contracts in FlashKDA dispatch by @ZenAlexa in #1299
  • [Feature] Support grouped heads and attention scaling in log-linear attention by @feiyunzhang in #1315
  • [Fix] Preserve RWKV7 orthogonal initialization for low-precision weights by @adenzhou1350 in #1320
  • [Fix] Fuse stable ELU+1 feature maps by @vickiiimu in #1265
  • [Fix] Keep ELU+1 compatible with Ascend by @yzhangcs in #1325
  • [Fix] Unify program ID overflow handling in ops by @yzhangcs in #1329
  • [Fix] Remove stale BT from CP autotune key by @AmesingFlank in #1330
  • [Fix] Keep MoM shared memory in chunk mode during training by @adenzhou1350 in #1333
  • [Fix] Mask the decay gate for padded steps in SimpleGatedLinearAttention by @AmirF194 in #1273
  • [CI] Define contribution standards and validate large PRs by @yzhangcs in #1335
  • [Fix] Avoid unsafe Hopper GLA backward configurations by @yzhangcs in #1337
  • [Refactor] Separate utility concerns and simplify project imports by @yzhangcs in #1338
  • [Attn] Add stick-breaking attention by @harshithkantamneni in #1319
  • [Ops] Add a global Gluon backend switch by @yzhangcs in #1340
  • [Model] Add stick-breaking attention layer and model by @yzhangcs in #1339
  • [Perf] Add opt-in Gluon causal convolution by @yzhangcs in #1328
  • [Model] Add Cyclic Flow Attention (CyFA) by @Chyxx in #1326
  • [Perf] Add Triton-Ascend backend for GDN2 fused recurrent inference by @Ginray in #1270
  • [Docs] Clarify public API compatibility contracts by @yzhangcs in #1344
  • [LightNet] Handle padded inputs with varlen normalization by @adenzhou1350 in #1334
  • [Fix] Restore TileLang attention shape and pipeline support by @yzhangcs in #1351
  • [Refactor] Centralize dispatch and operation backend registration (breaking backend API) by @yzhangcs in #1347
  • [Refactor] Group normalization implementations and backends by @yzhangcs in #1348
  • [Refactor] Keep convolution backends with their operation by @yzhangcs in #1349
  • [Ascend] Make Retention chunk / fused_chunk / parallel / fused_recurrent paths run on Ascend NPU by @quipiter in #1307
  • [Refactor] Flatten normalization module layout by @yzhangcs in #1355
  • [Refactor] Separate causal and long convolution layouts by @yzhangcs in #1357
  • [Refactor] Register module backend implementations at their entry points by @yzhangcs in #1223
  • [Docs] Simplify repository skills and contribution guidance by @yzhangcs in #1345
  • [Refactor] Consolidate legacy module imports by @yzhangcs in #1360
  • [Fix] Preserve softplus names in compiled Triton kernels by @huong8373tt-beep in #1353
  • [Fix] Accept local loss tensors in LinearLossParallel by @adenzhou1350 in #1362
  • [Refactor] Move quantized normalization into norm by @yzhangcs in #1366
  • [Fix] Avoid eager GRPO imports and spurious warnings by @zhiyuan1i in #1364
  • [Fix] Honor strides in vector activations by @adenzhou1350 in #1369
  • [Fix] Reduce GDN/KDA gate bias gradients before downcasting by @cyanseek in #1295
  • [KDA] Restore TileLang dispatch for the fused WY backward by @mazitovs in #1361
  • [Fix] Accept only the modes LightNetAttention can dispatch by @aniruddhaadak80 in #1305
  • [Perf] Respect gradient demand in SwiGLU linear by @cyanseek in #1296
  • [Fix] Forward the configured epsilon in the bit-linear norm by @aniruddhaadak80 in #1303
  • [GDN] Add configurable output gate activation by @alifurkanstahl in #1182
  • [Fix] Honor 1D strides in Ascend activations and follow module aliases in test selection by @sunyi0505 in #1374
  • [CE] Add an opt-in Gluon backend for fused cross entropy by @yzhangcs in #1378
  • [Fix] Make KDA safe_gate diagonal path independent of future gates by @xy200303 in #1242
  • [CI] Re-evaluate Ascend CI triggers on PR description edits by @zhudada0120 in #1386
  • [Fix] Version-gate Blackwell eight-warp chunk configs by @arcusbuilds in #1399
  • [KDA] Run context-parallel KDA on Ascend NPU by @2313247981 in #1398
  • [Fix] Restore intra-card CP dispatch for the KDA forward by @Windyskr in #1400
  • [Ops] Add Triton-Ascend support for preconditioned GDN by @sunyi0505 in #1423
  • [CI] Validate PR commit subjects by @zhiyuan1i in #1486
  • [Fix] Match HGRN lower bounds to expanded recurrent width by @yzhangcs in #1435
  • [Refactor] Consolidate shared modeling helpers by @yzhangcs in #1493
  • [Fix] Preserve GDN-2 prefix causality with fixed anchors by @yzhangcs in #1496
  • [Perf] Stop token_shift from re-tuning and recompiling for new input sizes by @Windyskr in #1494
  • [Perf] Stop the path attention backward from recompiling for every new length by @Windyskr in #1495
  • [Conv] Add CUDA backend for contiguous context parallelism by @Tytskiy in #1492
  • [Docs] Add large PR decomposition and native stack workflow by @yzhangcs in #1501
  • [Fix] Allocate IPLR varlen final states per sequence by @adenzhou1350 in #1419
  • [Conv] Refactor CUDA context parallel convolution by @yzhangcs in #1498
  • [Fix] Preserve prompt inputs during cached generation by @yzhangcs in #1506
  • [Refactor] Separate model cache and generation utilities by @yzhangcs in #1502
  • [Refactor] Keep preconditioner state inside recurrent cache tuples by @yzhangcs in #1507
  • [Refactor] Adopt the Transformers 5 model and generation APIs by @yzhangcs in #1503
  • [Refactor] Share causal LM logits and token-normalized loss by @yzhangcs in #1505
  • [Refactor] Reuse Transformers embedding access across FLA models by @yzhangcs in #1508
  • [Refactor] Share the standard causal LM forward by @yzhangcs in #1509
  • [Refactor] Share common model weight initialization by @yzhangcs in #1510
  • [Fix] Preserve explicit packed boundaries in preconditioned layers by @syc632 in #1409
  • [Refactor] Separate model configuration validation from layer construction by @yzhangcs in #1511
  • [Test] Consolidate common model tests and preserve specialized coverage by @yzhangcs in #1515
  • [Refactor] Consolidate shared model construction helpers by @yzhangcs in #1517
  • [Refactor] Share attention state updates across FLA caches by @yzhangcs in #1516
  • [Refactor] Share decoder forward across FLA models by @yzhangcs in #1523
  • [Refactor] Register utils Ascend implementations at their entry points by @yzhangcs in #1526
  • [Perf] Use 128-wide chunk_o backward tiles on A100-class GPUs for 16-bit inputs by @Windyskr in #1514
  • [Perf] Build the retention and lightning-attention decay on the device by @Windyskr in #1513
  • [Ops] Add Sinkhorn normalization kernels by @yzhangcs in #1524
  • [Fix] Share additive gates between GSA and LightNet by @yzhangcs in #1536
  • [Refactor] Reuse common recurrent stages in GSA by @yzhangcs in #1537
  • [Fix] Preserve GSA chunk state gradients and grouped heads by @yzhangcs in #1538
  • [Refactor] Simplify GSA chunk stage orchestration by @yzhangcs in #1539
  • [Refactor] Separate GSA chunk forward and backward kernels by @yzhangcs in #1533
  • [Refactor] Unify GSA tests and simplify recurrent backward by @yzhangcs in #1542
  • [Fix] Keep cached multi-token WallAttention causal by @samridh10exe in #1397
  • [Misc] Pin the Ascend NPU extra to Triton-Ascend 3.6.0 by @zheliuyu in #1530
  • [Refactor] Organize CP tests and share distributed setup by @yzhangcs in #1518
  • [Fix] Re-mask padded logits after scale and softcap in Ascend fused CE by @sunyi0505 in #1544
  • [Fix] Guard partial-chunk level scales by @zhiyuan1i in #1488
  • [Fix] Use bf16x3 for safe-gate backward dots by @zhiyuan1i in #1554
  • [CE] Re-mask padded logits in logsumexp kernels by @zhiyuan1i in #1552
  • [CI] Optimize NPU CI by skipping Ascend CI on push to main by @zheliuyu in #1557
  • [Fix] Derive activation row strides without extra copies by @zhiyuan1i in #1487
  • [CP] Require high-precision affine state chains by @yzhangcs in #1546
  • [Fix] Preserve log-linear cache state transformations by @zhiyuan1i in #1561
  • [Fix] Return no MesaNet final states when disabled by @zhiyuan1i in #1562
  • [Fix] Include KL gradients in GRPO backward by @zhiyuan1i in #1563
  • [Fix] Preserve PKDA safe-gate causality by @zhiyuan1i in #1566
  • [Fix] Align cached attention mask with truncated KV by @zhiyuan1i in #1567
  • [Fix] Match Mamba2 Triton convolution contracts by @zhiyuan1i in #1574
  • [Fix] Preserve recurrent states for empty packed sequences by @zhiyuan1i in #1571
  • [Fix] Preserve sliding-window context for cached chunks by @zhiyuan1i in #1568
  • [Fix] Preserve ATK state for empty sequences by @zhiyuan1i in #1573
  • [Fix] Isolate CyFA empty-sequence gradients by @zhiyuan1i in #1572
  • [Fix] Respect packed boundaries with attention masks by @zhiyuan1i in #1575
  • [Fix] Route cached MesaNet multi-token continuation by @zhiyuan1i in #1569
  • [Fix] Stabilize MomentumDeltaNet sum normalization by @zhiyuan1i in #1564
  • [Fix] Disable DeltaFormer cache by default by @zhiyuan1i in #1565

New Contributors

Full Changelog: v0.5.2...v0.6.0

Contributors

jepio, feiyunzhang, and 57 other contributors
Assets 2
Loading