Skip to content

v0.5.1

Latest
Latest

Choose a tag to compare

View all tags
@yzhangcs yzhangcs released this 18 Jun 18:16
v0.5.1
2e38c1f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

What's Changed

  • [Typo] Fix typos in kernel comments [skip test] by @yurekami in #856
  • [KDA] Support use_beta_sigmoid_in_kernel by @zhiyuan1i in #858
  • [CI] Disable backend when testing models and bump tilelang by @zhiyuan1i in #861
  • [Fix] GatedDeltaNet step mode crash when conv cache is None on cold Triton autotuner cache by @CiprianFlorin-Ifrim in #860
  • [Cache] FLA autotune cache by @sBobHuang in #798
  • Add Mamba3 by @Phoenix8215 in #855
  • [README] Add Mamba-3 news entry and models table item by @yzhangcs in #865
  • [Common] Support transpose_state_layout in fused_recurrent (GLA / SimpleGLA / RetNet) by @yzhangcs in #864
  • [Docs] Reorganize ENVs.md into themed sections by @yzhangcs in #866
  • [Conv] Fix causal_conv1d backward producing wrong gradients for non-contiguous dy by @4332001876 in #871
  • [Typo] Restore comment-typo fixes regressed by #816 [skip test] by @yurekami in #873
  • [Fix] Improve fused loss gradient accumulation by @yzhangcs in #874
  • [Fix] Fix dk in normalized linear attention by @ayghri in #875
  • [Common] Switch chunk paths to exp2 by @yzhangcs in #867
  • [GDN] Fix WY backward when gating is disabled by @yzhangcs in #863
  • [KDA] Fix chunk size 32 path by @yzhangcs in #876
  • [Docs] Refresh README overview by @yzhangcs in #877
  • [Ops] Add AttnRes operator by @yzhangcs in #878
  • [Ops] Export fused AttnRes from ops [skip test] by @yzhangcs in #879
  • [Models] Apply AttnRes to model definitions by @yzhangcs in #880
  • [Ops] Fuse output RMSNorm into attnres kernels by @yzhangcs in #883
  • [Bench] Add torch.profiler pass and run-context header to throughput bench by @yzhangcs in #886
  • [Lint] Fix benchmark_training_throughput.py [skip test] by @yzhangcs in #887
  • [Ops] Switch attnres to list-input + pointer-table API by @yzhangcs in #884
  • [Fix] Add functools.cache for op dispatch to reduce CPU overhead by @KevinZeng08 in #881
  • [Model] Add YOCO model implementation by @Shomvel in #857
  • [Fix] Enable torch.compile on Python 3.10 by @yzhangcs in #894
  • [Docs] Document Ruff ignore rationale by @yzhangcs in #896
  • [Fix] Define FLA package import contract by @yzhangcs in #897
  • [Ops] Allocate attnres fwd output in residual dtype to reduce IO by @yzhangcs in #898
  • [Fix] Tolerate configs without head_dim in throughput bench by @yzhangcs in #899
  • [Enhance] Extend tensor_cache to a bounded queue by @HAOCHENYE in #895
  • Add Raven model integration by @AvivBick in #900
  • [Raven] Remove dead params and fix o_norm affine consistency by @yzhangcs in #901
  • [Raven] Fix negative RoPE offset on left-padded prefill by @yzhangcs in #902
  • [Ops] Skip None args in autotune restore_value to avoid clone crash by @kasper0406 in #893
  • [Ops] Rename transpose_state_layout to state_v_first and extend it to GLA / SimpleGLA chunk by @yzhangcs in #905
  • [Fix] Fix packed varlen launch by @sablin39 in #904
  • [Fix] Correct RWKV7 gate output chunk mask by @sablin39 in #907
  • [Ops] Replace attnres int64 ptr table with padded tensor tuple by @wyc-ruiker in #912
  • [Ops] Tidy attnres fused kernels: drop DTYPE, rename BL to L2 by @yzhangcs in #914
  • [Perf] Read each attnres residual once in fwd and bwd by @yzhangcs in #916
  • [Bench] Fix throughput-bench crash on configs without num_heads by @yzhangcs in #917
  • [Fix] Large-offset Pointer Arithmetic in Blackwell GPUs by @Costa-SM in #911
  • [GDN] Support use_beta_sigmoid_in_kernel & allow_neg_eigval by @yzhangcs in #919
  • [Activation] Add SigmoidGLU and fix GeLUFunction backward by @yzhangcs in #921
  • [Docs] Expand contributor guidelines by @yzhangcs in #923
  • [Docs] Add AGENTS.md guide for AI coding agents by @yzhangcs in #922
  • [Test] Add zero-gate regime to chunk_gated_delta_rule tests by @yzhangcs in #924
  • [Git] Ignore Python virtual environment directories by @yzhangcs in #925
  • Make varlen index helpers torch.compile-friendly by @kasper0406 in #909
  • [Test] Reorganize ops/utils tests to mirror source layout by @yzhangcs in #926
  • [Ops] Feat: Add triton-ascend backend support for FLA module kernels by @zheliuyu in #927
  • [GDN2] Add Gated DeltaNet 2 (GDN-2) ops, layer, and tests by @shahvishal8465 in #920
  • [CI] Update H100 PyTorch test matrix by @zhiyuan1i in #929
  • [CI] Add agent workflow guardrails and PR title validation by @zhiyuan1i in #930
  • [Attn] Add Wall attention (per-channel multiplicative-decay) op, layer, and model by @tea-more in #928
  • [Refactor] Rename gated_delta_net_v2 (GDN-2) to gdn2 and align the layer with fla conventions by @yzhangcs in #931
  • [Attn] Wall: add AttnRes support and align layer/model with fla conventions by @yzhangcs in #932
  • [Refactor] Split fla.utils into maintainable package by @zhiyuan1i in #933
  • [CI] Gate H100 Triton baseline by environment by @zhiyuan1i in #936
  • [Fix] Detect consumer Blackwell (SM120) as Blackwell by @kasper0406 in #940
  • [Refactor] Decouple torch/triton backend from base deps by @kashif in #937
  • [Layer] Add PowGLU (Power-Gated Linear Unit) activation by @yzhangcs in #951
  • [Attn] Add Parallax (parameterized local linear attention) op, layer and model by @Yifei-Zuo in #939
  • [Fix] parallel_attn: keep NV==1 in backward so dq/dk reduce over full V by @kasper0406 in #941
  • [Fix] FLA cache window updates and reset behavior by @Michael-RDev in #910

New Contributors

Full Changelog: v0.5.0...v0.5.1

Contributors

kashif, kasper0406, and 20 other contributors
Assets 2
Loading