Skip to content

v0.1.0

Latest
Latest

Choose a tag to compare

View all tags
@Flink-ddd Flink-ddd released this 21 Sep 08:44
· 88 commits to main since this release
v0.1.0
b31fac9
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

RL-Kernel v0.1.0 is our first public release for kernel-level train–inference consistency in RL post-training.

This release includes 174 merged pull requests from 28 GitHub contributors, covering end-to-end Qwen3-8B consistency across vime, vLLM rollout and Megatron-LM training, with CUDA, ROCm, Ascend(Partial) and MUSA(In progress) support.

  • End-to-end train–inference consistency: Qwen3-8B VIME integration and strict train/rollout LogP validation on CUDA and ROCm (#377, #388, #400).

  • Canonical multi-topology execution: 14 H100 training/rollout TP/CP configurations with 28,672 active LogPs and zero byte mismatches (#432).

  • ROCm deterministic execution: strict Attention, distributed FFN, vocab-parallel LogP, deterministic RCCL, direct-paged execution, MFMA GEMM and Triton Attention (#319, #325, #328, #356, #390, #396, #437).

  • Full-model validation: Qwen3-8B Dense WS1 train/inference closeout across CUDA and Triton-on-CUDA (#315).

  • Reproducibility: profile-driven CUDA/ROCm workflows through the unified rlk reproduction path (#426, #430).

  • Ascend: deterministic kernel coverage and Qwen3-8B Dense cross-configuration validation (#405, #406, #417).

  • MUSA: Moore Threads platform support, Triton-backed operator dispatch and native deterministic kernel coverage (#346, #375).

Train–Inference Consistency

Hardware & Platforms

  • [FEAT][platforms]: implement hardware-aware context for ROCm and CUDA (#1) by @Flink-ddd
  • [fea][Multiplatform]: add multi-architecture build support (#316) by @Flink-ddd

CUDA

ROCm

  • [FEAT][kernels]: add ROCm FlashAttention backend (#104) by @FED4
  • feat(attention): add strict bitwise ROCm path (#319) by @inaniloquentee
  • feat(ffn): add deterministic distributed Triton FFN for ROCm (#325) by @frank-2077
  • feat(logprob): add deterministic ROCm vocab-parallel path (#328) by @hihaluemen
  • [ROCm] Deterministic fused linear logp in Triton (#347) by @KJLdefeated
  • [FEAT][distributed]: add deterministic ROCm-RCCL transport collectives (#356) by @Flink-ddd
  • [Bugfix] Fix ROCm build for deterministic collectives (#380) by @Flink-ddd
  • [feat][rocm]: add end-to-end Vime rollout-training attention ablation matrix (#385) by @Flink-ddd
  • [perf][rocm]: accelerate strict VIME rollout with direct paged CK and graph-safe execution (#390) by @Flink-ddd
  • Update H100 and MI300X distributed benchmark data (#391) by @frank-2077
  • [perf][rocm]: restore full Graph and cache strict Attention schedules (#393) by @inaniloquentee
  • fix(rocm): stabilize direct paged CK attention for bitwise R/R (#394) by @inaniloquentee
  • perf(rocm): MFMA batch-invariant GEMM and chunked Triton attention for strict R/R (#396) by @Flink-ddd
  • docs(rocm): publish G10/G11 200-step RL results (#400) by @inaniloquentee
  • [FEAT][ROCm]: promote deterministic cross-config runtime and kernel validation to main (#401) by @Flink-ddd
  • perf(rocm): restore VIME TP4 decode throughput (#403) by @inaniloquentee
  • perf(rocm): reduce strict no-reuse rollout overhead (#437) by @inaniloquentee

Ascend

  • [draft][WS1][kernels] Batch-invariant logprob Ascend C Kernel (#297) by @erfgss
  • [WS1][kernels] Deterministic attention Ascend C kernel (#320) by @zhangj1an
  • feat(ascend): add prefix-shared attention Ascend C kernel (#340) by @zhangj1an
  • feat(ascend): add deterministic collective Ascend C kernel (#355) by @zhangj1an
  • fix: restore NPU dispatch for PR #361 unit-tests (#363) by @maxiaosong1124
  • feat(ascend): add batch-invariant RMSNorm Ascend C operator (#364) by @erfgss
  • [WS1][Ascend] [Qwen3-8b] Embedding ops (#369) by @zhangj1an
  • [WS1][Ascend] [Qwen3-8b] Fused logp ops (#370) by @zhangj1an
  • [WS1][Ascend] [Qwen3-8b] LM head ops (#371) by @zhangj1an
  • [WS1][Ascend] [Qwen3-8b] Fused linear logp ops (#372) by @zhangj1an
  • [skill] add ws1 ascend kernel (#373) by @zhangj1an
  • feat(ascend): add RoPE kernel (#378) by @erfgss
  • feat(ascend): add SwiGLU forward and backward kernels (#381) by @erfgss
  • [WS1][kernels] Batch-invariant deterministic GEMM Ascend C kernel (#405) by @zhangj1an
  • [WS1][Ascend] Qwen3-8B Dense train-inference closeout for BF16 execution vs FP32 reference (#406) by @zhangj1an
  • [FEAT][Ascend]: promote deterministic cross-config runtime and kernel validation to main (#417) by @Flink-ddd

MUSA

RL Runtime & Framework Integration

Distributed & Communication

Kernels & Operators

  • [FEAT][dispatcher]: implement unified kernel registry and enhanced logger (#2) by @Flink-ddd
  • [FEAT][Kernel] Implement Fused sampling op (#3) by @Flink-ddd
  • feat: add native fused logp RL kernel variants (#33) by @inaniloquentee
  • [REFACTOR] Implement Hardware Abstraction Layer for Operator Wrappers (#37) by @Flink-ddd
  • [refactor]: globally rename project to RL-Kernel and fix CPU fallback precision (#53) by @Flink-ddd
  • feat(kernels): implement pure Triton FlashAttention fallback (#41) (#57) by @EthanZero2Hero
  • FEAT: implement Prefix-Shared Fused Attention for GRPO logprob computation (#71) by @Flink-ddd
  • feat: grpo single gpu example (#80) by @inaniloquentee
  • [BUG] Fix temperature applied twice in FlashInfer sampling path (#86) by @KJLdefeated
  • [FEAT][kernels]: implement fused GRPO loss with in-place group reward normalization (#93) by @KJLdefeated
  • [FEAT][kernels]: implement Fused Policy Ratio and KL Penalty kernel (#97) by @KJLdefeated
  • feat(ws1): NativeRMSNormOp pure-PyTorch ground-truth reference + numerical contract tests (#160) by @maxiaosong1124
  • fix(ratio_kl): sanitize out-of-range action ids at masked positions (#162) by @Dnoob
  • feat(ws1): NativeSiLUOp + NativeSwiGLUOp pure-PyTorch ground-truth references + numerical contract tests (#166) by @maxiaosong1124
  • feat(ws1): Add PyTorch RoPE reference operator (#167) by @frank-2077
  • feat(ws1): Add PyTorch matmul reference operator (#168) by @frank-2077
  • feat(ws1): NativeEmbeddingOp pure-PyTorch ground-truth reference + numerical contract tests (#169) by @maxiaosong1124
  • feat(ws1): NativeLMHeadOp pure-PyTorch ground-truth reference + numerical contract tests (#170) by @maxiaosong1124
  • [WS1][kernels] Batch-invariant deterministic GEMM (fwd + bwd) (#180) by @Flink-ddd
  • feat(kernels): add fused masking + variable-length pack-and-pad op (#182) by @Chen-BUPT
  • :feat(ws1): add NativeAttentionOp pure-PyTorch standard-softmax reference (#188) by @maxiaosong1124
  • [FEAT][kernels] Add tensor-parallel linear_logp path (#189) by @inaniloquentee
  • feat(ws1): add NativeKVCacheAttnOp pure-PyTorch KV-cache attention reference (#195) by @maxiaosong1124
  • feat(ws1):Add kernel gtest operator checker and used logprob to test gtest (#197) by @frank-2077
  • [WS1][kernels] Batch-invariant logprob (Native, Triton) (#199) by @hihaluemen
  • feat(kernels): add batch-invariant RMSNorm (#201) by @EthanZero2Hero
  • [WS2][RFC] Clarify cross-config logprob drift contract (#207) by @CyberSecurityErial
  • [WS1]test: relax native matmul batch-gradient comparison (#224) by @frank-2077
  • [WS2] Add CP-aware Attention contract and dispatch metadata (#236) by @Bsgg1
  • feat(attention): add deterministic CP reference (#238) by @inaniloquentee
  • feat(kernels): add Triton batch-invariant attention (#246) by @EthanZero2Hero
  • feat(attention): add single-gpu comparison harness (#253) by @inaniloquentee
  • [PERF][kernels] Write ratio_kl backward directly to policy dtype (#257) by @hongleng
  • feat(attention): add decode-stage KV-cache CP replay (#260) by @JLiu4Coding
  • [WS2][PR2][Logp] feat: add TP=1 logprob comparison harness (#262) by @hihaluemen
  • [WS2][PR3][Logp] Add deterministic vocab-parallel TP logprob reference (#265) by @KJLdefeated
  • [WS2][PR7][Attention] Add FlashInfer RoPE-fused paged attention scaffold (#279) by @inaniloquentee
  • [WS2][PR5][Logp] Add deterministic DP-aware GRPO loss on the TP-aware logprob path (#291) by @KJLdefeated
  • perf(gemm):Optimize deterministic GEMM tensor layouts (#343) by @frank-2077
  • Feat/strict embedding backward (#349) by @leonardHONG
  • Feat/strict qk norm reduction (#350) by @leonardHONG
  • [perf][gemm]: tile short-token FFN weight gradients (#351) by @Flink-ddd
  • [perf][gemm]: optimize long-token FFN deterministic path (#354) by @maxiaosong1124

Reproducibility & Benchmarking

  • feat: add RL-shaped kernel benchmark smoke runner (#43) by @inaniloquentee
  • feat(benchmarking): add automated performance profiling suite (#66) by @bitborne
  • [Bugfix][benchmarks]: fix VRAM baseline and add TRL comparison (#79) by @Flink-ddd
  • [WS2][RFC] Add modular cross-config alignment roadmap (#222) by @CyberSecurityErial
  • [WS2][GEMM][PR4]: add PR4 forward integration validation for Qwen3-8B (TP=2, CP=2) (#281) by @Flink-ddd
  • [Revert][WS2][GEMM][PR4]: add PR4 forward integration validation for Qwen3-8B (TP=2, CP=2) (#282) by @Flink-ddd
  • feat(attention): add CP backward drift validation (#284) by @inaniloquentee
  • [WS2][GEMM][3th merge] feat: add wrapper to ablation matrix (#322) by @frank-2077
  • Add profile-driven VIME reproduction launcher (#426) by @inaniloquentee
  • Add Qwen3-8B user guide, ROCm, and topology workflows (#430) by @inaniloquentee

Documentation & CI

Contributors

Thanks to everyone who contributed code, testing, hardware validation, documentation, infrastructure and review to RL-Kernel v0.1.0.

@Arlo-mt @Bsgg1 @Chen-BUPT @CyberSecurityErial @Dnoob @EthanZero2Hero @FED4
@Flink-ddd @JLiu4Coding @KJLdefeated @Liuzin521 @MengjieLee @RocMarshal @Zhifu-Liu
@bitborne @erfgss @frank-2077 @haoruilee @hihaluemen @hongleng @inaniloquentee
@leonardHONG @luoyueyuguang @maxiaosong1124 @ryankert01 @smarslou @z1ying @zhangj1an

Full History

https://github.com/RL-Align/RL-Kernel/commits/v0.1.0

Contributors

JLiu4Coding, MengjieLee, and 26 other contributors
Assets 2
Loading