[CAKE] Long-term CAKE-generated Kernel Progress Tracker

[yyihuang]

To track multi-platform kernels generated from CAKE 🍰.

We are more than happy to take any new kernel challenges!

Feel free to reach out/leave your comments/create sub-issues here to submit your customized CAKE kernel request. And we will bake it for you!

We know many of you have been asking for training kernels—and we heard you. You have had your CAKE going forward; now it is time to take it back! Follow the dedicated forward & backward tracker at #4642, bring us your toughest training bottleneck, and let us bake the full pass together. 🍰🔥

Pull requests

Recurrent KDA

Prefill

• feat(cake_kda): add optimized B200 recurrent prefill backend #4262
• feat(cake_kda): share recurrent prefill kernels across SM100 family #4313
• fix(cake_kda): support non-aligned recurrent prefill head counts #4351
• perf(cake_kda): add BT16 recurrent-KDA prefill portfolio #4675

Decode

• feat(cake_kda): add optimized B200 recurrent decode backend #4279
• feat(cake_kda): share recurrent decode kernels across SM100 family #4314
• feat(cake_kda): add serving-native packed decode backend #4378
• feat(cake_kda): add optimized H12 packed decode across SM100 family #4562

Prefill and decode

• feat(cake_kda): add strided prefill state checkpoints and packed decode #4445
• feat(cake_kda): add native unbounded-softplus Kimi-Linear kernels #4535
• feat(cake_kda): optimize small-BH recurrent-KDA prefill #4571

Gated Delta Rule

Context-parallel prefill

• feat(cake_gdn): add optimized SM100/SM103 context-parallel prefill backend #4539, from feat: add sm100 cp delta rule prefill #4078

Intermediate cache stride

• fix(gdn): honor intermediate cache view strides #4572 — honor intermediate cache step strides

Source-only Blackwell backend

• feat(cake_gdn): add source-only Cake backend for SM100 and SM103 #4581 — source-only Cake backend for SM100 and SM103

GEMM / BMM

BF16 BMM

• feat(bf16-bmm): add opt-in CAKE backend for SM100a and SM103a #4370 — opt-in CAKE backend for SM100a and SM103a

Deepgemm - FP8

• feat(cake_batch_deepgemm): add optimized SM100/SM103 batch DeepGEMM FP8 backend #4564

TinyGEMM2

• feat(cake_tinygemm2): add CAKE-generated SM100/SM103 tinygemm2 variants with bit-identical outputs #4274

Blackwell Router GEMM

• feat(cake_backend): add Blackwell Router GEMM #4594

AlphaMoE

• feat(cake_alphamoe): add optimized Blackwell fused W8A8 block-scale MoE #4287
• feat(cake_alphamoe): add Blackwell fused routing frontend #4339
• feat(cake_alphamoe): add Blackwell fused NVFP4 MoE #4340

DeepSeek Fused Routing

• feat(cake_backend): accelerate DeepSeek fused routing #4587 — current Cake/FlashInfer: B200 1.0194–1.2101x and GB300 1.2665–1.5738x (15/15 each)

Softmax

• feat(cake_softmax): add optimized Blackwell softmax #4282

FMHA

MiniMax Sparse Attention

• feat(cake_msa): add Blackwell minimal sparse attention source kernels #4355

NVFP4 Attention

• feat(attention): add explicit Wan hybrid attention backend #4543

TRTLLM Batch FMHA with KV Cache

• feat(cake_fmha): add native Blackwell DCP speculative decode #4518

VSA Block Sparse Attention

• feat(cake_vsa): add optimized SM100/SM103 block-sparse attention (VSA) #4593

MLA

TRTLLM MLA

• feat(cake_trtllm_mla): add Blackwell MLA source backend #4557 — Blackwell MLA source backend

TRTLLM Sparse MLA DSV4

• feat(cake_sparse_mla): add SM103 sparse MLA DSV4 backend #4573

Mamba

SSDCombined

• feat(cake_mamba): add Blackwell Mamba SSDCombined #4576

Selective State Update

• feat(cake_mamba): add Blackwell selective state update backend #4616