[Feature] [Kernel] cursor warp decode kernel for low latency small batch MOE inference

[ZelinMa557]

Checklist

• If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
• Please use English. Otherwise, it will be closed.

Motivation

In cursor's blog https://cursor.com/blog/warp-decode, cursor designs a new type of MOE kernel for low latency small batch MOE inference.

TokenSpeed also dispatches warp decode style kernel at small batch size (https://pytorch.org/blog/lightseek-tokenspeed-kernel/):

I tried to reproduce the result on Nvidia hopper GPUs under bf16 precision, and achieved up to 2.0x speedup on MOE ffn layer (https://github.com/ZelinMa557/warp_decode):

Maybe sglang can consider integrate warp decode style MOE kernels?

Reference:

• https://cursor.com/blog/warp-decode
• https://pytorch.org/blog/lightseek-tokenspeed-kernel/
• https://github.com/ZelinMa557/warp_decode

Related resources

No response