We need a KDA decode kernel

[icavan]

WIP, as titled

[yuan-luo]

I integrated cuLA prefill with cuteDSL decode kda in sglang. I can take this issue and submit the cutedsl decode kda kernel to cuLA.
sgl-project/sglang#22107

  GSM8K result (cuLA prefill + CuTeDSL decode, with CUDA graph):

  ┌───────────────────┬──────────────────────────────────┐
  │       Index       │                value             │
  ├───────────────────┼──────────────────────────────────┤
  │ Score             │ 0.88 (200 examples, > 0.85 )     │
  ├───────────────────┼──────────────────────────────────┤
  │ Total latency     │ 10.1s                            │
  ├───────────────────┼──────────────────────────────────┤
  │ Output throughput │ 2050.6 token/s                   │
  ├───────────────────┼──────────────────────────────────┤
  │ CUDA graph        │ Enabled                          │
  └───────────────────┴──────────────────────────────────┘

[yuan-luo]

#37

[icavan]

Thanks for your contribution @yuan-luo