PyTorch implementation of FlashMLA.
FlashMLA is an efficient MLA decoding kernel for Hopper GPUs, optimized for variable-length sequences serving. Currently released: BF16; Paged kvcache with block size of 64.
ccc0168/FlashMLA-PyTorch
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|