Float8 cache usage

[YLGH]

Hi! I'm playing with batch_decode_with_padded_kv_cache and wanted to test out the FP8 KVCache. I couldn't find some good instructions on the docs,

I've tried the following:

num_qo_heads = 32
num_kv_heads = 32
batch_size = 16
head_dim = 128 
padded_kv_len = 1024


q = torch.empty(
                batch_size,
                num_qo_heads,
                head_dim,
                device=torch.device("cuda"),
                dtype=torch.float8_e4m3fn,
            )
k_padded = torch.randn(batch_size, padded_kv_len, num_kv_heads, head_dim).to("cuda:0").to(torch.float8_e4m3fn)
v_padded = torch.randn(batch_size, padded_kv_len, num_kv_heads, head_dim).to("cuda:0").to(torch.float8_e4m3fn)
o = flashinfer.batch_decode_with_padded_kv_cache(
    q, k_padded, v_padded, "NHD", "NONE"
)

But it gives me a BatchDecodeWithPaddedKVCache kernel launch failed: supported data type.

How can I enable FP8 KV cache? Thanks in advance!

[zhyncs]

refer to #150

[yzh119]

@YLGH done in #156 .