Optimization of the standalone permute path

[Autumn1998]

1. Add new permute kernel, optimize the perf at the overlap case
2. unify the preprocess part of the fuse-path and non-fuse-path
3. Fix out-of-bound error on scan kernel

[dongwang4096]

From what I can tell, num_permuted_tokens is used to size the output tensors, but the standalone permute kernel still writes according to the actual destinations generated by preprocessing. If the provided value is smaller than the real padded token count, this could lead to writes past the allocated output buffer.

Would it make sense to either validate the provided capacity against the actual padded token count in blocking mode, or apply the same clipping/overflow handling there as well?

[Autumn1998]

From what I can tell, num_permuted_tokens is used to size the output tensors, but the standalone permute kernel still writes according to the actual destinations generated by preprocessing. If the provided value is smaller than the real padded token count, this could lead to writes past the allocated output buffer.

Would it make sense to either validate the provided capacity against the actual padded token count in blocking mode, or apply the same clipping/overflow handling there as well?

This truncation occurs during the scan — tokens exceeding num_permute_tokens will be marked as 'do not send' in the routing map after preprocessing, so the subsequent dispatch and permute operations will communicate based on the trimmed routing information.