Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
Implement MX4 quantization-dequantization ops
Usage:
Quantization:
where
A
is 1-D input tensor andsplit_sizes
is list of int containing number of elements in each rank e.g.,split_sizes = [1024, 2048]
for 2 ranks. Note that each value needs to be a multiple ofgroup size
.The output is ensured to be 16-byte aligned. Anything less than 16 bytes is padded to make 16 bytes.
Given that the
group_size
is 32, for the best performance, the value should be a multiple of 32 x 16 = 512.Dequantization:
Reviewed By: sryap
Differential Revision: D57145102