[PyTorch] Fix CP implementation with FP8#1483
Merged
Merged
Conversation
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
Signed-off-by: Xiaowei Ren <xren@nvidia.com>
for more information, see https://pre-commit.ci
Collaborator
Author
|
/te-ci pytorch L1 |
…ngine into xren/cp_v2.0_fix
xrennvidia
changed the title
Collaborator
Author
|
/te-ci pytorch L1 |
Collaborator
Author
|
/te-ci pytorch L1 |
cyanguwa
approved these changes
Feb 18, 2025
Collaborator
cyanguwa
left a comment
cyanguwa
left a comment
There was a problem hiding this comment.
There are some failures in the CI, but it doesn't look like they are related to CP?
Collaborator
Author
Yeah, I think so. All CP tests with H100 passed. Failed tests are for other functional units. |
Collaborator
Author
|
/te-ci pytorch L1 |
Collaborator
Author
|
The failed tests are unrelated to CP, and CP tests on Blackwell GPUs passed locally, so merge the PR. |
timmoon10
pushed a commit
that referenced
this pull request
Feb 21, 2025
* commit some debug code Signed-off-by: Xiaowei Ren <xren@nvidia.com> * add more debug info Signed-off-by: Xiaowei Ren <xren@nvidia.com> * debug code commit and typo fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> * a typo fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> * remove debug info Signed-off-by: Xiaowei Ren <xren@nvidia.com> * do not return lse Signed-off-by: Xiaowei Ren <xren@nvidia.com> * add amax_per_step for quantizers of CP Signed-off-by: Xiaowei Ren <xren@nvidia.com> * fix FP8 + CP Signed-off-by: Xiaowei Ren <xren@nvidia.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * bug fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> * bug fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> * dtype fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> * bug fix Signed-off-by: Xiaowei Ren <xren@nvidia.com> --------- Signed-off-by: Xiaowei Ren <xren@nvidia.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Xiaowei Ren <xren@login-preos01.a51.clusters.nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fix bugs of quantizations, dtypes, etc. of FP8+CP
Type of change
Checklist: