Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[booster] implemented mixed precision class #3151

Merged
merged 4 commits into from
Mar 17, 2023

Conversation

FrankLeeeee
Copy link
Contributor

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

Fixed #3054

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

This PR implemented the MixedPrecision components for the booster module. The API design is slightly different from that specified in #3054 with the following changes:

  1. created interface classes
  2. let optimizer handle gradient clipping
  3. modified the setup API for future extension like apex

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@FrankLeeeee FrankLeeeee self-assigned this Mar 16, 2023
@FrankLeeeee FrankLeeeee added the enhancement New feature or request label Mar 16, 2023
@FrankLeeeee FrankLeeeee enabled auto-merge (squash) March 17, 2023 02:42
@FrankLeeeee FrankLeeeee merged commit ed19290 into hpcaitech:main Mar 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

[booster] implement precision
2 participants