Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[llm] Added support for bfloat16 with deepspeed #3403

Merged
merged 2 commits into from
May 16, 2023
Merged

[llm] Added support for bfloat16 with deepspeed #3403

merged 2 commits into from
May 16, 2023

Conversation

tgaddair
Copy link
Collaborator

@tgaddair tgaddair commented May 16, 2023 •
edited

Example:

strategy:
  type: deepspeed
  bf16:
    enabled: true

@tgaddair tgaddair requested a review from arnavgarg1 May 16, 2023 19:19
arnavgarg1
arnavgarg1 approved these changes May 16, 2023
View reviewed changes
Copy link
Contributor

@arnavgarg1 arnavgarg1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Is this a top level key in the deepspeed config?

@tgaddair
Copy link
Collaborator Author

@arnavgarg1 , yes, added example in description.

@arnavgarg1
Copy link
Contributor

Nice!

@tgaddair tgaddair merged commit b9bece4 into master May 16, 2023
12 of 14 checks passed
@tgaddair tgaddair deleted the bfloat branch May 16, 2023 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@arnavgarg1 arnavgarg1 arnavgarg1 approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@tgaddair @arnavgarg1