Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pass use_triton flag to LayerNorm module #336

Merged
merged 1 commit into from Jun 21, 2022
Merged

Pass use_triton flag to LayerNorm module #336

merged 1 commit into from Jun 21, 2022

Conversation

SeanNaren
Copy link
Contributor

@SeanNaren SeanNaren commented Jun 21, 2022 •
edited

What does this PR do?

When creating the model, use_triton isn't passed to the Norm module in the factory. This is required when skipping the triton-based layer norm implementation when using BFloat16.

Before submitting

  • Did you have fun?
    • Make sure you had fun coding 馃檭
  • Did you read the contributor guideline?
  • Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
    • N/A
  • Did you make sure to update the docs?
    • N/A
  • Did you write any new necessary tests?
    • N/A
  • Did you update the changelog? (if needed)
    • N/A

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jun 21, 2022
dianaml0
dianaml0 approved these changes Jun 21, 2022
View reviewed changes
Copy link
Contributor

@dianaml0 dianaml0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @SeanNaren !

@dianaml0 dianaml0 merged commit 22c1bab into facebookresearch:main Jun 21, 2022
@SeanNaren SeanNaren deleted the fix/pass_triton branch June 22, 2022 18:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dianaml0 dianaml0 dianaml0 approved these changes

Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@SeanNaren @dianaml0 @facebook-github-bot