This repository has been archived by the owner on Nov 22, 2022. It is now read-only.
Division by Zero bug in MLM Metric Reporter #968
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
Summary: Pull Request resolved: facebookresearch#968 While using fp16 during MLM training there's a weird bug during metric reporting that causes a division by zero failure while computing the training speed (an example is below). Preventing this failure by adding a small value to the denominator. Example: f136667996 Reviewed By: chenyangyu1988 Differential Revision: D17293910 fbshipit-source-id: 2243e5fecb05ac666c3e084900e95d428ec55428
kartikayk
force-pushed
the
export-D17293910
branch
from
876b47e
to
d17ee9e
Compare
|
This pull request has been merged in f64cb10. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
While using fp16 during MLM training there's a weird bug during metric reporting that causes a division by zero failure while computing the training speed (an example is below). Preventing this failure by adding a small value to the denominator.
Example: f136667996
Reviewed By: chenyangyu1988
Differential Revision: D17293910