[PyTorch][Fix] Improve numerical stability of HistogramObserver #86522
Conversation
|
|
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/86522
Note: Links to docs will display an error until the docs builds have been completed. ✅ No Failures, 7 PendingAs of commit 557876e: This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
This pull request was exported from Phabricator. Differential Revision: D40149594 |
…rch#86522) Summary: Pull Request resolved: pytorch#86522 As titled, HistogramObserver may fail in a certain scenario. Specifically, we originally compute `hist_bin_width` as `(self.max_val - self.min_val) / (self.bins * upsample_rate)`. It's possible that the numerator part is close the the FP32 threshold (1.4e-45) and conducting the division will cause overflow. Bring some redundent computations to avoid such scenario. Test Plan: https://pxl.cl/2ggD4 (pytorch@04490e9) Reviewed By: jerryzh168 Differential Revision: D40149594 fbshipit-source-id: e14564cd2c65ab32b2600adf383f1d807af197ac
yulin0077
force-pushed
the
export-D40149594
branch
from
3b40a90
to
557876e
Compare
|
This pull request was exported from Phabricator. Differential Revision: D40149594 |
|
@pytorchbot merge (Initiating merge automatically since Phabricator Diff has merged) |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
Hey @yulin0077. |
Summary: Pull Request resolved: #86522 As titled, HistogramObserver may fail in a certain scenario. Specifically, we originally compute `hist_bin_width` as `(self.max_val - self.min_val) / (self.bins * upsample_rate)`. It's possible that the numerator part is close the the FP32 threshold (1.4e-45) and conducting the division will cause overflow. Bring some redundent computations to avoid such scenario. Test Plan: https://pxl.cl/2ggD4 (4fc0d53) Reviewed By: jerryzh168 Differential Revision: D40149594 fbshipit-source-id: 49ed9980d7a641c14dda5a103e76c0c16f29c665
Summary:
As titled, HistogramObserver may fail in a certain scenario.
Specifically, we originally compute
hist_bin_widthas(self.max_val - self.min_val) / (self.bins * upsample_rate). It's possible that the numerator part is close the the FP32 threshold (1.4e-45) and conducting the division will cause overflow.Bring some redundent computations to avoid such scenario.
Test Plan: https://pxl.cl/2ggD4 (04490e9)
Differential Revision: D40149594