-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add BatchNorm to AdditiveSharingTensor #3889
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3889 +/- ##
=======================================
Coverage 94.98% 94.99%
=======================================
Files 186 186
Lines 19075 19122 +47
=======================================
+ Hits 18119 18164 +45
- Misses 956 958 +2
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Just left a small suggestion
for i in range(80): | ||
if x is not None: | ||
y = C + 1 - var * (x * x) | ||
x = y * x / C | ||
else: | ||
y = C + 1 - var | ||
x = y / C |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why not executing the else outside, and removing condition in the for loop, since it's 1 vs 79 times.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree, however we have 80 for bad reason, in the near future we'll have 3 to 5 iterations :)
I'm waiting for the FALCON team to do this :D
Description
Add Batchnorm and tests
Fix a bug in .var() for evaluation on a specific dim
NOTE This implem will be improved with the current work on FALCON
Checklist