-
Notifications
You must be signed in to change notification settings - Fork 161
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change add_noise to match the target variance by scaling by its sqrt. #294
Change add_noise to match the target variance by scaling by its sqrt. #294
Conversation
Edited to avoid sqrt after type casting. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Thank you so much for this bug fix and the associated test!
I just left a few very minor comments/nits.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks a lot for making the changes! I forgot to check for chex.all_variants
last time so I have one more comment. Otherwise looks good to me!
Add a test for this noise scaling.
std -> standard_deviation shape update like params rename test_add_noise -> test_add_noise_has_correct_variance_scaling
remove parameterization; hard code parameters locally update with variants
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Thanks a lot!
There is a test failure now in the (recently added) test for noisy_sgd
in alias_test.py
which uses add_noise
. Do you have time to look into this? It's probably just a matter of changing the parameters so that with the slower decay of the noise it doesn't affect the test results but we should make sure this is indeed the reason.
I've rebased and changed eta to match the previous initial variance. It survives the test with the different decay rate.
The test passes now (and also passed if just removing the noisy_sgd line). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks a lot for doing this!
Resolve issue #281.
Add a test for this noise scaling which fails prior to this change.