A bug in the tutorial building_image_classifier.ipynb #54
Assignees
Labels
bug
Something isn't working
Comments
|
Yes, this looks like a bug. Thank you for reporting! |
facebook-github-bot
pushed a commit
that referenced
this issue
Sep 14, 2020
Summary: See #54 As correctly pointed out, we don't need to call `optimizer.zero_grad()` after virtual steps, only after real steps. Curiously, it doesn't seem to affect the accuracy: **Claimed in the tutorial**: 56.61 **This diff**: 58.54 **Re-run of the original**: 59.12 I think we need longer training to claim stable result, but in the meantime this change a) doesn't make things worse b) does make sense Reviewed By: sayanghosh Differential Revision: D23650453 fbshipit-source-id: ec1b49b10c9b1467428d46ae298f2ad0e8d142f8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
According to the description of the
virtual_step(self)method in the following lineopacus/opacus/privacy_engine.py
Line 335 in d68b2bd
the command
optimizer.zero_grad()should be called after a real step and not at each step.In the current tutorial, the command
optimizer.zero_grad()is called for every step (real or virtual) which seems to be wrong.opacus/tutorials/building_image_classifier.ipynb
Line 372 in d68b2bd
This line (
optimizer.zero_grad()) should be moved and placed right afteroptimizer.step()The text was updated successfully, but these errors were encountered: