-
Notifications
You must be signed in to change notification settings - Fork 45.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added a flag to allow skipping the first projection in small ResNets #10584
base: master
Are you sure you want to change the base?
Added a flag to allow skipping the first projection in small ResNets #10584
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you!
There is a failure detected by internal tests. I think we forgot to update configs. @ByzanTine |
@saberkun , indeed I had not updated the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please explain the relationship between use_first_projection
and self._use_first_projection
.
Description
This should fix #10583
In this issue, I described how the small ResNets implemented here have an extra convolution (for projection), that is neither present in the original paper or in PyTorch.
This PR allows to get rid of this extra convolution with a keyword argument that defaults to the previous behaviour so that this remains a non-breaking change.
Type of change
For a new feature or function, please create an issue first to discuss it
with us before submitting a pull request.
Note: I didn't wait for a discussion because this seemed like a relatively small and simple change,
so it's not bothering for me to have written it even if rejected in the end.
Tests
Test Configuration:
I added tests to check the parameter count.
I also offline checked the number of non-trainable parameters to checked that it matched against PyTorch's one.
Checklist