-
Notifications
You must be signed in to change notification settings - Fork 490
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Optimizer readme #680
Optimizer readme #680
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good, I think you are on a good point here - we do need to add registry for optimisers.
Lets add it and update how one can use their own optimiser like you suggested.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we do have a register_optimizer function now. so you can add this as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also another very important point:
We need to add docs for initialise_param_groups and update_param_groups, and demonstrate how it interacts with the optimiser. Thats the only way now we have in SG to support different configurations among different the different network parameters.
* first draft * comments * example added at top * warning about ddp samplers moved to subsection * comments resolved * grammar * re-phrase modules import explanation --------- Co-authored-by: Ofri Masad <ofrimasad@users.noreply.github.com>
I added a section, do you think its enough? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks very nice.
I think there is a minor issue in the code
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
The requirements were as follows:
Concerning "How to use your own optimizers": we currently don't support it because the only way to use an optimizer is to pass it as a string, and we don't have a registry for optimizers. Do we want to add this registry ?
Besides this, I don't see what else to show in this notebook, if you have an idea.