Optimizer readme

[Louis-Dupont]

The requirements were as follows:

• SG Optimizers
• How to use PyTorch native optimizers
• How to use your own optimizers (including in a recipe)

Concerning "How to use your own optimizers": we currently don't support it because the only way to use an optimizer is to pass it as a string, and we don't have a registry for optimizers. Do we want to add this registry ?
Besides this, I don't see what else to show in this notebook, if you have an idea.

[dagshub]

Join the discussion on DagsHub!

[shaydeci]

Looks good, I think you are on a good point here - we do need to add registry for optimisers.
Lets add it and update how one can use their own optimiser like you suggested.

[ofrimasad]

we do have a register_optimizer function now. so you can add this as well

[shaydeci]

Also another very important point:

We need to add docs for initialise_param_groups and update_param_groups, and demonstrate how it interacts with the optimiser. Thats the only way now we have in SG to support different configurations among different the different network parameters.

[Louis-Dupont]

Also another very important point:

We need to add docs for initialise_param_groups and update_param_groups, and demonstrate how it interacts with the optimiser. Thats the only way now we have in SG to support different configurations among different the different network parameters.

I added a section, do you think its enough?

[ofrimasad]

Looks very nice.
I think there is a minor issue in the code

[ofrimasad]

LGTM!