Simplify training: use Adam and remove the need for manual tuning

[daniel-j-h]

At the moment we are using stochastic gradient descent and a multi-step weight decay policy.

In this setup the user has to set

• the initial sgd learning rate
• the sgd momentum to use
• the weight decay milestones
• the weight decay factor

robosat/robosat/tools/train.py

Lines 83 to 85 in 2067cb7

	optimizer = SGD(net.parameters(), lr=model["opt"]["lr"], momentum=model["opt"]["momentum"])
	scheduler = MultiStepLR(optimizer, milestones=model["opt"]["milestones"], gamma=model["opt"]["gamma"])

And while this allows for great flexibility and control over details it might be to complicated for our users. We should look into replacing our current setup e.g. with the Adam optimizer only setting the initial learning rate and the weight decay.

We can then set these two values to reasonable defaults and users can get started without thinking too much about parameters and without having to run multiple experiments just to get basic parameters figured out.

Tasks

• Implement Adam optimizer with learning rate and weight decay
• Benchmark and check results; if it looks reasonable go for it
• Remove sgd parameters from config; use learning rate and weight decay only