Ambiguous whether learningRate should be set on config, or on Updater.

[stolsvik]

In conjunction with #3891 and #3890, a new problem appears:

If you use the concrete-class variant of specifying Updater (to enable you to set the momentum), you get the method setLearningRate(..). If using the Builder, you get the fluent variant of it as such:

.updater(new Nesterovs.Builder().learningRate(0.001d).momentum(0.98d).build())

However, this won't work, as it is the learningRate set directly on the NeuralNetConfiguration.Builder that is used. This defaults to 1e-1. So evidently the build employs the updater.setLearningRate(..), overwriting whatever the user has set on it.

Maybe a solution could be to not overwrite if the user has explicitly set it, i.e. use the idiom where it is default double.NaN, only overwrite if it still is NaN when building. Or something like that.

Also - as I've understood, the Updater can be layer-specific. This would then need to be handled for this case too - i.e. setting a specific learning rate on an Updater for a specific layer should not be overriden by the base config.

[AlexDBlack]

Now: only set on updater (no more .learningRate(double) method)
#3985

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.