Add better learning rate annealing scheduler + Learning rate finder

[mhham]

Two ideas that could allow improving training :

1. Add learning rate finder : as explained in here, and implemented here

2. Cosine Annealing with restart : https://github.com/roveo/pytorch/pull/1/commits/35891c46dbdec46e55a5fb725418c2880f631dac

[alanakbik]

That sounds like a great idea - if fact we've been recently thinking a lot about better methods to find good parameters than a full parameter sweep. Would you be interested in testing/integrating such approaches?

[alanakbik]

@kashif what do you think?

[kashif]

@mhham @alanakbik Indeed I have a PR in preparation that adds weight decay versions of optimizers after which we can add the corresponding schedulers that not only change the learning rate, but also the weight decay factor or momentum

[alanakbik]

Learning rate finder was just added for release-0.4 - we'll check out the cosine annealing in a later version.