Adam with decay

[benedict-96]

Implements a version of the Adam optimizer where we have

$$\eta = \exp\left(log\left(\frac{\eta_2}{\eta_1}\right) / \mathtt{n\_epochs}\right)^t\eta_1$$

for the learning rate.

[codecov]

Codecov Report

Attention: Patch coverage is 84.61538% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 63.31%. Comparing base (af93bf5) to head (d300189).
Report is 9 commits behind head on main.

Files	Patch %	Lines
...imizers/adam_optimizer_with_learning_rate_decay.jl	80.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #130      +/-   ##
==========================================
+ Coverage   62.53%   63.31%   +0.77%     
==========================================
  Files         111      114       +3     
  Lines        3222     3260      +38     
==========================================
+ Hits         2015     2064      +49     
+ Misses       1207     1196      -11     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.