Skip to content

Conversation

@lazyoracle
Copy link
Member

What

Update tf_sgd to use maxiters and include note on WIP TF Optimizers

Why

Closes #102 and associates

How

  • Add warnings to the adam, adadelta and rmsprop optimizers. These have incomplete/incorrect implementations while some of the default hyper-parameters have been altered without adequate reasoning or comments. Pending further investigations into these algorithms or without the option to alter these hyper-parameters from the user-space, these are essentially buggy and must be avoided. Also added # TODOs to make note of what needs to be done
  • Updated the tf_sgd optimizer to use the correct maxiters parameter instead of maxfun. This is because these optimizers allow for directly controlling the number of algorithm iterations and not function evaluations.
  • Added an Error Handling code in case someone passes maxfun to this optimizer
  • Provided an option for setting the learning rate and momentum from the user-space
  • Added and updated tests to also check for error handling code

@lazyoracle lazyoracle added bug Something isn't working tensorflow Tensorflow Performance and Implementation labels May 28, 2021
@lazyoracle lazyoracle added this to the 1.3 milestone May 28, 2021
@lazyoracle lazyoracle self-assigned this May 28, 2021
@codecov
Copy link

codecov bot commented May 28, 2021

Codecov Report

Merging #104 (2fc4436) into dev (b6dd98f) will increase coverage by 0.02%.
The diff coverage is 66.66%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #104      +/-   ##
==========================================
+ Coverage   57.32%   57.34%   +0.02%     
==========================================
  Files          35       35              
  Lines        5279     5291      +12     
==========================================
+ Hits         3026     3034       +8     
- Misses       2253     2257       +4     
Impacted Files Coverage Δ
c3/libraries/algorithms.py 19.23% <66.66%> (+1.71%) ⬆️
c3/optimizers/c1.py 73.73% <0.00%> (+1.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6dd98f...2fc4436. Read the comment docs.

@lazyoracle lazyoracle merged commit 7b1ccc4 into q-optimize:dev May 28, 2021
@lazyoracle lazyoracle deleted the tf-optim-fix branch May 28, 2021 22:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working tensorflow Tensorflow Performance and Implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Optimizer API Mismatch between Tensorflow and Scipy Optimizers

1 participant