Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.1 - tf.keras, TF2, Eager compatibility, more #16

Closed
wants to merge 1 commit into from

Conversation

OverLordGoldDragon
Copy link
Owner

New features:

  • Full compatibility with TensorFlow 2.0.0 and Keras 2.3.0 (keras + tf.keras)
  • Full compatibility with TensorFlow 1.14.0 and Keras 2.2.5 (keras + tf.keras)
  • Run-based weight decay normalization scheme, normalizing over arbitrary # of iterations independent of LR scheduler (e.g. over all epochs) (total_iterations_wd)
  • utils now contain all common optimizer ops, e.g. _apply_weight_decays()
  • Reference implementation compatible with tf.python.keras, TF2 + Keras 2.3.0; see docstring
  • Cleaner code

BUGFIXES:

  • lr_multipliers were not being applied for AdamW
  • SGDW was missing K.symbolic wrapper
  • and total_iterations != 0 was missing for NadamW and SGDW, yielding NaN optimizer weights for total_iterations = 0
  • TF reset seed method was incompatible w/ TF 2.0.0; added support for both

Misc changes:

  • Updated README.md w/ info on total_iterations_wd
  • Added README.md to keras_adamw on module version selection
  • Added unit tests for respective optimizers
  • Revamped test.sh; each test has a dedicated folder to ease coverage isolation
  • Revamped .travis.yml:
    • Added versioning builds w/ TF_KERAS, TF_EAGER, TF_VERSION, and KERAS_VERSION flags
    • Removed boolean KERAS flag; repo is based on Keras
    • Reorder buggy setuptools & conda update -all commands
  • Moved testing module requirements from requirements.txt to requirements-test.txt

@OverLordGoldDragon OverLordGoldDragon added the enhancement New feature or request label Oct 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant