Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Added custom weight decay for normalization layers #162

Merged
merged 8 commits into from
Nov 7, 2021
Merged

Conversation

frgfm
Copy link
Owner

@frgfm frgfm commented Nov 6, 2021

This PR introduces the following modifications:

  • adds an option in trainer methods to specify normalization layers' weight decay
  • updates unittests
  • adds the possibility in training scripts
  • updates mypy config

Closes #107

@frgfm frgfm added this to the 0.2.0 milestone Nov 6, 2021
@frgfm frgfm self-assigned this Nov 6, 2021
@codecov
Copy link

codecov bot commented Nov 6, 2021

Codecov Report

Merging #162 (670317b) into master (ee903c4) will decrease coverage by 0.05%.
The diff coverage is 95.12%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #162      +/-   ##
==========================================
- Coverage   93.96%   93.91%   -0.06%     
==========================================
  Files          51       51              
  Lines        2981     3005      +24     
==========================================
+ Hits         2801     2822      +21     
- Misses        180      183       +3     
Impacted Files Coverage Δ
holocron/trainer/utils.py 95.00% <90.47%> (-5.00%) ⬇️
holocron/trainer/core.py 90.99% <100.00%> (-0.40%) ⬇️
holocron/optim/wrapper.py 85.93% <0.00%> (+0.78%) ⬆️

@frgfm frgfm merged commit cd3a624 into master Nov 7, 2021
@frgfm frgfm deleted the norm-wd branch November 7, 2021 04:51
@frgfm frgfm added type: new feature and removed type: enhancement New feature or request labels Jan 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LAMB: Differences from the paper author's official implementation
1 participant