Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support freeze parameters #1544

Merged
merged 2 commits into from Mar 24, 2022
Merged

Support freeze parameters #1544

merged 2 commits into from Mar 24, 2022

Conversation

zachgk
Copy link
Contributor

@zachgk zachgk commented Mar 24, 2022

This adds a method to support freezing and unfreezing a parameter for transfer
learning. There is also a helper to (un)freeze all paremters in a block, but
without filtering. For more advanced use cases of (un)freezing part of the
parameters in a block, it should be implemented using block.getChildren(),
block.getDirectParameters() and/or block.getParameters().

fixes #392
fixes #1494

@zachgk zachgk requested a review from frankfliu as a code owner March 24, 2022 17:23
This adds a method to support freezing and unfreezing a parameter for transfer
learning. There is also a helper to (un)freeze all paremters in a block, but
without filtering. For more advanced use cases of (un)freezing part of the
parameters in a block, it should be implemented using block.getChildren(),
block.getDirectParameters() and/or block.getParameters().
api/src/main/java/ai/djl/nn/Parameter.java Outdated Show resolved Hide resolved
api/src/main/java/ai/djl/nn/Block.java Outdated Show resolved Hide resolved
@codecov-commenter
Copy link

codecov-commenter commented Mar 24, 2022

Codecov Report

Merging #1544 (cef2f29) into master (bb5073f) will decrease coverage by 1.25%.
The diff coverage is 50.84%.

@@             Coverage Diff              @@
##             master    #1544      +/-   ##
============================================
- Coverage     72.08%   70.82%   -1.26%     
- Complexity     5126     5363     +237     
============================================
  Files           473      500      +27     
  Lines         21970    23373    +1403     
  Branches       2351     2541     +190     
============================================
+ Hits          15838    16555     +717     
- Misses         4925     5544     +619     
- Partials       1207     1274      +67     
Impacted Files Coverage Δ
api/src/main/java/ai/djl/modality/cv/Image.java 69.23% <ø> (-4.11%) ⬇️
...i/djl/modality/cv/translator/BigGANTranslator.java 21.42% <ø> (-5.24%) ⬇️
...odality/cv/translator/BigGANTranslatorFactory.java 33.33% <0.00%> (+8.33%) ⬆️
...nslator/InstanceSegmentationTranslatorFactory.java 14.28% <0.00%> (-3.90%) ⬇️
.../modality/cv/translator/YoloTranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...i/djl/modality/cv/translator/YoloV5Translator.java 5.69% <0.00%> (ø)
...odality/cv/translator/YoloV5TranslatorFactory.java 8.33% <0.00%> (-1.67%) ⬇️
...pi/src/main/java/ai/djl/ndarray/BytesSupplier.java 54.54% <0.00%> (-12.13%) ⬇️
...pi/src/main/java/ai/djl/repository/Repository.java 83.33% <ø> (ø)
...l/training/loss/SigmoidBinaryCrossEntropyLoss.java 64.00% <0.00%> (ø)
... and 207 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 33c29ed...cef2f29. Read the comment docs.

@zachgk zachgk merged commit 2566416 into deepjavalibrary:master Mar 24, 2022
@zachgk zachgk deleted the freeze branch March 24, 2022 21:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Freezing some parameters during backpropagation Freeze a block for training
3 participants