-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature] Support using checkpoint in Swin Transformer to save memory. #557
Conversation
Codecov Report
@@ Coverage Diff @@
## master #557 +/- ##
==========================================
+ Coverage 79.92% 79.94% +0.02%
==========================================
Files 107 107
Lines 6102 6109 +7
Branches 989 1041 +52
==========================================
+ Hits 4877 4884 +7
Misses 1093 1093
Partials 132 132
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
@@ -36,6 +37,9 @@ class SwinBlock(BaseModule): | |||
Defaults to empty dict. | |||
norm_cfg (dict, optional): The config of norm layers. | |||
Defaults to dict(type='LN'). | |||
with_cp (bool, optional): Use checkpoint or not. Using checkpoint | |||
will save some memory while slowing down the training speed. | |||
Default: False. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please use Defaults to False
to keep the same with other docstrings.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
We will merge it after the release of this month.
open-mmlab#557) * add checkpoint in swin backbone * add checkpoint in swin backbone
Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.
Motivation
For larger batch size training in swin transformer.
Modification
Added checkpointing in swin transformer.
Checklist
Before PR:
After PR: