-
Notifications
You must be signed in to change notification settings - Fork 55
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add VisionTransformer model and training cfg #74
Conversation
fix memory problem
…into dev_vit_train
@@ -0,0 +1,20 @@ | |||
from libai.config import LazyCall |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个文件名要不改成 vit,更简单一点
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
我一开始是感觉太简单了所以才改成这个2333
configs/vit_imagenet.py
Outdated
optim.weight_decay = 0.05 | ||
|
||
# Set pipeline layers for paralleleism | ||
train.dist.pipeline_num_layers = model.cfg.depth |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这一行可以先暂时删掉
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK,我update一下
libai/config/arguments.py
Outdated
@@ -63,6 +63,8 @@ def default_argument_parser(epilog=None): | |||
"See documentation of `DefaultTrainer.resume_or_load()` for what it means.", | |||
) | |||
parser.add_argument("--eval-only", action="store_true", help="perform evaluation only") | |||
# TODO: Add fast debug mode | |||
parser.add_argument("--fast_debug", action="store_true", help="quickly debug") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
像上面那样,写成 --fast-debug
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
好,我修复一下
tools/train_vit.sh
Outdated
@@ -0,0 +1,16 @@ | |||
#!/usr/bin/env bash |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
应该不需要写这个文件,可以统一用 tools/train.sh
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
好,我试一下
TODO