Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

为什么验证比训练慢很多很多 #46

Closed
sakurasakura1996 opened this issue Oct 15, 2020 · 6 comments
Closed

为什么验证比训练慢很多很多 #46

sakurasakura1996 opened this issue Oct 15, 2020 · 6 comments

Comments

@sakurasakura1996
Copy link

你好,我在训练过程中,训练一轮很快,而每轮训练之后的validate很慢,验证集只有训练集的七分之一。我设置的参数如下:

train

TRAIN = {
"DATA_TYPE": 'Customer', #DATA_TYPE: VOC ,COCO or Customer
"TRAIN_IMG_SIZE": 512,
"AUGMENT": True,
"BATCH_SIZE": 8,
"MULTI_SCALE_TRAIN": True,
"IOU_THRESHOLD_LOSS": 0.5,
"YOLO_EPOCHS": 50,
"Mobilenet_YOLO_EPOCHS": 120,
"NUMBER_WORKERS": 4,
"MOMENTUM": 0.9,
"WEIGHT_DECAY": 0.0005,
"LR_INIT": 1e-4,
"LR_END": 1e-6,
"WARMUP_EPOCHS": 2 # or None
}

val

VAL = {
"TEST_IMG_SIZE": 512,
"BATCH_SIZE": 8,
"NUMBER_WORKERS": 4,
"CONF_THRESH": 0.005,
"NMS_THRESH": 0.5,
"MULTI_SCALE_VAL": True,
"FLIP_VAL": True,
"Visual": True
}
训练和验证时的batch-size一样大,为什么验证比训练要慢很多

@sakurasakura1996
Copy link
Author

我看到代码中如果训练数据是VOC格式和Customer格式的话,用的是Evaluator,COCO格式的话使用的是COCOAPIEvaluator,COCOAPIEvaluator中有使用到val['batch_size'],是我使用的是Customer格式,然后验证过程是一张一张图片进行计算的原因吗

@argusswift
Copy link
Owner

我看到代码中如果训练数据是VOC格式和Customer格式的话,用的是Evaluator,COCO格式的话使用的是COCOAPIEvaluator,COCOAPIEvaluator中有使用到val['batch_size'],是我使用的是Customer格式,然后验证过程是一张一张图片进行计算的原因吗

是开始验证的过程很慢吗,还是最后用best.pt验证的时候很慢

@sakurasakura1996
Copy link
Author

就是训练过程中,训练很快,但是有验证部分很慢。

@argusswift
Copy link
Owner

就是训练过程中,训练很快,但是有验证部分很慢。

刚开始的时候因为训练不完全,导致网络回归出很多冗余框,前向推理过程中NMS的速度很慢。后面的几轮稍微好一点,建议将开始验证epoch设置为30(CSPDarknet)或者是50(mobilenet).

@sakurasakura1996
Copy link
Author

就是训练过程中,训练很快,但是有验证部分很慢。

刚开始的时候因为训练不完全,导致网络回归出很多冗余框,前向推理过程中NMS的速度很慢。后面的几轮稍微好一点,建议将开始验证epoch设置为30(CSPDarknet)或者是50(mobilenet).

好的,十分感谢。

@abhigoku10
Copy link

@sakurasakura1996 @argusswift which variable sets the "It is recommended to set the initial verification epoch to 30 (CSPDarknet) or 50 (mobilenet)." ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants