Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nan in training on Windows #260

Closed
wjmicheal opened this issue Mar 26, 2022 · 7 comments
Closed

nan in training on Windows #260

wjmicheal opened this issue Mar 26, 2022 · 7 comments

Comments

@wjmicheal
Copy link

wjmicheal commented Mar 26, 2022

nan
大佬,你好,我使用demo中的数据进行训练时,box obj cls theta 等损失显示为nan?不知是什么原因

@wjmicheal
Copy link
Author

wjmicheal commented Mar 26, 2022

以下是训练后runs/train/exp 文件夹下相关文件
文件名分别为:train_batch0-train_batch7,val_batch0_labels,val_batch0_pred,val_batch1_labels,val_batch1_pred.
train_batch0
train_batch1
train_batch2
train_batch3
train_batch4
train_batch5
train_batch6
train_batch7
val_batch0_labels
val_batch0_pred
val_batch1_labels
val_batch1_pred

@wjmicheal
Copy link
Author

这是hyp.ymal和opt.ymal
hyp.txt
opt.txt

@wjmicheal
Copy link
Author

研究了好几天,运行detect.py检测结果是正常的,但是train一直出现nan,期待您的回复

@wjmicheal wjmicheal changed the title 为什么使用demo中的数据进行训练时,box obj cls theta 等损失均显示为nan为nan? 为什么使用demo中的数据进行训练时,box obj cls theta 等损失均显示为nan? Mar 26, 2022
@wjmicheal
Copy link
Author

这是result.csv 和result.png两个文件
results.csv
results

@wjmicheal
Copy link
Author

另外,我的run/train/exp文件夹下未生成 labels.jpg labels_correlogram.jpg 这两个文件

@hukaixuan19970627
Copy link
Owner

你是不是改过cuda代码?
windows编译不成功多半是因为环境版本不匹配或者编码格式无法识别,逻辑代码不能修改。
可以参考#224

@hukaixuan19970627
Copy link
Owner

如果你是windows+CUDA11训练过程中出现nan的话可以参考下这个issue:ultralytics/yolov5#5815

@hukaixuan19970627 hukaixuan19970627 changed the title 为什么使用demo中的数据进行训练时,box obj cls theta 等损失均显示为nan? nan in training on Windows Apr 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants