Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

关于refinedet-resnet #1

Open
pengfeidip opened this issue Mar 5, 2019 · 3 comments
Open

关于refinedet-resnet #1

pengfeidip opened this issue Mar 5, 2019 · 3 comments

Comments

@pengfeidip
Copy link

您好,我在实现resnet的refinedet(pytorch)。但是几次迭代后,ODM模块的loss出现了nan。我在vgg-refinedet上测试没有这种情况,且性能良好,因此想知道是不是基于resnet的refinedet有什么特殊的地方?(比如vgg中的conv4和conv5中的归一化操作)

iter 10/120000 || ARM_L Loss: 3.0050 ARM_C Loss: 13.3785 ODM_L Loss: 3.3522 ODM_C Loss: 4.3170 || 
iter 20/120000 || ARM_L Loss: 4.4867 ARM_C Loss: 14.0901 ODM_L Loss: 4.0826 ODM_C Loss: 3.5513 || 
iter 30/120000 || ARM_L Loss: 2.7802 ARM_C Loss: 0.3611 ODM_L Loss: 2.4115 ODM_C Loss: 3.3172 || 
iter 40/120000 || ARM_L Loss: 3.6324 ARM_C Loss: 0.8211 ODM_L Loss: 2.7962 ODM_C Loss: 4.5152 || 
iter 50/120000 || ARM_L Loss: 2.9621 ARM_C Loss: 2.7995 ODM_L Loss: nan ODM_C Loss: 12.1341 || 
iter 60/120000 || ARM_L Loss: 2.9899 ARM_C Loss: 2.7789 ODM_L Loss: nan ODM_C Loss: 12.0717 || 
@pengfeidip
Copy link
Author

@dd604 ,根据这个仓库的代码来看,好像refinedet-resnet 并没有什么特殊的地方....

@dd604
Copy link
Owner

dd604 commented Mar 15, 2019

你好,我这里对resnet101的处理是没有在conv4,5上增加其他的normalization,而是使用了batch norm,我看到官方的实现也是这样的。
我重构了一下工程,更新一下,目前我在320x320的上面是训练是没有问题,不过确实在后期偶尔会出现loss很大。在512x512上训练没有成功,后面是出现loss为nan的情况,我也在查找原因,如果你有发现也麻烦告诉我一声。

@dd604
Copy link
Owner

dd604 commented May 6, 2020

你好,训练coco loss为nan的情况已经解决了,希望对你能有所帮助。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants