Predicted boxes or scores contain Inf/NaN. Training has diverged

[henanwg]

When trying to run the training code, I encountered the following error:

In the previous process, my steps were:

• Refer to the readme, install torch, detectron and other environment dependencies based on conda

• Data preparation: vocdata in readme is used
  

• Modify operating errors：
  1、in the run_*.sh ,change into
  
  2、as the picture shows（Refer to：https://giters.com/fanq15/FewX/issues/40?amp=1）
  

• Intercept part of the training code and try to start training
  

then i meet the error：Predicted boxes or scores contain Inf/NaN. Training has diverged。I got the same error when I tried on two different servers (cuda10.2, cuda11.1 both torch1.8)

I tried to modify the learning rate-related settings（to different value , vary from 0.00005 to 0.95） in the configuration file, but no effect

[henanwg]

It may be problem of detectron version, I will try detectron0.3 next

[henanwg]

problem solved

[00why00]

problem solved

I am facing the same problem as you. Can you tell me how you solved it?

[henanwg]

using detectron0.3，the default version is 0.5

[all-for-code]

using detectron0.3，the default version is 0.5

How to install detectron0.3 when i use the server with cuda11.1

[Ellohiye]

使用Detectron0.3，默认版本为0.5

当我使用带cuda11.1的服务器时如何安装Detectron0.3
我也遇到这个问题！请问您是怎么解决的呢