Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gradients do not exist for variables #39

Open
CanshangD opened this issue Nov 8, 2019 · 5 comments
Open

Gradients do not exist for variables #39

CanshangD opened this issue Nov 8, 2019 · 5 comments

Comments

@CanshangD
Copy link

I have got the error “Gradients do not exist for variables” when training faster rcnn, and the loss is nan.

log:
W1108 09:59:39.735346 139647501481728 optimizer_v2.py:1029] Gradients do not exist for variables ['faster_rcnn/b_box_head/rcnn_class_conv1/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_conv1/bias:0', 'faster_rcnn/b_box_head/rcnn_class_bn1/gamma:0', 'faster_rcnn/b_box_head/rcnn_class_bn1/beta:0', 'faster_rcnn/b_box_head/rcnn_class_conv2/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_conv2/bias:0', 'faster_rcnn/b_box_head/rcnn_class_bn2/gamma:0', 'faster_rcnn/b_box_head/rcnn_class_bn2/beta:0', 'faster_rcnn/b_box_head/rcnn_class_logits/kernel:0', 'faster_rcnn/b_box_head/rcnn_class_logits/bias:0', 'faster_rcnn/b_box_head/rcnn_bbox_fc/kernel:0', 'faster_rcnn/b_box_head/rcnn_bbox_fc/bias:0'] when minimizing the loss.
epoch 0 2 nan

@MLForNerds
Copy link

Got the same issue, couldn't solve it. Kindly post the solution if you have solved it.

@IngLP
Copy link

IngLP commented Feb 24, 2020

I have the same issue and for me too the loss is NaN.
Please help

@xiaolng
Copy link

xiaolng commented Dec 23, 2020

same issue, loss becomes nan

@flori-shan
Copy link

same issue, please help

@jgarza9788
Copy link

can you please say what file you were running when you got this error?
what version(s) of pytorch and python etc ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants