Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RuntimeError: copy_if failed to synchronize: device-side assert triggered #38

Open
Crystal950504 opened this issue Nov 6, 2019 · 4 comments

Comments

@Crystal950504
Copy link

File "train_refinedet.py", line 289, in
train()
File "train_refinedet.py", line 203, in train
odm_loss_l, odm_loss_c = odm_criterion(out, targets)
File "/home/shaoyidu/Program/anaconda3/envs/pytorch/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(*input, **kwargs)
File "/home/shaoyidu/Program/LH/RefineDet.PyTorch/layers/modules/refinedet_multibox_loss.py", line 127, in forward
loss_c[pos.view(-1,1)] = 0 # filter out pos boxes for now
RuntimeError: copy_if failed to synchronize: device-side assert triggered

@huangshenneng
Copy link

Excuse me, have you solved this problem?

@zeroorhero
Copy link

Excuse me, have you solved this problem?
hello,have you solved the problem?Thank you!

@mhyz1n4
Copy link

mhyz1n4 commented Feb 21, 2020

Excuse me, have you solved this problem?
hello,have you solved the problem?Thank you!

I've got this issue because the number of classes is defined incorrectly. Make sure to change number of calsses variables and categories variable if you are using a custom dataset. Adding CUDA_LAUNCH_BLOCKING=1 command also helps during debugging.

@whsun21
Copy link

whsun21 commented Dec 25, 2020

dont have this error when using cuda 10.1, but cuda10.0 will trigger this error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants