Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Other GPU ids throw error #12

Open
sandeepjangir07 opened this issue Jul 5, 2021 · 4 comments
Open

Other GPU ids throw error #12

sandeepjangir07 opened this issue Jul 5, 2021 · 4 comments

Comments

@sandeepjangir07
Copy link

When using any other GPU devices ID, except 0, the code throws error.
"
Traceback (most recent call last):
File "test.py", line 12, in
opt = TestOptions().parse()
File "/home/jang_sa/phd/AI/domain_adaptation/TSIT/options/base_options.py", line 178, in parse
torch.cuda.set_device(opt.gpu_ids[0])
File "/home/jang_sa/Software/anaconda3/envs/tsit/lib/python3.7/site-packages/torch/cuda/init.py", line 263, in set_device
torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
"
The GPUs are available and device IDs are valid but still error is got !! any solution for this problem ?

@EndlessSora
Copy link
Owner

It seems working on my side. How many GPUs do you have?

@sandeepjangir07
Copy link
Author

It seems working on my side. How many GPUs do you have?

I have two GPU clusters. One with 8 GPUs and one with 5 but whenever I use CUDA_VISIBLE_DEVICES=[anything other than 0] and gpu_id=(anything other than 0) I get this error !! I think today, I will try to sit and debug it but if you have any hint of whats causing this, i would be very helpful.

thanks

@EndlessSora
Copy link
Owner

For example, when you modify the --gpu_ids 0 here to --gpu_ids 1, will it cause an error?

@sandeepjangir07
Copy link
Author

For example, when you modify the --gpu_ids 0 here to --gpu_ids 1, will it cause an error?

Hi,
Yes, I cannot do inference on other GPUs as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants