Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use multi-gpu (Colab example) #75

Closed
QAQOAO opened this issue Mar 27, 2021 · 4 comments
Closed

Can't use multi-gpu (Colab example) #75

QAQOAO opened this issue Mar 27, 2021 · 4 comments

Comments

@QAQOAO
Copy link

QAQOAO commented Mar 27, 2021

Hi, Thanks for your great work.

I just tried to run through provided Colab example using multi-gpu (0,1,2,3).

By changing use_multi_gpu from False to True, I got AssertionError: Invalid device id.

Does that mean Colab refuses to offer 4 GPUs or there are just some bugs?

By the way, I want to make sure whether my problem can be solved by the Informer model.

Let's say my custom dataset has 1 column of date and N columns of features and 1 target column that is numerical, and the goal is to predict the value of the target column using other columns in the same row.

Does this kind of problem called ''Multivariate predict Univariate'' so I choose 'MS' in args.features?

Thanks again.

@cookieminions
Copy link
Collaborator

Hi,

  1. You need to check if the colab provide enough gpus and you can modify args.divices according to the devices you have. We have test multi-gpu running in our cluster and it works well.
  2. Yes, you can use 'MS' to solve your problem, or use 'M' and select the last column of the result as your prediction.

@QAQOAO
Copy link
Author

QAQOAO commented Mar 27, 2021

Thanks for the quick response.

Does that mean the number of GPUs in the Colab would differ for different users even when they run the same provided Colab example for the first time?

Actually, I wonder how can I "check" how many GPUs can I use in Colab, and if it says that I can only use 2, how can I do to increase that number.

I had successfully used four GPUs once a time, the training process was very fast. However, whenever I trained afterward, I could use only two or even receive an error like "Invalid device id".

@cookieminions
Copy link
Collaborator

It seems that you can only use one GPU on google colab and that is the limit from Google.

The device id is related to the devices you can see and you need to set --devices as the ids of GPU you can use. For example, if you have 3 GPU, the device ids of them will be 0,1,2. And if the GPU 0 is used by other program, you can set --devices '1,2' to use GPU 1 and 2.

If you still recieve an error, you can provide us with the specific usage of the GPUs and more detailed error information.

@zhouhaoyi
Copy link
Owner

Suppose there is no more discussion. I will close this issue now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants