How to specify CPU or GPU?

[romanovzky]

I'd like to know how to move the model between CPU and GPU. Alternatively, I'd like to know if it's possible to specify the device before instantiating a model.

[alanakbik]

Hello @romanovzky the default behavior is that the model gets put on GPU if available and runs on CPU if there is no GPU.

This is done in the __init__.py, here:

device = None
if torch.cuda.is_available():
    device = torch.device('cuda:0')
else:
    device = torch.device('cpu')

The flair.device parameter gets called all over the code to move models and tensor to the device on which flair is run.

So if you would like to explicitly change this behavior, for instance to direct it to run on CPU even if you have a GPU available, you need to run this code before instantiating your model:

import flair, torch
flair.device = torch.device('cpu') 

This overwrites the flair.device with your desired destination.

Hope this clarifies!

[alanakbik]

Closing since question is answered (hopefully) - feel free to reopen if you have other questions.

[TDaudert]

Hi Alan,

I'm running the language model trainer on a server with 2 GPUs. Both GPUs are available. It works on GPU 0, however, when I try to change the device to GPU 1, I face the following error:

Traceback (most recent call last):
File "train_LM_flair.py", line 42, in
trainer.train('/home/tobdau/data/language_model1', learning_rate= 20, mini_batch_size=100, sequence_length=250, patience=20, max_epochs=20, anneal_factor=0.999)
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/trainers/language_model_trainer.py", line 336, in train
log.info(self.model.generate_text())
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/models/language_model.py", line 259, in generate_text
prediction, _, hidden = self.forward(input, hidden)
File "/home/tobdau/.local/lib/python3.6/site-packages/flair/models/language_model.py", line 71, in forward
encoded = self.encoder(input)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/modules/sparse.py", line 118, in forward
self.norm_type, self.scale_grad_by_freq, self.sparse)
File "/home/tobdau/.local/lib/python3.6/site-packages/torch/nn/functional.py", line 1454, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: arguments are located on different GPUs at /pytorch/aten/src/THC/generic/THCTensorIndex.cu:519

I use the following code:

flair.device = torch.device('cuda:1')

Best,
Tobias

[alanakbik]

Hello @TDaudert - thanks for spotting and reporting this. The error is likely here:

https://github.com/zalandoresearch/flair/blob/11850eeac9dcfaa89410f6748a9c71e56d331492/flair/models/language_model.py#L277-L278

i.e., we only check if cuda is available and if so then put on cuda:0. But in the rest of the code, we use the newer .to(device) method, like here:

https://github.com/zalandoresearch/flair/blob/11850eeac9dcfaa89410f6748a9c71e56d331492/flair/models/language_model.py#L128

I will put in a PR which should fix this error!

[alanakbik]

@TDaudert should now be fixed in master branch!