Use Reader's device by default

[julian-risch]

Proposed changes:

• init() method now runs initialize_device_settings(use_cuda=self.use_gpu) to get Reader's device
• Reader's device is now used as default device of eval(), eval_from_file(), and calibrate_confidence_scores() but can be overwritten when calling these methods
• doc strings are updated accordingly

Note that train() also calls initialize_device_settings(use_cuda=use_gpu,use_amp=use_amp) as before because train() can be initialized with other a different value for use_gpu than the Reader itself.

closes #1137

[tholor]

I think there's some overlap with #1182 where another util method get_device was introduced.
Can you please check if we can actually get rid of one of them?

[julian-risch]

I think there's some overlap with #1182 where another util method get_device was introduced.
Can you please check if we can actually get rid of one of them?

Well, the functionality of get_device is not really necessary from my point of you.
Instead of calling get_device, you could always call

self.device, _ = initialize_device_settings(use_cuda=self.use_gpu)

which internally calls:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

get_device is the same:

def get_device(use_gpu: bool = True) -> str:
    if use_gpu and torch.cuda.is_available():
        return "cuda"
    return "cpu"

What do you think?

[tholor]

Yep, seems like they are quite redundant. The only difference: one is returning a torch.device object, the other one a string.
Can you please check if the device object works also in the other locations in Haystack where get_device is currently used?
If yes we can remove get_device() :)

[julian-risch]

By using initialize_device_settings() instead of the get_device() method, the returned device is always a torch.device object.
When this device object is used to transfer models to the GPU (or CPU), the device could also be passed as a parameter of type string:

haystack/haystack/generator/transformers.py

Line 132 in afee4f3

self.model = RagTokenForGeneration.from_pretrained(model_name_or_path, revision=model_version).to(self.device)

"The torch.device argument in functions can generally be substituted with a string." https://pytorch.org/docs/stable/tensor_attributes.html#torch.torch.device

In rare cases, FARM checks the given device against strings "cpu" or "gpu" but it then extracts the type of the device as follows: gpu=device.type=="cuda"
https://github.com/deepset-ai/FARM/search?q=device.type

Let's use initialize_device_settings() whenever we need to know the device and let's use device.type if we need to check the device against a string.

[tholor]

Thanks for verifying! The changes look good, but can you please verify that the get_device() method is not used anywhere else and if that's the case delete it?

[julian-risch]

Thanks for verifying! The changes look good, but can you please verify that the get_device() method is not used anywhere else and if that's the case delete it?

It's not used. I deleted it now.