Fix GPU usage by removing device from Transformers class wrapper to use the device/device_map directly exposed by HF Transformers in kwargs

[cpcdoy]

The issue

Using the device parameter already provided by the Transformers wrapper doesn't work as intended. It first loads the model (by default on CPU) and then pushes the model to the device later. This is inefficient and hasn't been working at all on my end.

Example code snippet for the above:

from guidance import models

# This is confusing since the user can think this is given directly to the wrapped HF transformers class
gpt = models.Transformers('gpt2', device='cuda') 

The fix

Removing the device parameter altogether fixes this and removes the confusion of having a device parameter that isn't actually directly being used by HF Transformers but only by the wrapper itself.

Also, it doesn't affect anything else since there is a self.device = self.model_obj.device that gets the device from the model directly. Also, the tests seem to pass.

It is now only necessary to use device_map from HF Transformers directly through kwargs and the model is loaded correctly.

Example code snippet for the above:

from guidance import models

gpt = models.Transformers('gpt2', device_map='auto') # `auto` or any other device map

[slundberg]

Thanks @cpcdoy ! This is a great fix. Once the unit tests run I'll go ahead and merge.

[cpcdoy]

Fixes #536

[aadityabhatia]

I tried the following

mistral = models.TransformersChat("mistralai/Mistral-7B-Instruct-v0.2", device=0)
mistral = models.Transformers("mistralai/Mistral-7B-Instruct-v0.2", device=0)

Both of those statements resulted in:

TypeError: MistralForCausalLM.__init__() got an unexpected keyword argument 'device'

Is that expected? Running the latest version from this repo: pip install git+https://github.com/guidance-ai/guidance