When Using a GPU: HappyTextGeneration.generate_text() Error After Training/Evaluating

[EricFillion]

Reproduction:

• Connect to a Google Colab environment with a GPU
• pip install happytransformer==2.2.1
• Upload a text file called "text.txt"

from happytransformer import HappyGeneration

happy_gen = HappyGeneration()

happy_gen.train("test.txt")

output = happy_gen.generate_text("Artificial intelligence is ")`

Error message: RuntimeError: Input, output and indices must be on the current device

This problem is unique to HappyGeneration(). I have performed similar tests with HappyWordPrediction(), HappyQuestionAnswering() and HappyTextClassification() without finding any errors.

Quick fix: Run the following code in-between training/evaluating and using HappyTextGeneration.generate_text()
happy_gen.model.to("cpu")

[ted537]

The issue doesn't occur with the other HappyTransformers because they use pipelines. Since huggingface manages both the pipelines and the models, we can expect them to be in sync (i.e. both live on the same device). When we initialize our own tensors for use with HuggingFace objects, we must manually ensure that both objects live on the same device.

[ted537]

You probably need to use input_ids = input_ids.to(model._device)

[EricFillion]

Thanks Ted! I'll try that this weekend.

[EricFillion]

See #223. Published in version 2.2.2