-
Notifications
You must be signed in to change notification settings - Fork 66
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
When Using a GPU: HappyTextGeneration.generate_text() Error After Training/Evaluating #220
Comments
The issue doesn't occur with the other HappyTransformers because they use pipelines. Since huggingface manages both the pipelines and the models, we can expect them to be in sync (i.e. both live on the same device). When we initialize our own tensors for use with HuggingFace objects, we must manually ensure that both objects live on the same device. |
You probably need to use |
Thanks Ted! I'll try that this weekend. |
See #223. Published in version 2.2.2 |
Reproduction:
Error message: RuntimeError: Input, output and indices must be on the current device
This problem is unique to HappyGeneration(). I have performed similar tests with HappyWordPrediction(), HappyQuestionAnswering() and HappyTextClassification() without finding any errors.
Quick fix: Run the following code in-between training/evaluating and using HappyTextGeneration.generate_text()
happy_gen.model.to("cpu")
The text was updated successfully, but these errors were encountered: