Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Stream Generator on MacOS #3792

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

gravityrail
Copy link

@gravityrail gravityrail commented Jun 14, 2024

Quick fix for streaming responses on macOS. It was crashing somewhere in pytorch. Unfortunately I can't find the original post where I found this patch to give credit - too many tabs open! But it works for me.

For this to work, you must run with PYTORCH_ENABLE_MPS_FALLBACK=1 and call model.to(mps_device) on the XttsModel

@CLAassistant
Copy link

CLAassistant commented Jun 14, 2024

CLA assistant check
All committers have signed the CLA.

@eginhard
Copy link
Contributor

The crash occurs on any OS with transformers>=4.41. This fix was originally shared by @pseudotensor in idiap#31. But I just opened idiap#46 in our fork (since this repo is not updated anymore) which more generally aligns the streaming code to the upstream transformers implementation again, so it should be more resilient to future updates. It will be available in the next release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants