Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reuse Models on subsequent requests #231

Closed
mattkanwisher opened this issue May 11, 2023 · 2 comments
Closed

Reuse Models on subsequent requests #231

mattkanwisher opened this issue May 11, 2023 · 2 comments

Comments

@mattkanwisher
Copy link
Contributor

Maybe I'm reading code wrong, but it looks like it loads the Whisper model at least on every request. Seems like we should have some option of retaining models. I understand massive memory requirements so you may not want them all loaded forever.

@mudler
Copy link
Owner

mudler commented May 11, 2023

that's a fair point actually, we should treat whisper like the other models and keep that in memory

@mattkanwisher
Copy link
Contributor Author

fixed in #233

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants