Repository navigation
Optional idle unloading for larger speech models #1349
adamdanielczyk
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Could we revisit optional idle unloading for speech-recognition models, following #548 and #945?
I’m using FluidVoice 1.6.13 with Qwen3 ASR 1.7B for basic dictation, with no AI cleanup. FluidVoice was using roughly 3.2 GB of memory when checked on my Mac, although I haven’t isolated how much belongs to the speech model.
I understand the preference for keeping models loaded to minimize latency. My usage is occasional, though, and I’d happily accept a model-loading delay on the first dictation after a long idle period in exchange for releasing its memory.
The earlier discussion mentioned Parakeet’s roughly 500 MB footprint, but the tradeoff feels different with larger models such as Qwen3 ASR 1.7B. Handy’s configurable unloading option fits this usage pattern well.
Would you consider an opt-in timeout under advanced Voice Engine settings, with “Never” remaining the default? That would preserve the current experience while allowing users like me to choose lower idle memory use.
All reactions