Do you see any issues/benefits with caching different models in an LFU to speed up the loading?
#389
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Since loading a model takes about 10 seconds for me (depends on the GPU),
I decided to load 3 models in an LFU to bypass the initial loading.
I tested a couple and seems to reduce the initial loading time, and memory seems stable after couple transcribing too.
Just wondering if anyone has a better insight on this strategy or any issues this may cause in the long run?
Beta Was this translation helpful? Give feedback.
All reactions