Check if model is already loaded

[GavasHaluk]

Is there a way to check if a model is already loaded and skip the load_model step? I have seen that the load_model is invoked multiple times between my endpoint calls which is not ideal since it adds quite a bit of execution time especially with using larger models.

[baibhavKumar1]

Yes just add a flag , when you add it for first time, and if it still exists don't load the model again. Check if this works, I'm not sure.

…

On Wed, May 3, 2023 at 12:49 PM GavasHaluk ***@***.***> wrote: Is there a way to check if a model is already loaded and skip the load_model step? I have seen that the load_model is invoked multiple times between my endpoint calls which is not ideal since it adds quite a bit of execution time especially with using larger models. — Reply to this email directly, view it on GitHub <#1306>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A2GQSJ4HGIYL6H2MVOQG7DLXEIBJDANCNFSM6AAAAAAXUAFG4Y> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[jongwook]

Something like functools.cache can be handy for this purpose.

[xavierskip]

how to use functools.cache in this case?
I try

import whisper
from functools import cache

@cache
def load_model(model):
    return whisper.load_model(model)

whisper_model = load_model(Model)

seems not work.