CLI initialization takes 8 seconds for 3 second clip

[vinniyo]

Running 50 seconds of audio with a GPU certainly has it's benefits over CPU but running either with the command line seems to take about 5 seconds to start. Is there a way to start up whisper, load the model and have it ready to read the audio file?

[jongwook]

Because the command line needs to launch a fresh new process which will then read the model from the disk (it also does SHA256 hash check which may take a few seconds). Loading the model to the GPU should be faster but the CUDA runtime also takes some time to initialize. If you have multiple files, running whisper *.mp3 will load the model only once and run faster than running the commands for each file. (See #153)

In Python you can reuse the model returned by load_model() to avoid the startup delay, and to overkill, keeping an API server running like in #132 will allow you to send request from either the command line or Python without the loading delay.

[vinniyo]

I'll have to check out the API. Thank you!