Streaming or Daemon Mode?

[chrisspen]

The accuracy of the "large" model is pretty good. On my corpus it gets about 80% of words accurately transcribed. The large Vosk model gets 84%, but Whisper gets some things correct that Vosk misses, so still not bad.

The big problem is that Whisper, even this C++ implementation, is still considerably slow. On average, Vosk can transcribe large audio files in around 2 seconds. With Whisper, it takes on average 2 minutes.

Since I'm running Whisper on the command line for each sample file, is this slowness mainly due to loading the large 3GB ggml-large.bin model file for every run?

If so, is there any way to mitigate this by launching Whisper in some sort of "streaming" mode, so it keeps the model loaded into memory, accepts filenames via stdin, and dumps transcriptions via stdout? I don't see any option like that in the command line docs.

[UniversalTechno]

yes it need load model once ; but it need reworking the program so we need to check first if there is any option for that before contributing .hope gerganove will reply and see what we can do

[ggerganov]

You can use the server example

[UniversalTechno]

Many thanks for your reply Sorry to not pay attention for the whole content Have a nice weekend!!!

…

On Fri, Jan 5, 2024, 15:06 Georgi Gerganov ***@***.***> wrote: You can use the server example — Reply to this email directly, view it on GitHub <#1428 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A5LAND36XI3M2OG74A562TLYNACFFAVCNFSM6AAAAAA65TVXT6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZYG4ZTAOBRG4> . You are receiving this because you commented.Message ID: ***@***.***>