Whisper model download seems failed and No Text Found

[botbahlul]

Hi, I just tried your wonderful app, especially on Audio To Text features.
The Vosk part is fine, but the Whisper part seems failed when I tried to download the model. It seems that this app tried to download from wrong URL.
I tried to download the model manually from https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin
then rename it to small.bin. but when I tried to generate the subtitle it said "No text found".

[niksedk]

The Whisper download urls were changed - latest beta updated: https://github.com/SubtitleEdit/subtitleedit/releases/download/3.6.13/SubtitleEditBeta.zip

[botbahlul]

still got the same "NO TEXT FOUND" when using CPP engine

is there any limitation or minimum duration for video or is it the whisper.cpp engine problem?

I tried on this 22 seconds duration mp4 video with english voice

sum.mp4

runs fine when tried with OpenAI engine (whisper.exe)

[niksedk]

It could be your CPU - Whisper CPP does not support all CPUs.

Do check the file "error_log.txt" in the SE data folder (press Ctrl+Alt-Shift+D in the main SE window to open this folder)

is there any limitation or minimum duration for video?

No

[botbahlul]

no text found with this log

---

Date: 07/08/2023 18:06:39
SE: 3.6.13.140 - Microsoft Windows NT 10.0.19045.0 - 64-bit
Message: C:\Program Files\SubtitleEditBeta\Whisper\main.exe --language en --model "C:\Program Files\SubtitleEditBeta\Whisper\Models\small.bin" --output-srt --print-progress "C:\Users\Toshiba\AppData\Local\Temp\b6943261-42fb-4116-898c-c8c1bb07f4d7.wav"

---

Date: 07/08/2023 18:06:43
SE: 3.6.13.140 - Microsoft Windows NT 10.0.19045.0 - 64-bit
Message: Calling whisper (CPP) with : C:\Program Files\SubtitleEditBeta\Whisper\main.exe --language en --model "C:\Program Files\SubtitleEditBeta\Whisper\Models\small.bin" --output-srt --print-progress "C:\Users\Toshiba\AppData\Local\Temp\b6943261-42fb-4116-898c-c8c1bb07f4d7.wav"

whisper_model_load: mem required = 743.00 MB (+ 16.00 MB per decoder)

whisper_model_load: type = 3

whisper_model_load: ftype = 1

whisper_model_load: n_mels = 80

whisper_model_load: n_text_layer = 12

whisper_model_load: n_text_head = 12

whisper_model_load: n_text_state = 768

whisper_model_load: n_text_ctx = 448

whisper_model_load: n_audio_layer = 12

whisper_model_load: n_audio_head = 12

whisper_model_load: n_audio_state = 768

whisper_model_load: n_audio_ctx = 1500

whisper_model_load: n_vocab = 51865

whisper_model_load: loading model

whisper_init_from_file_no_state: loading model from 'C:\Program Files\SubtitleEditBeta\Whisper\Models\small.bin'
whisper_model_load: model ctx = 464.56 MB

whisper_model_load: adding 1608 extra tokens
whisper_init_state: kv cross size = 52.73 MB

whisper_init_state: kv self size = 15.75 MB

whisper_model_load: model size = 464.44 MB
Calling whisper CPP done in 00:00:02.8220264
Loading result from STDOUT

[botbahlul]

I was also tried the whisper.cpp command line version from https://github.com/regstuff/whisper.cpp_windows and got similar problem (no srt file output)

[botbahlul]

if it's CPU support problem, why does it work with OpenAI engine?

[botbahlul]

I think the problem is from the "main.exe"

Are you using your own build of whisper.cpp or from https://github.com/regstuff/whisper.cpp_windows ?

I tried 2 16000 Hz Mono WAV files and NO OUTPUT AT ALL when run it in command line

same results with the output of "main.exe" of https://github.com/regstuff/whisper.cpp_windows

Can you give me a sample of a WAV file that successfully recognized by this "main.exe"?

"whisper.exe" can recognize them successfully

[niksedk]

if it's CPU support problem, why does it work with OpenAI engine?

OpenAI whisper and Whisper CPP completely different.
CPU support is normally determined in compilations options.

[botbahlul]

if it's CPU support problem, why does it work with OpenAI engine?

OpenAI whisper and Whisper CPP completely different. CPU support is normally determined in compilations options.

can you give me a sample of WAV File that successfully recognized by "main.exe"?

I'm very curious with this "main.exe" because I've read that it's FASTER but until today I've never seen the proof because I failed to build it my self.
I'm to lazy to edit the source code because cpp is really a hard work 😂

[niksedk]

Sure. Here's a .wav sample file:
small-sample.zip

Should give something like this with tiny English:

1
00:00:00,000 --> 00:00:02,795
An attempt was made by
an American whistleblower to

2
00:00:02,807 --> 00:00:06,000
sabotage our operation and
expose you all to great danger.

3
00:00:06,001 --> 00:00:08,667
I'm happy to say that
three of our operatives

4
00:00:08,679 --> 00:00:11,000
worked together in
London and Barcelona

5
00:00:11,001 --> 00:00:13,161
to eliminate the threat
and retrieve the intelligence.

[botbahlul]

Sure. Here's a .wav sample file: small-sample.zip

Should give something like this with tiny English:

1
00:00:00,000 --> 00:00:02,795
An attempt was made by
an American whistleblower to

2
00:00:02,807 --> 00:00:06,000
sabotage our operation and
expose you all to great danger.

3
00:00:06,001 --> 00:00:08,667
I'm happy to say that
three of our operatives

4
00:00:08,679 --> 00:00:11,000
worked together in
London and Barcelona

5
00:00:11,001 --> 00:00:13,161
to eliminate the threat
and retrieve the intelligence.

why do I still get NO OUTPUT AT ALL?
I tested main.exe from your latest release and from https://github.com/ggerganov/whisper.cpp/releases/download/v1.4.0/whisper-bin-x64.zip with same no output results.

I'm using windows 10

what's wrong?

[botbahlul]

I just tried "main.exe" from this : https://github.com/Const-me/Whisper and IT HAS OUTPUT (I BUILD IT MY SELF WITH VISUAL STUDIO 2022 COMUNITY EDITION, NO OUTPUT WHEN I TRIED THE BUILD FROM ITS RELEASES PAGE)

C:\Users\Toshiba\Documents\GitHub\Whisper\x64\Debug>main -t 4 -m small.bin -l en -f sum.wav
Using GPU "AMD Radeon HD 7570M", feature level 11.0, effective flags Wave64 | UseReshapedMatMul
Loaded MEL filters, 62.8 kb RAM
Loaded vocabulary, 51865 strings, 3037.1 kb RAM
Loaded 479 GPU tensors, 464.445 MB VRAM
Computed CPU base frequency: 2.69389 GHz
Loaded model from "small.bin" to VRAM
Created source reader from the file "sum.wav"

[00:00:00.000 --> 00:00:03.000]   He never told me you were friends.
[00:00:03.000 --> 00:00:06.000]   Perhaps he didn't know you well enough.
[00:00:06.000 --> 00:00:11.000]   Cabot did tell me he had a source in Russia.
[00:00:11.000 --> 00:00:16.000]   To keep the back channels open?
[00:00:16.000 --> 00:00:20.000]   In hopes of staving off disaster.
[00:00:20.000 --> 00:00:22.000]   President Kennedy said...
    CPU Tasks
LoadModel       1.54761 seconds
RunComplete     59.8767 seconds
Run     59.7388 seconds
Callbacks       4.9925 milliseconds, 8 calls, 624.062 microseconds average
Spectrogram     1.88861 seconds, 5 calls, 377.722 milliseconds average
Sample  651.92 milliseconds, 61 calls, 10.6872 milliseconds average
Encode  51.4961 seconds, 2 calls, 25.748 seconds average
Decode  8.23404 seconds, 2 calls, 4.11702 seconds average
DecodeStep      7.58138 seconds, 61 calls, 124.285 milliseconds average
    GPU Tasks
LoadModel       400.852 milliseconds
Run     57.4399 seconds
Encode  50.9867 seconds, 2 calls, 25.4933 seconds average
EncodeLayer     39.5927 seconds, 24 calls, 1.6497 seconds average
Decode  6.45319 seconds, 2 calls, 3.22659 seconds average
DecodeStep      6.45319 seconds, 61 calls, 105.79 milliseconds average
DecodeLayer     4.84453 seconds, 732 calls, 6.61821 milliseconds average
    Compute Shaders
mulMatTiledEx   29.3794 seconds, 240 calls, 122.414 milliseconds average
mulMatTiled     6.31355 seconds, 290 calls, 21.7709 milliseconds average
mulMatByRowTiled        2.39239 seconds, 7139 calls, 335.116 microseconds average
addRepeatGelu   2.26745 seconds, 760 calls, 2.98349 milliseconds average
convolutionMain 2.18608 seconds, 2 calls, 1.09304 seconds average
convolutionMain2Fixed   1.80022 seconds, 2 calls, 900.11 milliseconds average
matReshapePanels        1.31724 seconds, 146 calls, 9.02222 milliseconds average
fmaRepeat1      1.21042 seconds, 2307 calls, 524.671 microseconds average
copyTranspose   1.20915 seconds, 1512 calls, 799.701 microseconds average
softMaxFixed    1.18434 seconds, 756 calls, 1.56659 milliseconds average
addRepeat       1.14237 seconds, 804 calls, 1.42086 milliseconds average
scaleInPlace    1.08718 seconds, 756 calls, 1.43807 milliseconds average
softMaxLong     821.583 milliseconds, 61 calls, 13.4686 milliseconds average
mulMatByRowTiledEx      711.794 milliseconds, 1416 calls, 502.679 microseconds average
norm    511.68 milliseconds, 2307 calls, 221.795 microseconds average
copyConvert     381.027 milliseconds, 1560 calls, 244.248 microseconds average
addRepeatEx     180.662 milliseconds, 2244 calls, 80.5091 microseconds average
softMax 95.8522 milliseconds, 732 calls, 130.946 microseconds average
addRepeatScale  75.6341 milliseconds, 1464 calls, 51.6626 microseconds average
convolutionPrep1        62.686 milliseconds, 4 calls, 15.6715 milliseconds average
convolutionPrep2        38.2248 milliseconds, 4 calls, 9.5562 milliseconds average
diagMaskInf     29.3873 milliseconds, 732 calls, 40.1466 microseconds average
add     2.8221 milliseconds, 2 calls, 1.41105 milliseconds average
addRows 737.2 microseconds, 61 calls, 12.0852 microseconds average
    Memory Usage
Model   859.685 KB RAM, 464.445 MB VRAM
Context 23.7503 MB RAM, 412.894 MB VRAM
Total   24.5898 MB RAM, 877.338 MB VRAM

When I tried "main.exe" from this : https://github.com/ggerganov/whisper.cpp IT HAS NO OUTPUT, BOTH on my self build and on its releases page.

I hope you can find the bug, I love this CPP because it's really FASTER!

Wait...
"main.exe" from https://github.com/Const-me/Whisper now is very slow and just hung out .. what's wrong?

C:\Users\Toshiba\Documents\GitHub\Whisper\x64\Debug>main -t 4 -p 4 -m small.bin -l en -f sum.wav
Using GPU "AMD Radeon HD 7570M", feature level 11.0, effective flags Wave64 | UseReshapedMatMul
Loaded MEL filters, 62.8 kb RAM
Loaded vocabulary, 51865 strings, 3037.1 kb RAM

[botbahlul]

It looks like these ".bin" models wont work smoothly on AMD GPU devices.
Sometimes (very rarely) it can be loaded well, but many times it won't.