main.exe silently crashes

[blazingzephyr]

I've tried using whisper.net and failed due to external errors (from whisper-cpp) occuring on model loading.
Then I've also tried using compiled binaries from v1.4.0 and then v1.2.0 and neither of them worked.
I've also tried several different models (mostly the base ones, including both ggml-model-whisper-base.en.bin and ggml-model-whisper-base.bin) with the sample wav and none of them work for me.

my prompt is
main.exe -m ggml-model-whisper-base.en.bin -f jfk.wav -l en -t 8

main.exe outputs the following:

whisper_init_from_file: loading model from 'ggml-model-whisper-base.en.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51864
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 512
whisper_model_load: n_audio_head  = 8
whisper_model_load: n_audio_layer = 6
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 512
whisper_model_load: n_text_head   = 8
whisper_model_load: n_text_layer  = 6
whisper_model_load: n_mels        = 80
whisper_model_load: f16           = 1
whisper_model_load: type          = 2
whisper_model_load: mem required  =  215.00 MB (+    6.00 MB per decoder)
whisper_model_load: kv self size  =    5.25 MB
whisper_model_load: kv cross size =   17.58 MB
whisper_model_load: adding 1607 extra tokens
whisper_model_load: model ctx     =  140.60 MB
whisper_model_load: model size    =  140.54 MB

After that the process just silently stops.

My computer specs are:

• CPU: Intel Xeon E5-2640
• GPU: NVIDIA GeForce GTX 1060 6GB
• OS: Win10 x64

Could you please tell me if it's a hardware issue or am I doing something wrong?

[blazingzephyr]

Okay, this is the line it silently stops at.

struct whisper_context * ctx = whisper_init_from_file(params.model.c_str());

Edit:
Okay, it just kills the process at ggml_init.

[blazingzephyr]

Nevermind, I guess it's a CPU problem, my CPU doesn't seem to support F16C and possible some other instructions, so I guess I can't fix it.

[blazingzephyr]

Tried to compile with CUDA support, hadn't helped either.

[ggerganov]

Try to build like this:

# build without F16C
mkdir build
cd build
cmake .. -DWHISPER_NO_F16C=ON
make -j

# run the model
./bin/main -m ../models/ggml-base.en.bin -f ../samples/jfk.wav

[blazingzephyr]

cmake .. -DWHISPER_NO_F16C=ON

There's no WHISPER_NO_F16C variable though.

Edit: At least according to the CMake warning

[blazingzephyr]

It seems like you can't compile with WHISPER_NO_F16C with MSVC so I'll try using another compiler tomorrow.

[blazingzephyr]

@ggerganov, can you please help me compile the source?
I've tried everything and I don't seem to understand how to turn WHISPER_NO_F16C flag on (If it's the only thing I need).
CMake doesn't see the flag

[ggerganov]

Try pulling latest master I think it should work now

[blazingzephyr]

@ggerganov, it didn't help. It still crashes even when I disable AVX, AVX2, F16C and FMA.
But I guess, in the end it's my CPU and there's nothing I can do to fix it except for getting a new one. Still thanks for your help though, it's really appreciated!

Edit: I'll try using whisper.cpp again when I get better hardware

[LazyDodo]

None of these CMAKE options actually do anything when using MSVC, as it'll just pass /arch:avx2 to the compiler regardless

change those /arch:avx2 to /arch:avx and you'll likely be ok on a E5-2640

[blazingzephyr]

None of these CMAKE options actually do anything when using MSVC, as it'll just pass /arch:avx2 to the compiler regardless

change those /arch:avx2 to /arch:avx and you'll likely be ok on a E5-2640

@LazyDodo Thank you! Your solution has helped me!
It's VERY slow with the tiny model used (164 seconds total), but it works! (I'll try using larger models tomorrow, even though it still takes a lot of time actually)
(Makes sense though, I have all instructions disabled other than AVX)

[blazingzephyr]

Never mind, it makes sense it's this slow as the hardware is quite old and I've disabled almost all of the instructions.

[blazingzephyr]

Alright, I will do more tests and provide more information tomorrow. Thank you for all the help!

[blazingzephyr]

@LazyDodo, @ggerganov Thank you for your help!
Not only did I manage to compile and run the model on my device, I have also managed to achieve very good performance by using the right compilation options! (disabling all the instructions but AVX, enabling BLAS and using several (4+) threads).

[AB0x]

Could you share how to make it run on older CPU i5 2400?