addon: implement node addon call whisper through cpp

[chenqianhe]

I tried to call Whisper in the node environment through addon. I hope I can contribute it to examples for reference.

The speed difference between calling in node environment and directly using cpp for reasoning is very small.

cpp(./main -m ggml-large.bin -l zh -f samples_jfk.wav):

whisper_init_from_file: loading model from 'ggml-large.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1280
whisper_model_load: n_audio_head  = 20
whisper_model_load: n_audio_layer = 32
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1280
whisper_model_load: n_text_head   = 20
whisper_model_load: n_text_layer  = 32
whisper_model_load: n_mels        = 80
whisper_model_load: f16           = 1
whisper_model_load: type          = 5
whisper_model_load: mem required  = 4641.00 MB (+   71.00 MB per decoder)
whisper_model_load: kv self size  =   70.00 MB
whisper_model_load: kv cross size =  234.38 MB
whisper_model_load: adding 1608 extra tokens
whisper_model_load: model ctx     = 2950.97 MB
whisper_model_load: model size    = 2950.66 MB

system_info: n_threads = 4 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | 

main: processing 'samples_jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = zh, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]  And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.


whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:     load time =  1884.19 ms
whisper_print_timings:      mel time =    16.54 ms
whisper_print_timings:   sample time =    11.85 ms /    27 runs (    0.44 ms per run)
whisper_print_timings:   encode time =  6082.55 ms /     1 runs ( 6082.55 ms per run)
whisper_print_timings:   decode time =  1759.77 ms /    27 runs (   65.18 ms per run)
whisper_print_timings:    total time =  9840.59 ms

node(node index.js --language=zh --model=../../ggml-large.bin --fname_inp=../../samples_jfk.wav):

whisper_init_from_file: loading model from '../../ggml-large.bin'
whisper_model_load: loading model
whisper_model_load: n_vocab       = 51865
whisper_model_load: n_audio_ctx   = 1500
whisper_model_load: n_audio_state = 1280
whisper_model_load: n_audio_head  = 20
whisper_model_load: n_audio_layer = 32
whisper_model_load: n_text_ctx    = 448
whisper_model_load: n_text_state  = 1280
whisper_model_load: n_text_head   = 20
whisper_model_load: n_text_layer  = 32
whisper_model_load: n_mels        = 80
whisper_model_load: f16           = 1
whisper_model_load: type          = 5
whisper_model_load: mem required  = 4641.00 MB (+   71.00 MB per decoder)
whisper_model_load: kv self size  =   70.00 MB
whisper_model_load: kv cross size =  234.38 MB
whisper_model_load: adding 1608 extra tokens
whisper_model_load: model ctx     = 2950.97 MB
whisper_model_load: model size    = 2950.66 MB

system_info: n_threads = 4 / 8 | AVX = 0 | AVX2 = 0 | AVX512 = 0 | FMA = 0 | NEON = 1 | ARM_FMA = 1 | F16C = 0 | FP16_VA = 1 | WASM_SIMD = 0 | BLAS = 1 | SSE3 = 0 | VSX = 0 | 

run: processing '../../samples_jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = zh, task = transcribe, timestamps = 1 ...


[00:00:00.000 --> 00:00:11.000]  And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.

whisper_print_timings:     fallbacks =   0 p /   0 h
whisper_print_timings:     load time =  2592.75 ms
whisper_print_timings:      mel time =    19.89 ms
whisper_print_timings:   sample time =    11.70 ms /    27 runs (    0.43 ms per run)
whisper_print_timings:   encode time =  6233.41 ms /     1 runs ( 6233.41 ms per run)
whisper_print_timings:   decode time =  2352.20 ms /    27 runs (   87.12 ms per run)
whisper_print_timings:    total time = 11323.46 ms

[ggerganov]

Great addition!

Do you think you can add a CI workflow to .build/workflows for building the addon and maybe running it using one of the "test" models available in the repo?
It would be really helpful, because the examples are growing and it becomes hard to maintain everything manually. Having an automated CI would make things easier.

[chenqianhe]

Great addition!

Do you think you can add a CI workflow to .build/workflows for building the addon and maybe running it using one of the "test" models available in the repo? It would be really helpful, because the examples are growing and it becomes hard to maintain everything manually. Having an automated CI would make things easier.

As you said, CI is necessary. I will try to complete the corresponding CI in a new PR.

[chenqianhe]

Great addition!
Do you think you can add a CI workflow to .build/workflows for building the addon and maybe running it using one of the "test" models available in the repo? It would be really helpful, because the examples are growing and it becomes hard to maintain everything manually. Having an automated CI would make things easier.

As you said, CI is necessary. I will try to complete the corresponding CI in a new PR.

I have completed CI in my own repo. After this pr is merged, I will open a new pr to merge CI with final path configuration.

[dkryaklin]

@chenqianhe great addition to this library! While testing I run into the problem that after building the actual node addon it hardcoded the path to the dylib file for some reason. Could you please suggest any idea on how to solve it?

[chenqianhe]

@chenqianhe great addition to this library! While testing I run into the problem that after building the actual node addon it hardcoded the path to the dylib file for some reason. Could you please suggest any idea on how to solve it?

@dkryaklin I guess this may not be caused by my use of cmake-js compilation, because this has not happened before. This time I wrote a target cmakekist under the whole project, so maybe I need to check it overall.

[chenqianhe]

@chenqianhe great addition to this library! While testing I run into the problem that after building the actual node addon it hardcoded the path to the dylib file for some reason. Could you please suggest any idea on how to solve it?

@dkryaklin I guess this may not be caused by my use of cmake-js compilation, because this has not happened before. This time I wrote a target cmakekist under the whole project, so maybe I need to check it overall.

@dkryaklin I checked it. dylib file path is appointed by @rpath.

> otool -L ../../build/whisper-addon.node
../../build/whisper-addon.node:
        @rpath/whisper-addon.node (compatibility version 0.0.0, current version 0.0.0)
        @rpath/libwhisper.dylib (compatibility version 0.0.0, current version 0.0.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.36.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1319.0.0)

The @rpath in the compiled result is specified to th project Release path by default

> otool -l ../../build/whisper-addon.node

.......

Load command 15
          cmd LC_RPATH
      cmdsize 72
         path /Users/chenqianhe/CLionProjects/whisper.cpp/build/Release (offset 12)

Generally, when using this dylib in a new project, you need to configure Runpath Search Paths. Or use install_name_tool to modify @rpath.

But I'm not sure whether we can optimize this thing by configuring the compilation configuration. I'm not particularly familiar with these on the mac. Maybe we need more professional people to help us solve this problem.
I hope there are not too many mistakes in what I said.

[dkryaklin]

@chenqianhe thanks! I did some research and was able to solve my problem by changing CMakeLists.txt

[dbredvick]

@dkryaklin By chance, what did you have to change to fix the path? I'm having the same issue right now.