whisper: optimize fft() function #2242
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ggerganov
reviewed
Jun 18, 2024
| std::vector<float> fft_in(frame_size, 0.0); | ||
| std::vector<float> fft_out(2 * frame_size); | ||
| std::vector<float> fft_in(frame_size * 2, 0.0); | ||
| std::vector<float> fft_out(frame_size * 2 * 2 * 2); |
There was a problem hiding this comment.
Is this correct - shouldn't it be just frame_size * 2 * 2?
There was a problem hiding this comment.
Suppose frame_size is 8,
For fft_in, we have series recursively 8, 4, 2, 1, so 8+4+2+1=15, 8*2 is enough.
A little complicated for fft_out, the series are 8*2(out)+8(even_fft)+8(odd_fft), 8+4+4, 4+2+2, 2+1+1, then 32+16+8+4=60, 8*2*2*2 is enough.
ggerganov
approved these changes
Jun 18, 2024
bygreencn
added a commit
to bygreencn/whisper.cpp
that referenced
this pull request
Jun 27, 2024
* bobqianic/fix-decoding: (436 commits) Add files via upload Update whisper.cpp whisper : optimize fft() function (ggerganov#2242) talk-llama : sync llama.cpp whisper : use ggml_backend_sched (ggerganov#2239) fix : remove extra files scripts : sync ggml-blas build : update make / cmake sync : ggml move BLAS to a separate backend (cont) (llama/6210) Vulkan Shader Refactor, Memory Debugging Option (llama/7947) scripts : stop sync whisper example from ggml cmake : fix sycl build (#0) ggml : remove OpenCL (#0) sycl : sync (#0) cuda : enable CUDA graphs (#0) talk-llama : sync llama.cpp cmake : fix CUDA build (#0) sync : ggml ggml : fix and optimize ppc64le (ggml/849) ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In log_mel_spectrogram_worker_thread() function, we allocate enough memory for fft_in and fft_out variable, and then we can avoid allocating memory for even, odd, even_fft and odd_fft in fft().
In my testing, this optimization can reduce mel time from 125 ms to 100 ms for 5-minute audio.