Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ggml : sync (ggml-alloc, GPU, eps, etc.) #1220

Merged
merged 4 commits into from
Sep 5, 2023
Merged

ggml : sync (ggml-alloc, GPU, eps, etc.) #1220

merged 4 commits into from
Sep 5, 2023

Conversation

ggerganov
Copy link
Owner

@ggerganov ggerganov commented Aug 29, 2023

This syncs the latest ggml library and will be needed for several new improvements in whisper.cpp:

  • reduce memory usage via ggml-alloc
  • add full GPU offloading with CUDA and Metal

Need to check carefully if this accidentally overrides some of the recent commits in whisper.cpp because the sync is manual and I could have make a mistake. But overall, there should be no change in the behavior of whisper.cpp with these changes

@ggerganov
Copy link
Owner Author

Would appreciate some help with testing this branch and confirming that it does not change the behavior from master

@adam-the
Copy link

These changes seem to be fine when using normal models, but quantized models produce broken output. For example, the jfk speech outputs only exclamation marks from my testing. I haven't changed any parameters.

@ggerganov ggerganov merged commit 59a3d0c into master Sep 5, 2023
18 checks passed
@ggerganov ggerganov deleted the sync branch September 5, 2023 10:54
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
* ggml : sync (ggml-alloc, GPU, eps, etc.)

* ggml : fix build

* wasm : fix build
jacobwu-b pushed a commit to jacobwu-b/Transcriptify-by-whisper.cpp that referenced this pull request Oct 24, 2023
* ggml : sync (ggml-alloc, GPU, eps, etc.)

* ggml : fix build

* wasm : fix build
vonstring pushed a commit to vonstring/whisper.cpp that referenced this pull request Nov 7, 2023
* ggml : sync (ggml-alloc, GPU, eps, etc.)

* ggml : fix build

* wasm : fix build
landtanin pushed a commit to landtanin/whisper.cpp that referenced this pull request Dec 16, 2023
* ggml : sync (ggml-alloc, GPU, eps, etc.)

* ggml : fix build

* wasm : fix build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants