Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: update whisper_cpp with CUBLAS, HIPBLAS, METAL, OPENBLAS, CLBLAST support #1302

Merged
merged 1 commit into from Dec 2, 2023
Merged

Conversation

wuxxin
Copy link
Contributor

@wuxxin wuxxin commented Nov 18, 2023

Description

This PR updates whisper_cpp to release 1.5 and adds GPU Support

i build and tested ROCM/HIPBLAS GPU support for whisper.
i did not test any other combination.
This is a draft. Please ignore if not fitting.

Notes for Reviewers

Signed commits

  • Yes, I signed my commits.

@wuxxin
Copy link
Contributor Author

wuxxin commented Nov 20, 2023

disabled WHISPER_METAL gpu acceleration, to make it work as before on metal, not causing regression. if build-apple works i would open a feature parity issue for whisper.cpp GPU on METAL support as reminder, but whisper.cpp would be up2date and gained GPU accel on CUDA, HIP

@wuxxin
Copy link
Contributor Author

wuxxin commented Nov 21, 2023

tests-apple fail (no matter if WHISPER_METAL=1 or not).

upstream ticket: ggerganov/whisper.cpp#1344
pr that works on fixing this: ggerganov/whisper.cpp#1530

ld: warning: -no_pie is deprecated when targeting new OS versions
Undefined symbols for architecture x86_64:
  "_MTLCopyAllDevices", referenced from:
      _ggml_metal_init in libwhisper.a(ggml-metal.o)
  "_MTLCreateSystemDefaultDevice", referenced from:
      _ggml_metal_init in libwhisper.a(ggml-metal.o)
  "_OBJC_CLASS_$_MTLComputePassDescriptor", referenced from:
      objc-class-ref in libwhisper.a(ggml-metal.o)
  "_OBJC_CLASS_$_NSBundle", referenced from:
      objc-class-ref in libwhisper.a(ggml-metal.o)
  "_OBJC_CLASS_$_NSProcessInfo", referenced from:
      objc-class-ref in libwhisper.a(ggml-metal.o)
  "_OBJC_CLASS_$_NSString", referenced from:
      objc-class-ref in libwhisper.a(ggml-metal.o)
ld: symbol(s) not found for architecture x86_64

@mudler
Copy link
Owner

mudler commented Nov 25, 2023

thanks @wuxxin - this is looking good from my side - let's wait for upstream and merge it once the CI is green

mudler
mudler previously approved these changes Nov 25, 2023
Copy link

netlify bot commented Nov 26, 2023

Deploy Preview for localai canceled.

Name Link
🔨 Latest commit 5a4ed4c
🔍 Latest deploy log https://app.netlify.com/sites/localai/deploys/656a852e698a090008aa4dde

@wuxxin
Copy link
Contributor Author

wuxxin commented Nov 26, 2023

pr ggerganov/whisper.cpp#1530 has been merged, commit hash updated to 1.15.1

@wuxxin
Copy link
Contributor Author

wuxxin commented Nov 28, 2023

@mudler pr ggerganov/whisper.cpp#1530 has been merged, commit hash updated to 1.15.1

@mudler mudler enabled auto-merge (squash) November 30, 2023 12:13
@mudler
Copy link
Owner

mudler commented Nov 30, 2023

mm seems it still fails @wuxxin

auto-merge was automatically disabled December 2, 2023 01:15

Head branch was pushed to by a user without write access

@mudler mudler enabled auto-merge (squash) December 2, 2023 07:39
@mudler mudler merged commit 3923024 into mudler:master Dec 2, 2023
18 checks passed
@wuxxin
Copy link
Contributor Author

wuxxin commented Dec 2, 2023

👍

@mudler mudler added the enhancement New feature or request label Dec 2, 2023
@mudler mudler changed the title update whisper_cpp with CUBLAS, HIPBLAS, METAL, OPENBLAS, CLBLAST support feat: update whisper_cpp with CUBLAS, HIPBLAS, METAL, OPENBLAS, CLBLAST support Dec 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants