whisper.cpp_macos_howto

How to install a whisper instance locally on an Apple silicon machine from scratch


1.) Install Xcode from the App Store:



Open Xcode app to agree to the license agreement. Install the MacOs version for your computer. You don't need the code completion addon.

Close Xcode afterwards.

Open Terminal



Install Xcode command line tools by typing following command:

xcode-select --install

Leave the terminal window open


2.) Download the latest version of the homebrew installer:

Klick here https://github.com/Homebrew/brew/releases/

Klick on the latest release to download it (Homebrew-4.4.32.pkg in this example)


Install the homebrew package you downloaded

switch to terminal and type:

echo 'eval $(/opt/homebrew/bin/brew shellenv)' >> $HOME/.zprofile (This would be different on intel macs)

Exit the shell:

exit

Close terminal


3.) Install all necessary things

Open Terminal again and enter the following command:

brew install git cmake wget ninja miniconda ffmpeg libomp llvm lld

If one of the programs that are being installed is asking for something - just always hit the Enter key or type yes or agree or whatever necessary for the default option

Find out which version of llvm you installed:

ls /opt/homebrew/Cellar/llvm

Add the variables for the compiler. Replace the Xes with your version number. Be aware of the apostrophes ( " ) and that they aren't set with the CC and CXX variables!

export CC=/opt/homebrew/Cellar/llvm/XX.X.X/bin/clang
export CXX=/opt/homebrew/Cellar/llvm/XX.X.X/bin/clang++
export LDFLAGS="-L/opt/homebrew/Cellar/llvm/XX.X.X/lib"
export CPPFLAGS="-I/opt/homebrew/Cellar/llvm/XX.X.X/include"
export SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk

It should look like this for the version 20.1.3 of llvm:
export CPPFLAGS="-I/opt/homebrew/Cellar/llvm/20.1.3/include"


4.) Activate Miniconda

Enter following command into terminal:

conda init --all

Close Terminal after the initialization


5.) Download whisper.cpp from github

Open Terminal again. You should see this: ( base ) as leading information of your shell. It means that you successfully installed miniconda.



Create a new folder for whisper:

mkdir Whisper

Enter the folder:

cd Whisper

Download the latest whisper release from github:

git clone https://github.com/ggml-org/whisper.cpp.git

Enter the new Directory:

cd whisper.cpp


5.) compile whisper

Type into terminal:

cmake -G Ninja -B build -DCMAKE_BUILD_TYPE=Release -DWHISPER_COREML=1 -DWHISPER_CCACHE=OFF

Compile:

cmake --build build -j --config Release

Ignore possible upcoming warnings


5.) Generate the libraries

Create Anaconda environment:

conda create -n py312-whisper python=3.12 -y

And activate it:

conda activate py312-whisper

Install python stuff:

pip install ane_transformers
pip install openai-whisper
pip install coremltools

Download the unmodified large model:

./models/download-ggml-model.sh large-v3

Enable coreml accelleration:

./models/generate-coreml-model.sh large-v3

Now test your local whisper instance:

./build/bin/whisper-cli -m models/ggml-large-v3.bin -f samples/jfk.wav -l auto

Quantize the model with the Q5 method to accellerate it further

./build/bin/quantize models/ggml-large-v3.bin models/ggml-large-v3-q5_0.bin q5_0

And run the test again:

./build/bin/whisper-cli -m models/ggml-large-v3-q5_0.bin -f samples/jfk.wav -l auto


Whisper is now up and running.