development.md

Development

Install prerequisites:

• Go
• C/C++ Compiler e.g. Clang on macOS, TDM-GCC (Windows amd64) or llvm-mingw (Windows arm64), GCC/Clang on Linux.

Then build and run Ollama from the root directory of the repository:

go run . serve

Note

Ollama includes native code compiled with CGO. From time to time these data structures can change and CGO can get out of sync resulting in unexpected crashes. You can force a full build of the native code by running go clean -cache first.

macOS (Apple Silicon)

macOS Apple Silicon supports Metal which is built-in to the Ollama binary. No additional steps are required.

macOS (Intel)

Install prerequisites:

• CMake or brew install cmake

Then, configure and build the project:

cmake -B build
cmake --build build

Lastly, run Ollama:

go run . serve

Windows

Install prerequisites:

• CMake
• Visual Studio 2022 including the Native Desktop Workload
• (Optional) AMD GPU support
  • ROCm
  • Ninja
• (Optional) NVIDIA GPU support
  • CUDA SDK
• (Optional) VULKAN GPU support
  • VULKAN SDK - useful for AMD/Intel GPUs
• (Optional) MLX engine support
  • CUDA 13+ SDK
  • cuDNN 9+

Then, configure and build the project:

cmake -B build
cmake --build build --config Release

Building for Vulkan requires VULKAN_SDK environment variable:

PowerShell

$env:VULKAN_SDK="C:\VulkanSDK\<version>"

CMD

set VULKAN_SDK=C:\VulkanSDK\<version>

Important

Building for ROCm requires additional flags:

cmake -B build -G Ninja -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++
cmake --build build --config Release

Lastly, run Ollama:

go run . serve

Windows (ARM)

Windows ARM does not support additional acceleration libraries at this time. Do not use cmake, simply go run or go build.

Linux

Install prerequisites:

• CMake or sudo apt install cmake or sudo dnf install cmake
• (Optional) AMD GPU support
  • ROCm
• (Optional) NVIDIA GPU support
  • CUDA SDK
• (Optional) VULKAN GPU support
  • VULKAN SDK - useful for AMD/Intel GPUs
  • Or install via package manager: sudo apt install vulkan-sdk (Ubuntu/Debian) or sudo dnf install vulkan-sdk (Fedora/CentOS)
• (Optional) MLX engine support
  • CUDA 13+ SDK
  • cuDNN 9+
  • OpenBLAS/LAPACK: sudo apt install libopenblas-dev liblapack-dev liblapacke-dev (Ubuntu/Debian)

Important

Ensure prerequisites are in PATH before running CMake.

Then, configure and build the project:

cmake -B build
cmake --build build

Lastly, run Ollama:

go run . serve

MLX Engine (Optional)

The MLX engine enables running safetensor based models. It requires building the MLX and MLX-C shared libraries separately via CMake. On MacOS, MLX leverages the Metal library to run on the GPU, and on Windows and Linux, runs on NVIDIA GPUs via CUDA v13.

macOS (Apple Silicon)

Requires the Metal toolchain. Install Xcode first, then:

xcodebuild -downloadComponent MetalToolchain

Verify it's installed correctly (should print "no input files"):

xcrun metal

Then build:

cmake -B build --preset MLX
cmake --build build --preset MLX --parallel
cmake --install build --component MLX

Note

Without the Metal toolchain, cmake will silently complete with Metal disabled. Check the cmake output for Setting MLX_BUILD_METAL=OFF which indicates the toolchain is missing.

Windows / Linux (CUDA)

Requires CUDA 13+ and cuDNN 9+.

cmake -B build --preset "MLX CUDA 13"
cmake --build build --target mlx --target mlxc --config Release --parallel
cmake --install build --component MLX --strip

Local MLX source overrides

To build against a local checkout of MLX and/or MLX-C (useful for development), set environment variables before running CMake:

export OLLAMA_MLX_SOURCE=/path/to/mlx
export OLLAMA_MLX_C_SOURCE=/path/to/mlx-c

For example, using the helper scripts with local mlx and mlx-c repos:

OLLAMA_MLX_SOURCE=../mlx OLLAMA_MLX_C_SOURCE=../mlx-c ./scripts/build_linux.sh

OLLAMA_MLX_SOURCE=../mlx OLLAMA_MLX_C_SOURCE=../mlx-c ./scripts/build_darwin.sh

$env:OLLAMA_MLX_SOURCE="../mlx"
$env:OLLAMA_MLX_C_SOURCE="../mlx-c"
./scripts/build_darwin.ps1

Docker

docker build .

ROCm

docker build --build-arg FLAVOR=rocm .

Running tests

To run tests, use go test:

go test ./...

Library detection

Ollama looks for acceleration libraries in the following paths relative to the ollama executable:

• ./lib/ollama (Windows)
• ../lib/ollama (Linux)
• . (macOS)
• build/lib/ollama (for development)

If the libraries are not found, Ollama will not run with any acceleration libraries.