llama.cpp on Windows with AMD ROCm (HIP) — Installation Guide

[jimw567]

llama.cpp is now built with ROCm 7.14 on Windows. If you are a long time llama.cpp user, you will find that some of DLLs (hipblas.dll, rocblas.dll, rocsolver.dll) are not bundled with llama.cpp anymore. This is due to the migration to multiarch ROCm package. Attached is an installation guide with instructions on where you can get those missing DLLs. Please let me know about any comments or questions.

llama.cpp on Windows with AMD ROCm (HIP) — Installation Guide

Target hardware: AMD Ryzen AI / Radeon iGPU or dGPU (gfx1150 / gfx1151 and compatible)
Target OS: Windows 11

---

Step 1: Install AMD Adrenalin Driver

ROCm 7.14 requires Adrenalin driver 26.6.4 or newer.

1. Download Adrenalin 26.6.4 from:
   https://www.amd.com/en/resources/support-articles/release-notes/RN-RAD-WIN-26-6-4.html#Downloads
2. Run the installer and follow the prompts
3. Reboot when complete

Verify the driver installed correctly:

Get-WmiObject Win32_VideoController | Where-Object { $_.Name -like "*AMD*" -or $_.Name -like "*Radeon*" } | Select-Object Name, DriverVersion, DriverDate

---

Step 2: Install ROCm (TheRock)

1. Create the installation directory and download the ROCm archive:

   mkdir C:\TheRock
   cd C:\TheRock
   curl -o therock-dist-windows-gfx1150-7.14.0.tar.gz https://repo.amd.com/rocm/tarball-multi-arch/therock-dist-windows-gfx1150-7.14.0.tar.gz

2. Extract the archive:

   mkdir build
   tar -xzf therock-dist-windows-gfx1150-7.14.0.tar.gz -C build --strip-components=1

3. Complete the post-installation environment setup by following the Post-installation section of AMD's official guide:
   https://rocm.docs.amd.com/en/latest/install/rocm.html?fam=ryzen&w=compute&os=windows&windows-ver=11&i=tar&gpu=ai-9-pro-465&gfx=gfx1150

---

Step 3: Download llama.cpp (ROCm 7.14 build)

1. Find the latest release tag and ROCm asset:

   $release = Invoke-RestMethod "https://api.github.com/repos/ggml-org/llama.cpp/releases/latest"
   $release.assets | Where-Object { $_.name -like "*win-rocm-7.14*" } | Select-Object name, browser_download_url

2. Download and extract (replace b10423 with the tag returned above):

   Invoke-WebRequest -Uri "https://github.com/ggml-org/llama.cpp/releases/download/b10423/llama-b10423-bin-win-rocm-7.14-x64.zip" -OutFile "llama-cpp-rocm.zip"
   Expand-Archive -Path "llama-cpp-rocm.zip" -DestinationPath "llama-cpp-rocm"

3. Confirm the ROCm backend is present:

   Get-ChildItem "llama-cpp-rocm" -Filter "ggml-hip.dll"

---

Step 4: Download a Model

Models must be in GGUF format. For a first test, a 4B Q4_K_M model is recommended — it fits comfortably in shared RAM and runs well on iGPUs.

Example — Qwen3.5-4B Q4_K_M (2.54 GB):

Invoke-WebRequest -Uri "https://huggingface.co/unsloth/Qwen3.5-4B-GGUF/resolve/main/Qwen3.5-4B-Q4_K_M.gguf" -OutFile "Qwen3.5-4B-Q4_K_M.gguf"

---

Step 5: Verify GPU Acceleration

Run the benchmark tool to confirm the ROCm backend loads and the GPU is being used:

.\llama-cpp-rocm\llama-bench.exe -m Qwen3.5-4B-Q4_K_M.gguf -ngl 99 -p 128 -n 128

Expected output:

ggml_cuda_init: found 1 ROCm devices:
  Device 0: AMD Radeon(TM) ... , gfx1150 ...
load_backend: loaded ROCm backend from ...\ggml-hip.dll
...
| model                   | backend | ngl |    test |          t/s |
| qwen35 4B Q4_K - Medium | ROCm    |  99 |   pp128 |    ~290 t/s  |
| qwen35 4B Q4_K - Medium | ROCm    |  99 |   tg128 |     ~10 t/s  |

If the backend column shows CPU instead of ROCm, the C:\TheRock\build\bin directory is not on PATH — repeat Step 2 item 3 and reopen PowerShell.

---

Step 6: Run Interactive Chat

.\llama-cpp-rocm\llama-cli.exe -m Qwen3.5-4B-Q4_K_M.gguf -ngl 99

-ngl 99 offloads all model layers to the GPU. Reduce this number if you run out of memory.

---

Troubleshooting

Symptom	Cause	Fix
Backend shows CPU, not ROCm	hipblas.dll not found	Ensure post-installation steps from AMD's guide were completed (Step 2 item 3) and reopen PowerShell
ggml-hip.dll not found on launch	Wrong zip downloaded	Ensure you downloaded the win-rocm-7.14-x64 zip, not the Vulkan zip
Out of memory / model fails to load	Not enough shared RAM allocated to GPU	Increase UMA frame buffer size in BIOS settings
Very slow performance	Model not fitting on GPU	Reduce -ngl value or use a smaller/more quantized model

llama-cpp-rocm-windows-install.md