This guide explains how to run Ollama with GPU acceleration on Windows using Docker Desktop and WSL2.
- Windows 10/11
- WSL2 enabled
- Docker Desktop installed
- NVIDIA GPU + latest GPU driver
- (Optional) CUDA Toolkit installed on Windows
Open PowerShell as Administrator and run:
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
Restart your computer.
Step 2 – Install Ubuntu for WSL
powershell
Copy code
wsl --install -d Ubuntu-24.04
If Ubuntu is already installed, skip this step.
Step 3 – Configure Docker Desktop
Open Docker Desktop
Settings → General → enable Use WSL 2 instead of Hyper-V
Settings → Resources → WSL Integration → enable Ubuntu-24.04
Wait until Docker shows Engine running
Step 4 – Test GPU Support in Docker
powershell
Copy code
docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi
If GPU information prints correctly, continue to Step 6.
If it fails, complete Step 5.
Step 5 – Install NVIDIA Container Toolkit inside Ubuntu (only if Step 4 failed)
Open Ubuntu:
powershell
Copy code
wsl -d Ubuntu-24.04
Run inside Ubuntu:
bash
Copy code
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
Exit:
bash
Copy code
exit
Run the GPU test again (Step 4).
Step 6 – Start Ollama with GPU
powershell
Copy code
docker run -d --gpus all -p 11434:11434 --name ollama ollama/ollama
Check status:
powershell
Copy code
docker ps
Step 7 – Download a Model
powershell
Copy code
docker exec -it ollama ollama run llama3
This downloads the model and opens a chat session.
Useful Commands
Command Purpose
docker restart ollama Restart the container
docker logs ollama Show logs
docker exec -it ollama bash Open inside the container
docker rm -f ollama Remove Ollama
wsl --shutdown Restart WSL engine
Completion
After step 7, Ollama with GPU is successfully running on Windows through Docker and WSL2.
# Run Ollama with GPU on Windows (Docker + WSL2)
This guide explains how to run Ollama with GPU acceleration on Windows using Docker Desktop and WSL2.
---
## Requirements
- Windows 10/11
- WSL2 enabled
- Docker Desktop installed
- NVIDIA GPU + latest GPU driver
- (Optional) CUDA Toolkit installed on Windows
---
## Step 1 – Enable WSL2
Open PowerShell as Administrator and run:
```powershell
dism.exe /online /enable-feature /featurename:Microsoft-Windows-Subsystem-Linux /all /norestart
dism.exe /online /enable-feature /featurename:VirtualMachinePlatform /all /norestart
Restart your computer.
Step 2 – Install Ubuntu for WSL
powershell
Copy code
wsl --install -d Ubuntu-24.04
If Ubuntu is already installed, skip this step.
Step 3 – Configure Docker Desktop
Open Docker Desktop
Settings → General → enable Use WSL 2 instead of Hyper-V
Settings → Resources → WSL Integration → enable Ubuntu-24.04
Wait until Docker shows Engine running
Step 4 – Test GPU Support in Docker
powershell
Copy code
docker run --rm --gpus all nvidia/cuda:12.2.0-base-ubuntu22.04 nvidia-smi
If GPU information prints correctly, continue to Step 6.
If it fails, complete Step 5.
Step 5 – Install NVIDIA Container Toolkit inside Ubuntu (only if Step 4 failed)
Open Ubuntu:
powershell
Copy code
wsl -d Ubuntu-24.04
Run inside Ubuntu:
bash
Copy code
distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/$distribution/libnvidia-container.list \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
Exit:
bash
Copy code
exit
Run the GPU test again (Step 4).
Step 6 – Start Ollama with GPU
powershell
Copy code
docker run -d --gpus all -p 11434:11434 --name ollama ollama/ollama
Check status:
powershell
Copy code
docker ps
Step 7 – Download a Model
powershell
Copy code
docker exec -it ollama ollama run llama3
This downloads the model and opens a chat session.
Useful Commands
Command Purpose
docker restart ollama Restart the container
docker logs ollama Show logs
docker exec -it ollama bash Open inside the container
docker rm -f ollama Remove Ollama
wsl --shutdown Restart WSL engine
Completion
After step 7, Ollama with GPU is successfully running on Windows through Docker and WSL2.