Skip to content

Latest commit

 

History

History
110 lines (79 loc) · 4.16 KB

02-nixos-docker-nvidia-setup.org

File metadata and controls

110 lines (79 loc) · 4.16 KB

02. Setup NixOS to use Docker with NVIDIA Container toolkit support

This step is also fairly straightforward. During the time of this writing, NVIDIA container toolkit does not support v2 cgroups, so Nixos will have to be configured to use the previous version.

Use Docker with NVIDIA container toolkit support

Update the nvidia gpu settings to enable nvidia container toolkit.

nvidia_gpu.nix

{ config, lib, pkgs, ... }:

{

  services.xserver.videoDrivers = [ "nvidia" ];

  hardware.nvidia = {
    open = false;
    nvidiaSettings = true;
    modesetting.enable = true;

    package = config.boot.kernelPackages.nvidiaPackages.stable;
  };

  hardware.nvidia-container-toolkit = {
    enable = true;
  };

}

As previously mentioned, the configuration for docker is in a separate file as below. Enable it.

docker.nix

{ config, lib, pkgs, ... }:

{
  virtualisation.docker.enable = true;
  virtualisation.docker.enableOnBoot = true;
}

Include the gpu config in the main configuration

configuration.nix

{ config, lib, pkgs, ... }:

{
  imports = [
    ./hardware-configuration.nix

    .
    .
    # nvidia gpu
    ./nvidia_gpu.nix

    # virtualisation
    ./docker.nix

    .
    .

  ];

.
.
.

Rebuild the nixos configuration and run tests as mentioned below.

Test that the docker nvidia integration works

Having made a note of the CUDA version running on your NixOS host, get an image from docker hub with a matching CUDA version.

If there’s no exact match, you can try to get an image that has a slightly older minor version, but not the major one. For eg. if you have CUDA 12.7 on your NixOS host, getting an image with CUDA 12.6 should be compatible. Always match the CUDA major version, and get the least old minor version image.

Note that you will not be able to see all the pids as you could on the host, unless you use the –pid=host flag.

> docker run --rm --device nvidia.com/gpu=all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
Wed Mar 12 14:58:27 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 565.77                 Driver Version: 565.77         CUDA Version: 12.7     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 3090        Off |   00000000:01:00.0 Off |                  N/A |
|  0%   45C    P8             29W /  370W |      15MiB /  24576MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+
|   1  NVIDIA GeForce RTX 3090        Off |   00000000:02:00.0  On |                  N/A |
| 51%   48C    P8             39W /  370W |    1032MiB /  24576MiB |     18%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
+-----------------------------------------------------------------------------------------+