Transcription server and RAG indexing engine

Managing the VM container and Host CUDA drivers

(this is primarily a backup document describing how to fix my proxmox environment if needed)

This currently runs on a privileged LXC container. Privileged might not be required, but since I trust myself and this is purely privately hosted I’m ok with having this be slightly insecure. Generally, you need to install NVIDIA drivers with kernel modules on the host PVE, configure GPU passthrough to the LXC, then install the same driver without kernel modules on the client.

# Host

$ chmod +x NVIDIA-Linux-550.144.03.run

# Run this, basically say yes to everything
$ ./NVIDIA-Linux-550.144.03.run


# Then, check you nvidia settings
$ ls -l /dev/nvidia*
crw-rw-rw- 1 root root 195,   0 Feb 20 19:11 /dev/nvidia0         (1)
crw-rw-rw- 1 root root 195, 255 Feb 20 19:11 /dev/nvidiactl
crw-rw-rw- 1 root root 511,   0 Feb 20 19:11 /dev/nvidia-uvm      (2)
crw-rw-rw- 1 root root 511,   1 Feb 20 19:11 /dev/nvidia-uvm-tools

/dev/nvidia-caps:
total 0
cr-------- 1 root root 236, 1 Feb 20 19:11 nvidia-cap1
cr--r--r-- 1 root root 236, 2 Feb 20 19:11 nvidia-cap2

1. Check this ID and ensure it’s matched with the LXC conf

2. Check this ID and ensure it’s matched with the LXC conf

In your LXC conf, you need to include the following lines

lxc.cgroup2.devices.allow: c 195:* rwm (1)
lxc.cgroup2.devices.allow: c 511:* rwm (2)
lxc.mount.entry: /dev/nvidia0 dev/nvidia0 none bind,optional,create=file
lxc.mount.entry: /dev/nvidiactl dev/nvidiactl none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm dev/nvidia-uvm none bind,optional,create=file
lxc.mount.entry: /dev/nvidia-uvm-tools dev/nvidia-uvm-tools none bind,optional,create=file

Finally, run the NV driver installer on the container as follows:

$ ./NVIDIA-Linux-550.144.03.run --no-kernel-module

And go through saying yes.