Cannot enable GDRcopy using Nvidia driver CRD due to wrong indentation in 0500_daemonset.yaml

[age9990]

1. Quick Debug Information

• OS/Version(e.g. RHEL8.6, Ubuntu22.04): Ubuntu22.04
• Kernel Version: 6.2.0-39
• Container Runtime Type/Version(e.g. Containerd, CRI-O, Docker):CRI-O
• K8s Flavor/Version(e.g. K8s, OCP, Rancher, GKE, EKS):K8s
• GPU Operator Version:v24.3.0

2. Issue or feature description

When enable GDRcopy in nvidia driver CR, driver daemonset is not changed and error log showed in gpu operator pod.
{"level":"error","ts":"2024-05-03T06:29:33.398Z","msg":"Error while syncing state","controller":"nvidia-driver-controller","object":{"name":"default"},"namespace":"","name":"default","reconcileID":"a902d530-65d4-480e-8157-0e0c21d0a332","error":"failed to create k8s objects from manifests: failed to render kubernetes manifests: error rendering file /opt/gpu-operator/manifests/state-driver/0500_daemonset.yaml: failed to unmarshal manifest /opt/gpu-operator/manifests/state-driver/0500_daemonset.yaml: error converting YAML to JSON: yaml: line 195: did not find expected key"}

Looking into this file, the indentation is not correct, missing two spaces from L493 to L496.

gpu-operator/manifests/state-driver/0500_daemonset.yaml

Lines 478 to 498 in 0fe1e8d

	volumeMounts:
	- name: run-nvidia
	mountPath: /run/nvidia
	mountPropagation: HostToContainer
	- name: var-log
	mountPath: /var/log
	- name: dev-log
	mountPath: /dev/log
	readOnly: true
	{{- if and (.Openshift) (.Runtime.OpenshiftDriverToolkitEnabled) }}
	- name: shared-nvidia-driver-toolkit
	mountPath: /mnt/shared-nvidia-driver-toolkit
	{{- end}}
	{{- if and .AdditionalConfigs .AdditionalConfigs.VolumeMounts }}
	{{- range .AdditionalConfigs.VolumeMounts }}
	- name: {{ .Name }}
	mountPath: {{ .MountPath }}
	subPath: {{ .SubPath }}
	readOnly: {{ .ReadOnly }}
	{{- end }}
	{{- end }}

Once I fixed the indentation and rebuilt the image, the GDRcopy can be enabled with no error.

[cdesiniotis]

@age9990 thanks for reporting this issue.

[cdesiniotis]

I have a fix out for this here: https://gitlab.com/nvidia/kubernetes/gpu-operator/-/merge_requests/1083

[cdesiniotis]

GPU Operator 24.6.0 has been released and contains the fix for this issue.