Update Memory Hotplug hostvolume to /sys/devices/system

[JunAr7112]

Description

This PR is created in response to this bug. Essentially, on some systems the /sys/devices/system/memory/auto_online_blocks files is not present and cannot be mounted as a hostvolume for systems that don't have CONFIG_MEMORY_HOTPLUG=y. auto_online_blocks is a Linux sysfs knob for memory hotplug.

Solution:

In manifests/state-driver/0500_daemonset.yaml and assets/state-driver/0500_daemonset.yaml switch to using a wider mountpath on /sys/devices/system rather than directly mounting /sys/devices/system/memory/auto_online_blocks.

Checklist

• [ x] No secrets, sensitive information, or unrelated changes
• [ x] Lint checks passing (make lint)
• [ x] Generated assets in-sync (make validate-generated-assets)
• [ x] Go mod artifacts in-sync (make validate-modules)
• [ x] Test cases are added for new code paths

Testing

Added TestDriverSysfsMemoryOnlineVolumeUsesStableParentDirectory, which verifies the rendered driver DaemonSet has the /sys/devices/system mount. It finds the volume named sysfs-memory-online and checks:

HostPath.Path == "/sys/devices/system"
HostPath.Type == corev1.HostPathDirectory
Finds the nvidia-driver-ctr container.

Finds that container’s sysfs-memory-online volume mount and checks:

MountPath == "/sys/devices/system"
SubPath == ""
So the test protects the exact behavior we want: the operator should mount the stable parent directory, not the optional /sys/devices/system/memory/auto_online_blocks file.

[tariq1890]

@JunAr7112 Have you tested this on a system where CONFIG_MEMORY_HOTPLUG is unset?

[JunAr7112]

@JunAr7112 Have you tested this on a system where CONFIG_MEMORY_HOTPLUG is unset?

Yes I setup a StarlingX system without CONFIG_MEMORY_HOTPLUG=y and verified that it could create the mount on the pod without any errors:

sysadmin@localhost:~$ grep CONFIG_MEMORY_HOTPLUG /boot/config-$(uname -r)
grep: /boot/config-6.12.0-1-amd64: No such file or directory

sysadmin@localhost:~ $ POD=$(kubectl get pod -n gpu-operator -l app=nvidia-driver-daemonset -o jsonpath='{.items[0].metadata.name}')
sysadmin@localhost:~ $ kubectl describe pod -n gpu-operator "$POD"
| awk '/^ sysfs-memory-online:/,/^ nv-firmware:/'
sysfs-memory-online:
Type: HostPath (bare host directory volume)
Path: /sys/devices/system
HostPathType: Directory

sysadmin@localhost:~$ kubectl describe pod -n gpu-operator "$POD"
| awk '/^Events:/,0'
| grep -nE 'Created container: nvidia-driver-ctr|Started container nvidia-driver-ctr|failed to mkdir|CreateContainerError|auto_online_blocks'
9: Normal Created 79s (x6 over 5m57s) kubelet Created container: nvidia-driver-ctr
10: Normal Started 79s (x6 over 5m57s) kubelet Started container nvidia-driver-ctr

[tariq1890]

Can you retitle this PR according to the latest changes you've made in this commit?

[JunAr7112]

Can you retitle this PR according to the latest changes you've made in this commit?

Updated to match the commit

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[JunAr7112]

/ok-to-test f048f1e