feat: add ExtraVolumes and ExtraVolumeMounts to all daemonset specs#2507
Open
75asu wants to merge 1 commit into
Open
feat: add ExtraVolumes and ExtraVolumeMounts to all daemonset specs#250775asu wants to merge 1 commit into
75asu wants to merge 1 commit into
Conversation
Signed-off-by: Asutosh Panda <asutosh.pda@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #1532
cc @guptaNswati -- you added the
help wanted+featurelabels and invited a PR. Here it is.Description
Adds
ExtraVolumes([]corev1.Volume) andExtraVolumeMounts([]corev1.VolumeMount) fields to every daemonset-managing component spec, so ClusterPolicy users can mount arbitrary host paths or other volume sources into the operator-managed daemonsets. The motivating case (issue #1532) is GKE, where NVIDIA driver libraries live at a non-standard path (/home/kubernetes/bin/nvidia/) and have to be exposed todcgm-exporterandgpu-feature-discoverypods that don't request a GPU resource.The PR is "full sweep" across all 16 daemonset specs in
ClusterPolicySpec-- not just the two mentioned in the issue body -- for consistency:DriverSpec,ToolkitSpec,DevicePluginSpec,SandboxDevicePluginSpec,KataDevicePluginSpec,DCGMExporterSpec,DCGMSpec,NodeStatusExporterSpec,GPUFeatureDiscoverySpec,MIGManagerSpec,KataManagerSpec,CCManagerSpec,VFIOManagerSpec,VGPUManagerSpec,VGPUDeviceManagerSpec,ValidatorSpec.How it works
A single helper
applyExtraVolumes(obj, vols, mounts)is added near the existingaddExtraAnnotationshelper. EachTransform*daemonset function gets one new line at the end, afterapplyHostNetworkConfig, calling the helper with the matchingconfig.<Component>.ExtraVolumes/ExtraVolumeMounts. Mounts are appended to every container in the pod spec; init containers are intentionally left untouched (they are operator-owned setup steps, not user-facing application containers).Two transforms share a parent spec by design:
TransformMPSControlDaemonreads fromDevicePluginSpec(same asTransformDevicePlugin)TransformSandboxValidatorreads fromValidatorSpec(same asTransformValidator)Both are commented inline to make the sharing explicit.
Example usage after this change
Note on diff size
The CRD yaml files (
config/crd/bases/,bundle/manifests/,deployments/gpu-operator/crds/) gain ~36k lines each. That is the expected expansion:corev1.Volumeis a tagged union over ~30 volume source types, and the OpenAPI schema for it inlines all of them. Multiplied across 16 specs and mirrored to three CRD locations, the line count grows fast. All three yaml files are output ofmake manifests+make sync-crds; only the four hand-written files matter for review:api/nvidia/v1/clusterpolicy_types.go-- the 32 field declarationscontrollers/object_controls.go-- the helper + 18 call sitescontrollers/transforms_test.go-- new test case forTransformDCGMExporterapi/nvidia/v1/zz_generated.deepcopy.go-- autogen from controller-genobjectChecklist
make lint) -- 0 issuesmake validate-generated-assets)make validate-modules) -- all modules verifiedTesting
transform_dcgm_exporter_with_extra_volumes_and_volume_mountsadded toTestTransformDCGMExporterexercising bothExtraVolumes(single hostPath volume) andExtraVolumeMounts(single mount). Verifies the volume is appended to the pod spec and the mount is appended to every container.controllerstest suite passes locally.