Skip to content

Allow MPS control daemonset to be explicitly disabled #1178

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Allow MPS control daemonset to be explicitly disabled #1178

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea0731b
Add support disable MPS
marinoborges Feb 25, 2025
5099d16
Clarify purpose of mps.enabled config
marinoborges Mar 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions deployments/helm/nvidia-device-plugin/templates/daemonset-device-plugin.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -135,8 +135,10 @@ spec:
name: nvidia-device-plugin-ctr
command: ["nvidia-device-plugin"]
env:
{{- if .Values.mps.enabled }}
- name: MPS_ROOT
value: {{ .Values.mps.root }}
{{- end }}
{{- if typeIs "string" .Values.migStrategy }}
- name: MIG_STRATEGY
value: {{ .Values.migStrategy }}
Expand Down Expand Up @@ -205,13 +207,15 @@ spec:
mountPath: /driver-root
readOnly: true
{{- end }}
{{- if .Values.mps.enabled }}
# The MPS /dev/shm is needed to allow for MPS daemon health-checking.
- name: mps-shm
mountPath: /dev/shm
- name: mps-root
mountPath: /mps
- name: cdi-root
mountPath: /var/run/cdi
{{- end }}
{{- if $options.hasConfigMap }}
- name: available-configs
mountPath: /available-configs
Expand All @@ -226,22 +230,24 @@ spec:
- name: device-plugin
hostPath:
path: /var/lib/kubelet/device-plugins
{{- if .Values.mps.enabled }}
- name: mps-root
hostPath:
path: {{ .Values.mps.root }}
type: DirectoryOrCreate
- name: mps-shm
hostPath:
path: {{ .Values.mps.root }}/shm
- name: cdi-root
hostPath:
path: /var/run/cdi
type: DirectoryOrCreate
{{- end }}
{{- if typeIs "string" .Values.nvidiaDriverRoot }}
- name: driver-root
hostPath:
path: {{ .Values.nvidiaDriverRoot }}
{{- end }}
- name: cdi-root
hostPath:
path: /var/run/cdi
type: DirectoryOrCreate
{{- if $options.hasConfigMap }}
- name: available-configs
configMap:
Expand Down
2 changes: 1 addition & 1 deletion deployments/helm/nvidia-device-plugin/templates/daemonset-mps-control-daemon.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- if .Values.devicePlugin.enabled }}
{{- if and .Values.devicePlugin.enabled .Values.mps.enabled }}
---
{{- $options := (include "nvidia-device-plugin.options" . | fromJson) }}
{{- $configMapName := (include "nvidia-device-plugin.configMapName" .) | trim }}
Expand Down
2 changes: 2 additions & 0 deletions deployments/helm/nvidia-device-plugin/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,8 @@ nfd:
- vendor

mps:
# Enable MPS DaemonSet (user still needs to activate MPS configuration by labeling a node)
enabled: true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a user, I think at a minimum there needs to be comments placed above this line to explain its use. If I see this in a values file, I may interpret mps.enabled: true to mean I, as a user, want MPS enabled right out of the gate. But that's not what this does. When set to false MPS will never work even if I later activate an MPS configuration by labeling a node.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a comment clarifying this

# root specifies the location where files and folders for managing MPS will
# be created. This includes a daemon-specific /dev/shm and pipe and log
# directories.
Expand Down