feat(api): expose runtimeClassName on InferenceServiceSpec (closes #375)

[Defilan]

Closes #375.

Summary

Adds RuntimeClassName *string to InferenceServiceSpec. The deployment builder forwards it to PodSpec.RuntimeClassName. Most commonly set to "nvidia" on clusters where the NVIDIA Container Runtime is not configured as the cluster default.

Originally surfaced via a Discord question yesterday: a community member's GPU pods were scheduling onto the GPU node but never getting the device files bind-mounted. They needed to set runtimeClassName: nvidia on the Pod, and there was no way to plumb that through InferenceServiceSpec. The available workarounds were either reconfiguring containerd globally or running a Kyverno mutating webhook against the inference label selector. Both clunkier than just having the field on the CRD.

Changes

File	Change
api/v1alpha1/inferenceservice_types.go	Add RuntimeClassName *string \json:"runtimeClassName,omitempty"`` field with usage docs
api/v1alpha1/zz_generated.deepcopy.go	make generate
config/crd/bases/...	make manifests
charts/llmkube/templates/crds/inferenceservices.yaml	make chart-crds
internal/controller/deployment_builder.go	One-line RuntimeClassName: isvc.Spec.RuntimeClassName in PodSpec construction
internal/controller/inferenceservice_deployment_test.go	Two unit tests: set path asserts *PodSpec.RuntimeClassName == "nvidia", unset path asserts PodSpec.RuntimeClassName == nil

Net: +125 lines, no deletions. Field is optional and nil-safe; existing clusters see no behavior change.

Why now

Tagged for the 0.7.6 release window (the same window that closed #374 via PR #376). It's a small, isolated, user-visible feature that closes a real Discord-reported gap. Doesn't overlap with any other open PR (#340 lives in runtime_*.go; this lives in deployment_builder.go).

Out of scope (deferred to follow-ups if needed)

• Validating that the named RuntimeClass actually exists in the cluster — Kubernetes returns a clear error at pod admission, no need to duplicate
• Auto-detecting the cluster's GPU runtime configuration — users should know their cluster
• A runtimeClass example in the README — happy to add as a one-paragraph follow-up if the team wants it

Test plan

• make manifests generate fmt vet clean
• make chart-crds synced (no drift)
• go test ./internal/controller/... ./pkg/agent/... passes (new tests included)
• golangci-lint v2.4.0 reports 0 issues
• Manual verification on a kind cluster: deploy with and without runtimeClassName: nvidia, confirm kubectl get pod -o yaml shows the expected runtimeClassName value

Related

• Issue [FEATURE] Expose runtimeClassName on InferenceServiceSpec #375 (the request)
• PR fix: derive metal InferenceService phase from Endpoints, not desiredReplicas (closes #374) #376 (sibling RuntimeClass-adjacent fix on the metal-accelerator path; merged earlier today)
• Discord origin (the community member who reported it)

[codecov]

Codecov Report

❌ Patch coverage is 20.00000% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
api/v1alpha1/zz_generated.deepcopy.go	0.00%	4 Missing ⚠️

📢 Thoughts on this report? Let us know!