Conversation
…delRef mutable (closes #301) The Deployment selector.matchLabels included inference.llmkube.dev/model: <modelRef-value>, which Kubernetes treats as immutable post-creation. Editing spec.modelRef on a running InferenceService accepted the CR change but every reconcile failed at the apiserver with "field is immutable", so the pod kept running the old model with no error surfaced to the user. Split labels into two sets: - deploymentSelectorLabels: just app + inference.llmkube.dev/service. These never change over the InferenceService's lifetime, so the selector is stable and the apiserver Update succeeds. - The full label set (including model) still ships on Deployment.ObjectMeta.Labels and Pod template labels so kubectl filtering with -l inference.llmkube.dev/model=<name> keeps working. When the user now edits modelRef, the selector is byte-identical, the controller updates the Pod template (new model label, new init container, new model path), and Kubernetes does a normal rolling update. Two regression tests: - selector must not contain inference.llmkube.dev/model - swapping modelRef on the same InferenceService must produce Deployments whose selectors are deeply equal (the exact apiserver immutability constraint) Signed-off-by: Christopher Maher <chris@mahercode.io>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #301.
Bug
Deployment.Spec.Selector.MatchLabelsincludedinference.llmkube.dev/model: <modelRef-value>. Kubernetes treats Deployment selectors as immutable after creation, so editingspec.modelRefon a running InferenceService:This blocked common workflows like swapping a quant level or rolling to a different upstream fork.
Fix
Split the label set:
deploymentSelectorLabels(new helper):app+inference.llmkube.dev/service. Both are derived fromisvc.Nameand never change over the InferenceService's lifetime, so the selector is byte-stable across reconciles and the apiserver `Update` succeeds.Deployment.ObjectMeta.Labelsand Pod template labels, so `kubectl get pods -l inference.llmkube.dev/model=` keeps working as a filter.When the user now edits
modelRef, the selector stays identical, the controller updates the Pod template (new model label, new init container args, new model path), and Kubernetes does a standard rolling update.Test plan