Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating Karpenter v1beta1 Spec #318

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

logyball
Copy link

I started receiving error messages from kubeconform when using the instanceProfile field of Karpenter rather than the the role for the EC2NodeClass resource. When looking up the actual CRD definition, this should not be happening:

Role Definition:

role:
                description: |-
                  Role is the AWS identity that nodes use. This field is immutable.
                  This field is mutually exclusive from instanceProfile.
                  Marking this field as immutable avoids concerns around terminating managed instance profiles from running instances.
                  This field may be made mutable in the future, assuming the correct garbage collection and drift handling is implemented
                  for the old instance profiles on an update.
                type: string
                x-kubernetes-validations:
                - message: role cannot be empty
                  rule: self != ''
                - message: immutable field changed
                  rule: self == oldSelf

InstanceProfile definition:

instanceProfile:
                description: |-
                  InstanceProfile is the AWS entity that instances use.
                  This field is mutually exclusive from role.
                  The instance profile should already have a role assigned to it that Karpenter
                   has PassRole permission on for instance launch using this instanceProfile to succeed.
                type: string
                x-kubernetes-validations:
                - message: instanceProfile cannot be empty
                  rule: self != ''

Finally, the logical condition of them together:

required:
            - amiFamily
            - securityGroupSelectorTerms
            - subnetSelectorTerms
            type: object
            x-kubernetes-validations:
            - message: amiSelectorTerms is required when amiFamily == 'Custom'
              rule: 'self.amiFamily == ''Custom'' ? self.amiSelectorTerms.size() !=
                0 : true'
            - message: must specify exactly one of ['role', 'instanceProfile']
              rule: (has(self.role) && !has(self.instanceProfile)) || (!has(self.role)
                && has(self.instanceProfile))
            - message: changing from 'instanceProfile' to 'role' is not supported.
                You must delete and recreate this node class if you want to change
                this.
              rule: (has(oldSelf.role) && has(self.role)) || (has(oldSelf.instanceProfile)
                && has(self.instanceProfile))

I suspect these validations were added, and then the spec was updated. I have updated all specs for v1beta1 via:

# Installing on a test cluster:

❯ kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.sh_nodepools.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.sh_nodeclaims.yaml
kubectl apply -f https://raw.githubusercontent.com/aws/karpenter/main/pkg/apis/crds/karpenter.k8s.aws_ec2nodeclasses.yaml

customresourcedefinition.apiextensions.k8s.io/nodepools.karpenter.sh created
customresourcedefinition.apiextensions.k8s.io/nodeclaims.karpenter.sh created
customresourcedefinition.apiextensions.k8s.io/ec2nodeclasses.karpenter.k8s.aws created

# Running the utility:

 ❯ ./Utilities/crd-extractor.sh 

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant