diff --git a/charts/karpenter/README.md b/charts/karpenter/README.md index e832af62990e..dff07fdae245 100644 --- a/charts/karpenter/README.md +++ b/charts/karpenter/README.md @@ -29,7 +29,7 @@ helm upgrade --install --namespace karpenter --create-namespace \ | additionalAnnotations | object | `{}` | Additional annotations to add into metadata. | | additionalClusterRoleRules | list | `[]` | Specifies additional rules for the core ClusterRole. | | additionalLabels | object | `{}` | Additional labels to add into metadata. | -| affinity | object | `{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"karpenter.sh/provisioner-name","operator":"DoesNotExist"},{"key":"karpenter.sh/nodepool","operator":"DoesNotExist"}]}]}},"podAntiAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":[{"topologyKey":"kubernetes.io/hostname"}]}}` | Affinity rules for scheduling the pod. If an explicit label selector is not provided for pod affinity or pod anti-affinity one will be created from the pod selector labels. | +| affinity | object | `{"nodeAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":{"nodeSelectorTerms":[{"matchExpressions":[{"key":"karpenter.sh/nodepool","operator":"DoesNotExist"}]}]}},"podAntiAffinity":{"requiredDuringSchedulingIgnoredDuringExecution":[{"topologyKey":"kubernetes.io/hostname"}]}}` | Affinity rules for scheduling the pod. If an explicit label selector is not provided for pod affinity or pod anti-affinity one will be created from the pod selector labels. | | controller.env | list | `[]` | Additional environment variables for the controller pod. | | controller.envFrom | list | `[]` | | | controller.extraVolumeMounts | list | `[]` | Additional volumeMounts for the controller pod. | @@ -82,7 +82,7 @@ helm upgrade --install --namespace karpenter --create-namespace \ | settings.clusterEndpoint | string | `""` | Cluster endpoint. If not set, will be discovered during startup (EKS only) | | settings.clusterName | string | `""` | Cluster name. | | settings.featureGates | object | `{"drift":true}` | Feature Gate configuration values. Feature Gates will follow the same graduation process and requirements as feature gates in Kubernetes. More information here https://kubernetes.io/docs/reference/command-line-tools-reference/feature-gates/#feature-gates-for-alpha-or-beta-features | -| settings.featureGates.drift | bool | `true` | drift is in BETA and is enabled by default. Setting drift to false disables the drift disruption method to watch for drift between currently deployed nodes and the desired state of nodes set in provisioners and node templates | +| settings.featureGates.drift | bool | `true` | drift is in BETA and is enabled by default. Setting drift to false disables the drift disruption method to watch for drift between currently deployed nodes and the desired state of nodes set in nodepools and nodeclasses | | settings.interruptionQueue | string | `""` | interruptionQueue is disabled if not specified. Enabling interruption handling may require additional permissions on the controller service account. Additional permissions are outlined in the docs. | | settings.isolatedVPC | bool | `false` | If true then assume we can't reach AWS services which don't have a VPC endpoint This also has the effect of disabling look-ups to the AWS pricing endpoint | | settings.reservedENIs | string | `"0"` | Reserved ENIs are not included in the calculations for max-pods or kube-reserved This is most often used in the VPC CNI custom networking setup https://docs.aws.amazon.com/eks/latest/userguide/cni-custom-network.html | diff --git a/charts/karpenter/templates/webhooks-core.yaml b/charts/karpenter/templates/webhooks-core.yaml index 61b2ee7116e9..55297a1d96f6 100644 --- a/charts/karpenter/templates/webhooks-core.yaml +++ b/charts/karpenter/templates/webhooks-core.yaml @@ -20,17 +20,6 @@ webhooks: failurePolicy: Fail sideEffects: None rules: - - apiGroups: - - karpenter.sh - apiVersions: - - v1alpha5 - operations: - - CREATE - - UPDATE - resources: - - provisioners - - provisioners/status - scope: '*' - apiGroups: - karpenter.sh apiVersions: diff --git a/charts/karpenter/templates/webhooks.yaml b/charts/karpenter/templates/webhooks.yaml index e52fc722aab1..481574349b4b 100644 --- a/charts/karpenter/templates/webhooks.yaml +++ b/charts/karpenter/templates/webhooks.yaml @@ -20,17 +20,6 @@ webhooks: failurePolicy: Fail sideEffects: None rules: - - apiGroups: - - karpenter.k8s.aws - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - awsnodetemplates - - awsnodetemplates/status - scope: '*' - apiGroups: - karpenter.k8s.aws apiVersions: @@ -42,17 +31,6 @@ webhooks: - ec2nodeclasses - ec2nodeclasses/status scope: '*' - - apiGroups: - - karpenter.sh - apiVersions: - - v1alpha5 - operations: - - CREATE - - UPDATE - resources: - - provisioners - - provisioners/status - scope: '*' --- apiVersion: admissionregistration.k8s.io/v1 kind: ValidatingWebhookConfiguration @@ -75,17 +53,6 @@ webhooks: failurePolicy: Fail sideEffects: None rules: - - apiGroups: - - karpenter.k8s.aws - apiVersions: - - v1alpha1 - operations: - - CREATE - - UPDATE - resources: - - awsnodetemplates - - awsnodetemplates/status - scope: '*' - apiGroups: - karpenter.k8s.aws apiVersions: @@ -97,15 +64,4 @@ webhooks: - ec2nodeclasses - ec2nodeclasses/status scope: '*' - - apiGroups: - - karpenter.sh - apiVersions: - - v1alpha5 - operations: - - CREATE - - UPDATE - resources: - - provisioners - - provisioners/status - scope: '*' {{- end }} \ No newline at end of file diff --git a/charts/karpenter/values.yaml b/charts/karpenter/values.yaml index 51b57dea1589..9caec9cb61eb 100644 --- a/charts/karpenter/values.yaml +++ b/charts/karpenter/values.yaml @@ -67,8 +67,6 @@ affinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - - key: karpenter.sh/provisioner-name - operator: DoesNotExist - key: karpenter.sh/nodepool operator: DoesNotExist podAntiAffinity: @@ -199,5 +197,5 @@ settings: featureGates: # -- drift is in BETA and is enabled by default. # Setting drift to false disables the drift disruption method to watch for drift between currently deployed nodes - # and the desired state of nodes set in provisioners and node templates + # and the desired state of nodes set in nodepools and nodeclasses drift: true diff --git a/go.mod b/go.mod index eb1bda11118f..30a2a739b8a9 100644 --- a/go.mod +++ b/go.mod @@ -28,7 +28,7 @@ require ( k8s.io/utils v0.0.0-20230726121419-3b25d923346b knative.dev/pkg v0.0.0-20231010144348-ca8c009405dd sigs.k8s.io/controller-runtime v0.16.3 - sigs.k8s.io/karpenter v0.32.2-0.20231129204947-79dc3457680f + sigs.k8s.io/karpenter v0.32.2-0.20231205044115-1cf3df4ba3cc ) require ( diff --git a/go.sum b/go.sum index 986b3699278b..aa3df996249b 100644 --- a/go.sum +++ b/go.sum @@ -761,8 +761,8 @@ sigs.k8s.io/controller-runtime v0.16.3 h1:2TuvuokmfXvDUamSx1SuAOO3eTyye+47mJCigw sigs.k8s.io/controller-runtime v0.16.3/go.mod h1:j7bialYoSn142nv9sCOJmQgDXQXxnroFU4VnX/brVJ0= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= -sigs.k8s.io/karpenter v0.32.2-0.20231129204947-79dc3457680f h1:XFIp800IY0oJ0UQsp1zMbHih7dybqCqy+DzyVvxJKkY= -sigs.k8s.io/karpenter v0.32.2-0.20231129204947-79dc3457680f/go.mod h1:6FTggQw2OXqtUm2h2lTMlw/ej3TMvCmauOSYPQz/nWo= +sigs.k8s.io/karpenter v0.32.2-0.20231205044115-1cf3df4ba3cc h1:lkSGPb7CaayNByKoDJc3jxogAOuy1L0YM1N6G6PzwTQ= +sigs.k8s.io/karpenter v0.32.2-0.20231205044115-1cf3df4ba3cc/go.mod h1:7hPB7kdImFHAFp7pqPUqgBDOrh8GEcRnHT5uIlsXMKA= sigs.k8s.io/structured-merge-diff/v4 v4.3.0 h1:UZbZAZfX0wV2zr7YZorDz6GXROfDFj6LvqCRm4VUVKk= sigs.k8s.io/structured-merge-diff/v4 v4.3.0/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= diff --git a/hack/docs/instancetypes_gen_docs.go b/hack/docs/instancetypes_gen_docs.go index 736e01c72110..1b7842c8dc8f 100644 --- a/hack/docs/instancetypes_gen_docs.go +++ b/hack/docs/instancetypes_gen_docs.go @@ -15,9 +15,7 @@ limitations under the License. package main import ( - "bytes" "context" - "encoding/json" "flag" "fmt" "log" @@ -34,17 +32,17 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client" "sigs.k8s.io/controller-runtime/pkg/client/fake" "sigs.k8s.io/controller-runtime/pkg/manager" + "sigs.k8s.io/karpenter/pkg/apis/v1beta1" + + coreoperator "sigs.k8s.io/karpenter/pkg/operator" + coreoptions "sigs.k8s.io/karpenter/pkg/operator/options" + coretest "sigs.k8s.io/karpenter/pkg/test" awscloudprovider "github.com/aws/karpenter/pkg/cloudprovider" "github.com/aws/karpenter/pkg/operator" "github.com/aws/karpenter/pkg/operator/options" "github.com/aws/karpenter/pkg/test" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" - coreoperator "sigs.k8s.io/karpenter/pkg/operator" - coreoptions "sigs.k8s.io/karpenter/pkg/operator/options" - coretest "sigs.k8s.io/karpenter/pkg/test" - "github.com/aws/karpenter/pkg/apis/v1alpha1" "sigs.k8s.io/karpenter/pkg/cloudprovider" "sigs.k8s.io/karpenter/pkg/utils/resources" ) @@ -100,14 +98,6 @@ func main() { cp := awscloudprovider.New(op.InstanceTypesProvider, op.InstanceProvider, op.EventRecorder, op.GetClient(), op.AMIProvider, op.SecurityGroupProvider, op.SubnetProvider) - provider := v1alpha1.AWS{SubnetSelector: map[string]string{ - "*": "*", - }} - var buf bytes.Buffer - enc := json.NewEncoder(&buf) - if err := enc.Encode(provider); err != nil { - log.Fatalf("encoding provider, %s", err) - } instanceTypes, err := cp.GetInstanceTypes(ctx, nil) if err != nil { log.Fatalf("listing instance types, %s", err) @@ -156,7 +146,7 @@ below are the resources available with some assumptions and after the instance o // we don't want to show a few labels that will vary amongst regions delete(labelNameMap, v1.LabelTopologyZone) - delete(labelNameMap, v1alpha5.LabelCapacityType) + delete(labelNameMap, v1beta1.CapacityTypeLabelKey) labelNames := lo.Keys(labelNameMap) diff --git a/hack/docs/metrics_gen_docs.go b/hack/docs/metrics_gen_docs.go index ccf183e0564f..d01e362f5654 100644 --- a/hack/docs/metrics_gen_docs.go +++ b/hack/docs/metrics_gen_docs.go @@ -156,7 +156,7 @@ func getMetricsFromPackages(packages ...*ast.Package) []metricInfo { func bySubsystem(metrics []metricInfo) func(i int, j int) bool { subSystemSortOrder := map[string]int{} - subSystemSortOrder["provisioner"] = 1 + subSystemSortOrder["nodepool"] = 1 subSystemSortOrder["nodes"] = 2 subSystemSortOrder["pods"] = 3 subSystemSortOrder["cloudprovider"] = 4 diff --git a/pkg/apis/v1alpha1/awsnodetemplate.go b/pkg/apis/v1alpha1/awsnodetemplate.go deleted file mode 100644 index 7722bdf8443b..000000000000 --- a/pkg/apis/v1alpha1/awsnodetemplate.go +++ /dev/null @@ -1,119 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "fmt" - - "github.com/mitchellh/hashstructure/v2" - v1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" -) - -// Subnet contains resolved Subnet selector values utilized for node launch -type Subnet struct { - // ID of the subnet - // +required - ID string `json:"id"` - // The associated availability zone - // +required - Zone string `json:"zone"` -} - -// SecurityGroup contains resolved SecurityGroup selector values utilized for node launch -type SecurityGroup struct { - // ID of the security group - // +required - ID string `json:"id"` - // Name of the security group - // +optional - Name string `json:"name,omitempty"` -} - -// AMI contains resolved AMI selector values utilized for node launch -type AMI struct { - // ID of the AMI - // +required - ID string `json:"id"` - // Name of the AMI - // +optional - Name string `json:"name,omitempty"` - // Requirements of the AMI to be utilized on an instance type - // +required - Requirements []v1.NodeSelectorRequirement `json:"requirements"` -} - -// AWSNodeTemplateStatus contains the resolved state of the AWSNodeTemplate -type AWSNodeTemplateStatus struct { - // Subnets contains the current Subnet values that are available to the - // cluster under the subnet selectors. - // +optional - Subnets []Subnet `json:"subnets,omitempty"` - // SecurityGroups contains the current Security Groups values that are available to the - // cluster under the SecurityGroups selectors. - // +optional - SecurityGroups []SecurityGroup `json:"securityGroups,omitempty"` - // AMI contains the current AMI values that are available to the - // cluster under the AMI selectors. - // +optional - AMIs []AMI `json:"amis,omitempty"` -} - -// AWSNodeTemplateSpec is the top level specification for the AWS Karpenter Provider. -// This will contain configuration necessary to launch instances in AWS. -type AWSNodeTemplateSpec struct { - // UserData to be applied to the provisioned nodes. - // It must be in the appropriate format based on the AMIFamily in use. Karpenter will merge certain fields into - // this UserData to ensure nodes are being provisioned with the correct configuration. - // +optional - UserData *string `json:"userData,omitempty"` - AWS `json:",inline"` - // AMISelector discovers AMIs to be used by Amazon EC2 tags. - // +optional - AMISelector map[string]string `json:"amiSelector,omitempty" hash:"ignore"` - // DetailedMonitoring controls if detailed monitoring is enabled for instances that are launched - // +optional - DetailedMonitoring *bool `json:"detailedMonitoring,omitempty"` -} - -// AWSNodeTemplate is the Schema for the AWSNodeTemplate API -// +kubebuilder:object:root=true -// +kubebuilder:resource:path=awsnodetemplates,scope=Cluster,categories=karpenter -// +kubebuilder:subresource:status -type AWSNodeTemplate struct { - metav1.TypeMeta `json:",inline"` - metav1.ObjectMeta `json:"metadata,omitempty"` - - Spec AWSNodeTemplateSpec `json:"spec,omitempty"` - Status AWSNodeTemplateStatus `json:"status,omitempty"` -} - -func (a *AWSNodeTemplate) Hash() string { - hash, _ := hashstructure.Hash(a.Spec, hashstructure.FormatV2, &hashstructure.HashOptions{ - SlicesAsSets: true, - IgnoreZeroValue: true, - ZeroNil: true, - }) - - return fmt.Sprint(hash) -} - -// AWSNodeTemplateList contains a list of AWSNodeTemplate -// +kubebuilder:object:root=true -type AWSNodeTemplateList struct { - metav1.TypeMeta `json:",inline"` - metav1.ListMeta `json:"metadata,omitempty"` - Items []AWSNodeTemplate `json:"items"` -} diff --git a/pkg/apis/v1alpha1/awsnodetemplate_defaults.go b/pkg/apis/v1alpha1/awsnodetemplate_defaults.go deleted file mode 100644 index be97dd6b01a8..000000000000 --- a/pkg/apis/v1alpha1/awsnodetemplate_defaults.go +++ /dev/null @@ -1,23 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "context" -) - -// SetDefaults for the AWSNodeTemplate -func (a *AWSNodeTemplate) SetDefaults(_ context.Context) { -} diff --git a/pkg/apis/v1alpha1/awsnodetemplate_validation.go b/pkg/apis/v1alpha1/awsnodetemplate_validation.go deleted file mode 100644 index ee40667d1e59..000000000000 --- a/pkg/apis/v1alpha1/awsnodetemplate_validation.go +++ /dev/null @@ -1,120 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "context" - "fmt" - "regexp" - - admissionregistrationv1 "k8s.io/api/admissionregistration/v1" - "knative.dev/pkg/apis" - - "sigs.k8s.io/karpenter/pkg/utils/functional" -) - -const ( - userDataPath = "userData" - amiSelectorPath = "amiSelector" -) - -var ( - amiRegex = regexp.MustCompile("ami-[0-9a-z]+") -) - -func (a *AWSNodeTemplate) SupportedVerbs() []admissionregistrationv1.OperationType { - return []admissionregistrationv1.OperationType{ - admissionregistrationv1.Create, - admissionregistrationv1.Update, - } -} - -func (a *AWSNodeTemplate) Validate(ctx context.Context) (errs *apis.FieldError) { - return errs.Also( - apis.ValidateObjectMetadata(a).ViaField("metadata"), - a.Spec.validate(ctx).ViaField("spec"), - ) -} - -func (a *AWSNodeTemplateSpec) validate(_ context.Context) (errs *apis.FieldError) { - return errs.Also( - a.AWS.Validate(), - a.validateUserData(), - a.validateAMISelector(), - a.validateAMIFamily(), - a.validateTags(), - ) -} - -func (a *AWSNodeTemplateSpec) validateUserData() (errs *apis.FieldError) { - if a.UserData == nil { - return nil - } - if a.LaunchTemplateName != nil { - errs = errs.Also(apis.ErrMultipleOneOf(userDataPath, launchTemplatePath)) - } - return errs -} - -func (a *AWSNodeTemplateSpec) validateAMIFamily() (errs *apis.FieldError) { - if a.AMIFamily == nil { - return nil - } - if *a.AMIFamily == AMIFamilyCustom && a.AMISelector == nil { - errs = errs.Also(apis.ErrMissingField(amiSelectorPath)) - } - return errs -} - -//nolint:gocyclo -func (a *AWSNodeTemplateSpec) validateAMISelector() (errs *apis.FieldError) { - if a.AMISelector == nil { - return nil - } - if a.LaunchTemplateName != nil { - errs = errs.Also(apis.ErrMultipleOneOf(amiSelectorPath, launchTemplatePath)) - } - var idFilterKeyUsed string - for key, value := range a.AMISelector { - if key == "" || value == "" { - errs = errs.Also(apis.ErrInvalidValue("\"\"", fmt.Sprintf("%s['%s']", amiSelectorPath, key))) - } - if key == "aws-ids" || key == "aws::ids" { - idFilterKeyUsed = key - for _, amiID := range functional.SplitCommaSeparatedString(value) { - if !amiRegex.MatchString(amiID) { - fieldValue := fmt.Sprintf("\"%s\"", amiID) - message := fmt.Sprintf("%s['%s'] must be a valid ami-id (regex: %s)", amiSelectorPath, key, amiRegex.String()) - errs = errs.Also(apis.ErrInvalidValue(fieldValue, message)) - } - } - } - } - if idFilterKeyUsed != "" && len(a.AMISelector) > 1 { - errs = errs.Also(apis.ErrGeneric(fmt.Sprintf("%q filter is mutually exclusive, cannot be set with a combination of other filters in", idFilterKeyUsed), amiSelectorPath)) - } - return errs -} - -func (a *AWSNodeTemplateSpec) validateTags() (errs *apis.FieldError) { - for k := range a.Tags { - for _, pattern := range RestrictedTagPatterns { - if pattern.MatchString(k) { - errs = errs.Also(apis.ErrInvalidKeyName(k, "tags", fmt.Sprintf("tag contains a restricted tag matching %q", pattern.String()))) - } - } - } - return errs -} diff --git a/pkg/apis/v1alpha1/doc.go b/pkg/apis/v1alpha1/doc.go deleted file mode 100644 index b4b9e62295db..000000000000 --- a/pkg/apis/v1alpha1/doc.go +++ /dev/null @@ -1,20 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -// +kubebuilder:skip -// +k8s:openapi-gen=true -// +k8s:deepcopy-gen=package,register -// +k8s:defaulter-gen=TypeMeta -// +groupName=karpenter.k8s.aws -package v1alpha1 // doc.go is discovered by codegen diff --git a/pkg/apis/v1alpha1/provider.go b/pkg/apis/v1alpha1/provider.go deleted file mode 100644 index 0fe2f8071b06..000000000000 --- a/pkg/apis/v1alpha1/provider.go +++ /dev/null @@ -1,198 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "k8s.io/apimachinery/pkg/api/resource" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" -) - -// AWS contains parameters specific to this cloud provider -// +kubebuilder:object:root=true -type AWS struct { - // TypeMeta includes version and kind of the extensions, inferred if not provided. - // +optional - metav1.TypeMeta `json:",inline" hash:"ignore"` - // AMIFamily is the AMI family that instances use. - // +optional - AMIFamily *string `json:"amiFamily,omitempty"` - // Context is a Reserved field in EC2 APIs - // https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html - // +optional - Context *string `json:"context,omitempty"` - // InstanceProfile is the AWS identity that instances use. - // +optional - InstanceProfile *string `json:"instanceProfile,omitempty"` - // SubnetSelector discovers subnets by tags. A value of "" is a wildcard. - // +optional - SubnetSelector map[string]string `json:"subnetSelector,omitempty" hash:"ignore"` - // SecurityGroups specify the names of the security groups. - // +optional - SecurityGroupSelector map[string]string `json:"securityGroupSelector,omitempty" hash:"ignore"` - // Tags to be applied on ec2 resources like instances and launch templates. - // +optional - Tags map[string]string `json:"tags,omitempty"` - // LaunchTemplate parameters to use when generating an LT - LaunchTemplate `json:",inline,omitempty"` -} - -type LaunchTemplate struct { - // LaunchTemplateName for the node. If not specified, a launch template will be generated. - // NOTE: This field is for specifying a custom launch template and is exposed in the Spec - // as `launchTemplate` for backwards compatibility. - // +optional - LaunchTemplateName *string `json:"launchTemplate,omitempty" hash:"ignore"` - // MetadataOptions for the generated launch template of provisioned nodes. - // - // This specifies the exposure of the Instance Metadata Service to - // provisioned EC2 nodes. For more information, - // see Instance Metadata and User Data - // (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) - // in the Amazon Elastic Compute Cloud User Guide. - // - // Refer to recommended, security best practices - // (https://aws.github.io/aws-eks-best-practices/security/docs/iam/#restrict-access-to-the-instance-profile-assigned-to-the-worker-node) - // for limiting exposure of Instance Metadata and User Data to pods. - // If omitted, defaults to httpEndpoint enabled, with httpProtocolIPv6 - // disabled, with httpPutResponseLimit of 2, and with httpTokens - // required. - // +optional - MetadataOptions *MetadataOptions `json:"metadataOptions,omitempty"` - // BlockDeviceMappings to be applied to provisioned nodes. - // +optionals - BlockDeviceMappings []*BlockDeviceMapping `json:"blockDeviceMappings,omitempty"` -} - -// MetadataOptions contains parameters for specifying the exposure of the -// Instance Metadata Service to provisioned EC2 nodes. -type MetadataOptions struct { - // HTTPEndpoint enables or disables the HTTP metadata endpoint on provisioned - // nodes. If metadata options is non-nil, but this parameter is not specified, - // the default state is "enabled". - // - // If you specify a value of "disabled", instance metadata will not be accessible - // on the node. - // +optional - HTTPEndpoint *string `json:"httpEndpoint,omitempty"` - - // HTTPProtocolIPv6 enables or disables the IPv6 endpoint for the instance metadata - // service on provisioned nodes. If metadata options is non-nil, but this parameter - // is not specified, the default state is "disabled". - // +optional - HTTPProtocolIPv6 *string `json:"httpProtocolIPv6,omitempty"` - - // HTTPPutResponseHopLimit is the desired HTTP PUT response hop limit for - // instance metadata requests. The larger the number, the further instance - // metadata requests can travel. Possible values are integers from 1 to 64. - // If metadata options is non-nil, but this parameter is not specified, the - // default value is 1. - // +optional - HTTPPutResponseHopLimit *int64 `json:"httpPutResponseHopLimit,omitempty"` - - // HTTPTokens determines the state of token usage for instance metadata - // requests. If metadata options is non-nil, but this parameter is not - // specified, the default state is "optional". - // - // If the state is optional, one can choose to retrieve instance metadata with - // or without a signed token header on the request. If one retrieves the IAM - // role credentials without a token, the version 1.0 role credentials are - // returned. If one retrieves the IAM role credentials using a valid signed - // token, the version 2.0 role credentials are returned. - // - // If the state is "required", one must send a signed token header with any - // instance metadata retrieval requests. In this state, retrieving the IAM - // role credentials always returns the version 2.0 credentials; the version - // 1.0 credentials are not available. - // +optional - HTTPTokens *string `json:"httpTokens,omitempty"` -} - -type BlockDeviceMapping struct { - // The device name (for example, /dev/sdh or xvdh). - DeviceName *string `json:"deviceName,omitempty"` - // EBS contains parameters used to automatically set up EBS volumes when an instance is launched. - EBS *BlockDevice `json:"ebs,omitempty"` -} - -type BlockDevice struct { - // DeleteOnTermination indicates whether the EBS volume is deleted on instance termination. - DeleteOnTermination *bool `json:"deleteOnTermination,omitempty"` - - // Encrypted indicates whether the EBS volume is encrypted. Encrypted volumes can only - // be attached to instances that support Amazon EBS encryption. If you are creating - // a volume from a snapshot, you can't specify an encryption value. - Encrypted *bool `json:"encrypted,omitempty"` - - // IOPS is the number of I/O operations per second (IOPS). For gp3, io1, and io2 volumes, - // this represents the number of IOPS that are provisioned for the volume. For - // gp2 volumes, this represents the baseline performance of the volume and the - // rate at which the volume accumulates I/O credits for bursting. - // - // The following are the supported values for each volume type: - // - // * gp3: 3,000-16,000 IOPS - // - // * io1: 100-64,000 IOPS - // - // * io2: 100-64,000 IOPS - // - // For io1 and io2 volumes, we guarantee 64,000 IOPS only for Instances built - // on the Nitro System (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-types.html#ec2-nitro-instances). - // Other instance families guarantee performance up to 32,000 IOPS. - // - // This parameter is supported for io1, io2, and gp3 volumes only. This parameter - // is not supported for gp2, st1, sc1, or standard volumes. - IOPS *int64 `json:"iops,omitempty"` - - // KMSKeyID (ARN) of the symmetric Key Management Service (KMS) CMK used for encryption. - KMSKeyID *string `json:"kmsKeyID,omitempty"` - - // SnapshotID is the ID of an EBS snapshot - SnapshotID *string `json:"snapshotID,omitempty"` - - // Throughput to provision for a gp3 volume, with a maximum of 1,000 MiB/s. - // Valid Range: Minimum value of 125. Maximum value of 1000. - Throughput *int64 `json:"throughput,omitempty"` - - // VolumeSize in GiBs. You must specify either a snapshot ID or - // a volume size. The following are the supported volumes sizes for each volume - // type: - // - // * gp2 and gp3: 1-16,384 - // - // * io1 and io2: 4-16,384 - // - // * st1 and sc1: 125-16,384 - // - // * standard: 1-1,024 - VolumeSize *resource.Quantity `json:"volumeSize,omitempty" hash:"string"` - - // VolumeType of the block device. - // For more information, see Amazon EBS volume types (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html) - // in the Amazon Elastic Compute Cloud User Guide. - VolumeType *string `json:"volumeType,omitempty"` -} - -func DeserializeProvider(raw []byte) (*AWS, error) { - a := &AWS{} - _, gvk, err := codec.UniversalDeserializer().Decode(raw, nil, a) - if err != nil { - return nil, err - } - if gvk != nil { - a.SetGroupVersionKind(*gvk) - } - return a, nil -} diff --git a/pkg/apis/v1alpha1/provider_validation.go b/pkg/apis/v1alpha1/provider_validation.go deleted file mode 100644 index be230566abf1..000000000000 --- a/pkg/apis/v1alpha1/provider_validation.go +++ /dev/null @@ -1,266 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "fmt" - "regexp" - "strings" - - "github.com/aws/aws-sdk-go/service/ec2" - "k8s.io/apimachinery/pkg/api/resource" - "knative.dev/pkg/apis" - - "sigs.k8s.io/karpenter/pkg/utils/functional" -) - -const ( - launchTemplatePath = "launchTemplate" - securityGroupSelectorPath = "securityGroupSelector" - fieldPathSubnetSelectorPath = "subnetSelector" - amiFamilyPath = "amiFamily" - metadataOptionsPath = "metadataOptions" - instanceProfilePath = "instanceProfile" - blockDeviceMappingsPath = "blockDeviceMappings" -) - -var ( - minVolumeSize = *resource.NewScaledQuantity(1, resource.Giga) - maxVolumeSize = *resource.NewScaledQuantity(64, resource.Tera) - subnetRegex = regexp.MustCompile("subnet-[0-9a-z]+") - securityGroupRegex = regexp.MustCompile("sg-[0-9a-z]+") -) - -func (a *AWS) Validate() (errs *apis.FieldError) { - return errs.Also( - a.validate().ViaField("provider"), - ) -} - -func (a *AWS) validate() (errs *apis.FieldError) { - return errs.Also( - a.validateLaunchTemplate(), - a.validateSubnets(), - a.validateSecurityGroups(), - a.validateTags(), - a.validateMetadataOptions(), - a.validateAMIFamily(), - a.validateBlockDeviceMappings(), - ) -} - -func (a *AWS) validateLaunchTemplate() (errs *apis.FieldError) { - if a.LaunchTemplateName == nil { - return nil - } - if a.SecurityGroupSelector != nil { - errs = errs.Also(apis.ErrMultipleOneOf(launchTemplatePath, securityGroupSelectorPath)) - } - if a.MetadataOptions != nil { - errs = errs.Also(apis.ErrMultipleOneOf(launchTemplatePath, metadataOptionsPath)) - } - if a.AMIFamily != nil { - errs = errs.Also(apis.ErrMultipleOneOf(launchTemplatePath, amiFamilyPath)) - } - if a.InstanceProfile != nil { - errs = errs.Also(apis.ErrMultipleOneOf(launchTemplatePath, instanceProfilePath)) - } - if len(a.BlockDeviceMappings) != 0 { - errs = errs.Also(apis.ErrMultipleOneOf(launchTemplatePath, blockDeviceMappingsPath)) - } - return errs -} - -func (a *AWS) validateSubnets() (errs *apis.FieldError) { - if a.SubnetSelector == nil { - errs = errs.Also(apis.ErrMissingField(fieldPathSubnetSelectorPath)) - } - var idFilterKeyUsed string - for key, value := range a.SubnetSelector { - if key == "" || value == "" { - errs = errs.Also(apis.ErrInvalidValue("\"\"", fmt.Sprintf("%s['%s']", fieldPathSubnetSelectorPath, key))) - } - if key == "aws-ids" || key == "aws::ids" { - idFilterKeyUsed = key - for _, subnetID := range functional.SplitCommaSeparatedString(value) { - if !subnetRegex.MatchString(subnetID) { - fieldValue := fmt.Sprintf("\"%s\"", subnetID) - message := fmt.Sprintf("%s['%s'] must be a valid subnet-id (regex: %s)", fieldPathSubnetSelectorPath, key, subnetRegex.String()) - errs = errs.Also(apis.ErrInvalidValue(fieldValue, message)) - } - } - } - } - if idFilterKeyUsed != "" && len(a.SubnetSelector) > 1 { - errs = errs.Also(apis.ErrGeneric(fmt.Sprintf("%q filter is mutually exclusive, cannot be set with a combination of other filters in", idFilterKeyUsed), fieldPathSubnetSelectorPath)) - } - return errs -} - -//nolint:gocyclo -func (a *AWS) validateSecurityGroups() (errs *apis.FieldError) { - if a.LaunchTemplateName != nil { - return nil - } - if a.SecurityGroupSelector == nil { - errs = errs.Also(apis.ErrMissingField(securityGroupSelectorPath)) - } - var idFilterKeyUsed string - for key, value := range a.SecurityGroupSelector { - if key == "" || value == "" { - errs = errs.Also(apis.ErrInvalidValue("\"\"", fmt.Sprintf("%s['%s']", securityGroupSelectorPath, key))) - } - if key == "aws-ids" || key == "aws::ids" { - idFilterKeyUsed = key - for _, securityGroupID := range functional.SplitCommaSeparatedString(value) { - if !securityGroupRegex.MatchString(securityGroupID) { - fieldValue := fmt.Sprintf("\"%s\"", securityGroupID) - message := fmt.Sprintf("%s['%s'] must be a valid group-id (regex: %s)", securityGroupSelectorPath, key, securityGroupRegex.String()) - errs = errs.Also(apis.ErrInvalidValue(fieldValue, message)) - } - } - } - } - if idFilterKeyUsed != "" && len(a.SecurityGroupSelector) > 1 { - errs = errs.Also(apis.ErrGeneric(fmt.Sprintf("%q filter is mutually exclusive, cannot be set with a combination of other filters in", idFilterKeyUsed), securityGroupSelectorPath)) - } - return errs -} - -func (a *AWS) validateTags() (errs *apis.FieldError) { - // Avoiding a check on number of tags (hard limit of 50) since that limit is shared by user - // defined and Karpenter tags, and the latter could change over time. - for tagKey, tagValue := range a.Tags { - if tagKey == "" { - errs = errs.Also(apis.ErrInvalidValue(fmt.Sprintf( - "the tag with key : '' and value : '%s' is invalid because empty tag keys aren't supported", tagValue), "tags")) - } - } - return errs -} - -func (a *AWS) validateMetadataOptions() (errs *apis.FieldError) { - if a.MetadataOptions == nil { - return nil - } - return errs.Also( - a.validateHTTPEndpoint(), - a.validateHTTPProtocolIpv6(), - a.validateHTTPPutResponseHopLimit(), - a.validateHTTPTokens(), - ).ViaField(metadataOptionsPath) -} - -func (a *AWS) validateHTTPEndpoint() *apis.FieldError { - if a.MetadataOptions.HTTPEndpoint == nil { - return nil - } - return a.validateStringEnum(*a.MetadataOptions.HTTPEndpoint, "httpEndpoint", ec2.LaunchTemplateInstanceMetadataEndpointState_Values()) -} - -func (a *AWS) validateHTTPProtocolIpv6() *apis.FieldError { - if a.MetadataOptions.HTTPProtocolIPv6 == nil { - return nil - } - return a.validateStringEnum(*a.MetadataOptions.HTTPProtocolIPv6, "httpProtocolIPv6", ec2.LaunchTemplateInstanceMetadataProtocolIpv6_Values()) -} - -func (a *AWS) validateHTTPPutResponseHopLimit() *apis.FieldError { - if a.MetadataOptions.HTTPPutResponseHopLimit == nil { - return nil - } - limit := *a.MetadataOptions.HTTPPutResponseHopLimit - if limit < 1 || limit > 64 { - return apis.ErrOutOfBoundsValue(limit, 1, 64, "httpPutResponseHopLimit") - } - return nil -} - -func (a *AWS) validateHTTPTokens() *apis.FieldError { - if a.MetadataOptions.HTTPTokens == nil { - return nil - } - return a.validateStringEnum(*a.MetadataOptions.HTTPTokens, "httpTokens", ec2.LaunchTemplateHttpTokensState_Values()) -} - -func (a *AWS) validateAMIFamily() *apis.FieldError { - if a.AMIFamily == nil { - return nil - } - return a.validateStringEnum(*a.AMIFamily, amiFamilyPath, SupportedAMIFamilies) -} - -func (a *AWS) validateStringEnum(value, field string, validValues []string) *apis.FieldError { - for _, validValue := range validValues { - if value == validValue { - return nil - } - } - return apis.ErrInvalidValue(fmt.Sprintf("%s not in %v", value, strings.Join(validValues, ", ")), field) -} - -func (a *AWS) validateBlockDeviceMappings() (errs *apis.FieldError) { - for i, blockDeviceMapping := range a.BlockDeviceMappings { - if err := a.validateBlockDeviceMapping(blockDeviceMapping); err != nil { - errs = errs.Also(err.ViaFieldIndex(blockDeviceMappingsPath, i)) - } - } - return errs -} - -func (a *AWS) validateBlockDeviceMapping(blockDeviceMapping *BlockDeviceMapping) (errs *apis.FieldError) { - return errs.Also(a.validateDeviceName(blockDeviceMapping), a.validateEBS(blockDeviceMapping)) -} - -func (a *AWS) validateDeviceName(blockDeviceMapping *BlockDeviceMapping) *apis.FieldError { - if blockDeviceMapping.DeviceName == nil { - return apis.ErrMissingField("deviceName") - } - return nil -} - -func (a *AWS) validateEBS(blockDeviceMapping *BlockDeviceMapping) (errs *apis.FieldError) { - if blockDeviceMapping.EBS == nil { - return apis.ErrMissingField("ebs") - } - for _, err := range []*apis.FieldError{ - a.validateVolumeType(blockDeviceMapping), - a.validateVolumeSize(blockDeviceMapping), - } { - if err != nil { - errs = errs.Also(err.ViaField("ebs")) - } - } - return errs -} - -func (a *AWS) validateVolumeType(blockDeviceMapping *BlockDeviceMapping) *apis.FieldError { - if blockDeviceMapping.EBS.VolumeType != nil { - return a.validateStringEnum(*blockDeviceMapping.EBS.VolumeType, "volumeType", ec2.VolumeType_Values()) - } - return nil -} - -func (a *AWS) validateVolumeSize(blockDeviceMapping *BlockDeviceMapping) *apis.FieldError { - // If an EBS mapping is present, one of volumeSize or snapshotID must be present - if blockDeviceMapping.EBS.SnapshotID != nil && blockDeviceMapping.EBS.VolumeSize == nil { - return nil - } else if blockDeviceMapping.EBS.VolumeSize == nil { - return apis.ErrMissingField("volumeSize") - } else if blockDeviceMapping.EBS.VolumeSize.Cmp(minVolumeSize) == -1 || blockDeviceMapping.EBS.VolumeSize.Cmp(maxVolumeSize) == 1 { - return apis.ErrOutOfBoundsValue(blockDeviceMapping.EBS.VolumeSize.String(), minVolumeSize.String(), maxVolumeSize.String(), "volumeSize") - } - return nil -} diff --git a/pkg/apis/v1alpha1/register.go b/pkg/apis/v1alpha1/register.go deleted file mode 100644 index c4f572bdd8dc..000000000000 --- a/pkg/apis/v1alpha1/register.go +++ /dev/null @@ -1,154 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1 - -import ( - "fmt" - "regexp" - - "github.com/aws/aws-sdk-go/service/ec2" - v1 "k8s.io/api/core/v1" - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - "k8s.io/apimachinery/pkg/runtime" - "k8s.io/apimachinery/pkg/runtime/schema" - "k8s.io/apimachinery/pkg/runtime/serializer" - "k8s.io/apimachinery/pkg/util/sets" - - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" -) - -var ( - LabelDomain = "karpenter.k8s.aws" - - CapacityTypeSpot = ec2.DefaultTargetCapacityTypeSpot - CapacityTypeOnDemand = ec2.DefaultTargetCapacityTypeOnDemand - AWSToKubeArchitectures = map[string]string{ - "x86_64": v1alpha5.ArchitectureAmd64, - v1alpha5.ArchitectureArm64: v1alpha5.ArchitectureArm64, - } - WellKnownArchitectures = sets.NewString( - v1alpha5.ArchitectureAmd64, - v1alpha5.ArchitectureArm64, - ) - RestrictedLabelDomains = []string{ - LabelDomain, - } - RestrictedTagPatterns = []*regexp.Regexp{ - // Adheres to cluster name pattern matching as specified in the API spec - // https://docs.aws.amazon.com/eks/latest/APIReference/API_CreateCluster.html - regexp.MustCompile(`^kubernetes\.io/cluster/[0-9A-Za-z][A-Za-z0-9\-_]*$`), - regexp.MustCompile(fmt.Sprintf("^%s$", regexp.QuoteMeta(v1alpha5.ProvisionerNameLabelKey))), - regexp.MustCompile(fmt.Sprintf("^%s$", regexp.QuoteMeta(v1alpha5.MachineManagedByAnnotationKey))), - } - AMIFamilyBottlerocket = "Bottlerocket" - AMIFamilyAL2 = "AL2" - AMIFamilyUbuntu = "Ubuntu" - AMIFamilyWindows2019 = "Windows2019" - AMIFamilyWindows2022 = "Windows2022" - AMIFamilyCustom = "Custom" - SupportedAMIFamilies = []string{ - AMIFamilyBottlerocket, - AMIFamilyAL2, - AMIFamilyUbuntu, - AMIFamilyWindows2019, - AMIFamilyWindows2022, - AMIFamilyCustom, - } - SupportedContainerRuntimesByAMIFamily = map[string]sets.Set[string]{ - AMIFamilyBottlerocket: sets.New("containerd"), - AMIFamilyAL2: sets.New("dockerd", "containerd"), - AMIFamilyUbuntu: sets.New("dockerd", "containerd"), - AMIFamilyWindows2019: sets.New("dockerd", "containerd"), - AMIFamilyWindows2022: sets.New("dockerd", "containerd"), - } - - Windows2019 = "2019" - Windows2022 = "2022" - WindowsCore = "Core" - Windows2019Build = "10.0.17763" - Windows2022Build = "10.0.20348" - ResourceNVIDIAGPU v1.ResourceName = "nvidia.com/gpu" - ResourceAMDGPU v1.ResourceName = "amd.com/gpu" - ResourceAWSNeuron v1.ResourceName = "aws.amazon.com/neuron" - ResourceHabanaGaudi v1.ResourceName = "habana.ai/gaudi" - ResourceAWSPodENI v1.ResourceName = "vpc.amazonaws.com/pod-eni" - ResourcePrivateIPv4Address v1.ResourceName = "vpc.amazonaws.com/PrivateIPv4Address" - NVIDIAacceleratorManufacturer AcceleratorManufacturer = "nvidia" - AWSAcceleratorManufacturer AcceleratorManufacturer = "aws" - - LabelInstanceHypervisor = LabelDomain + "/instance-hypervisor" - LabelInstanceEncryptionInTransitSupported = LabelDomain + "/instance-encryption-in-transit-supported" - LabelInstanceCategory = LabelDomain + "/instance-category" - LabelInstanceFamily = LabelDomain + "/instance-family" - LabelInstanceGeneration = LabelDomain + "/instance-generation" - LabelInstanceLocalNVME = LabelDomain + "/instance-local-nvme" - LabelInstanceSize = LabelDomain + "/instance-size" - LabelInstanceCPU = LabelDomain + "/instance-cpu" - LabelInstanceMemory = LabelDomain + "/instance-memory" - LabelInstanceNetworkBandwidth = LabelDomain + "/instance-network-bandwidth" - LabelInstancePods = LabelDomain + "/instance-pods" - LabelInstanceGPUName = LabelDomain + "/instance-gpu-name" - LabelInstanceGPUManufacturer = LabelDomain + "/instance-gpu-manufacturer" - LabelInstanceGPUCount = LabelDomain + "/instance-gpu-count" - LabelInstanceGPUMemory = LabelDomain + "/instance-gpu-memory" - LabelInstanceAMIID = LabelDomain + "/instance-ami-id" - LabelInstanceAcceleratorName = LabelDomain + "/instance-accelerator-name" - LabelInstanceAcceleratorManufacturer = LabelDomain + "/instance-accelerator-manufacturer" - LabelInstanceAcceleratorCount = LabelDomain + "/instance-accelerator-count" - AnnotationNodeTemplateHash = LabelDomain + "/nodetemplate-hash" -) - -var ( - Scheme = runtime.NewScheme() - codec = serializer.NewCodecFactory(Scheme, serializer.EnableStrict) - Group = "karpenter.k8s.aws" - SchemeGroupVersion = schema.GroupVersion{Group: Group, Version: "v1alpha1"} - SchemeBuilder = runtime.NewSchemeBuilder(func(scheme *runtime.Scheme) error { - scheme.AddKnownTypes(SchemeGroupVersion, - &AWSNodeTemplate{}, - &AWSNodeTemplateList{}, - ) - metav1.AddToGroupVersion(scheme, SchemeGroupVersion) - return nil - }) -) - -func init() { - Scheme.AddKnownTypes(schema.GroupVersion{Group: v1alpha5.ExtensionsGroup, Version: "v1alpha1"}, &AWS{}) - v1alpha5.RestrictedLabelDomains = v1alpha5.RestrictedLabelDomains.Insert(RestrictedLabelDomains...) - v1alpha5.WellKnownLabels = v1alpha5.WellKnownLabels.Insert( - LabelInstanceHypervisor, - LabelInstanceEncryptionInTransitSupported, - LabelInstanceCategory, - LabelInstanceFamily, - LabelInstanceGeneration, - LabelInstanceSize, - LabelInstanceLocalNVME, - LabelInstanceCPU, - LabelInstanceMemory, - LabelInstanceNetworkBandwidth, - LabelInstancePods, - LabelInstanceGPUName, - LabelInstanceGPUManufacturer, - LabelInstanceGPUCount, - LabelInstanceGPUMemory, - LabelInstanceAcceleratorName, - LabelInstanceAcceleratorManufacturer, - LabelInstanceAcceleratorCount, - v1.LabelWindowsBuild, - ) -} - -type AcceleratorManufacturer string diff --git a/pkg/apis/v1alpha1/suite_test.go b/pkg/apis/v1alpha1/suite_test.go deleted file mode 100644 index c1fcf7603a92..000000000000 --- a/pkg/apis/v1alpha1/suite_test.go +++ /dev/null @@ -1,322 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha1_test - -import ( - "context" - "fmt" - "strings" - "testing" - - "github.com/Pallinder/go-randomdata" - "github.com/mitchellh/hashstructure/v2" - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - . "knative.dev/pkg/logging/testing" - "knative.dev/pkg/ptr" - - metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" - - "github.com/aws/aws-sdk-go/aws" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/test" -) - -var ctx context.Context - -func TestAPIs(t *testing.T) { - ctx = TestContextWithLogger(t) - RegisterFailHandler(Fail) - RunSpecs(t, "Validation") -} - -var _ = Describe("Validation", func() { - var ant *v1alpha1.AWSNodeTemplate - - BeforeEach(func() { - ant = &v1alpha1.AWSNodeTemplate{ - ObjectMeta: metav1.ObjectMeta{Name: strings.ToLower(randomdata.SillyName())}, - Spec: v1alpha1.AWSNodeTemplateSpec{ - AWS: v1alpha1.AWS{ - SubnetSelector: map[string]string{"foo": "bar"}, - SecurityGroupSelector: map[string]string{"foo": "bar"}, - }, - }, - } - }) - - Context("SubnetSelector", func() { - It("should succeed with a valid subnet selector", func() { - ant.Spec.SubnetSelector = map[string]string{ - "key1": "value1", - "key2": "value2", - "key3": "value3", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should succeed with a valid id subnet selector", func() { - ant.Spec.SubnetSelector = map[string]string{ - "aws-ids": "subnet-123,subnet-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - - ant.Spec.SubnetSelector = map[string]string{ - "aws::ids": "subnet-123,subnet-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should fail when a id subnet selector is used in combination with tags", func() { - ant.Spec.SubnetSelector = map[string]string{ - "aws-ids": "subnet-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - - ant.Spec.SubnetSelector = map[string]string{ - "aws::ids": "subnet-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - }) - }) - Context("SecurityGroupSelector", func() { - It("should succeed with a valid security group selector", func() { - ant.Spec.SecurityGroupSelector = map[string]string{ - "key1": "value1", - "key2": "value2", - "key3": "value3", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should succeed with a valid id security group selector", func() { - ant.Spec.SecurityGroupSelector = map[string]string{ - "aws-ids": "sg-123,sg-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - - ant.Spec.SecurityGroupSelector = map[string]string{ - "aws::ids": "sg-123,sg-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should fail when a id security group selector is used in combination with tags", func() { - ant.Spec.SecurityGroupSelector = map[string]string{ - "aws-ids": "sg-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - - ant.Spec.SecurityGroupSelector = map[string]string{ - "aws::ids": "sg-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - }) - }) - Context("AMISelector", func() { - It("should succeed with a valid ami selector", func() { - ant.Spec.AMISelector = map[string]string{ - "key1": "value1", - "key2": "value2", - "key3": "value3", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should succeed with a valid id ami selector", func() { - ant.Spec.AMISelector = map[string]string{ - "aws-ids": "ami-123,ami-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - - ant.Spec.AMISelector = map[string]string{ - "aws::ids": "ami-123,ami-456", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should fail when a id ami selector is used in combination with tags", func() { - ant.Spec.AMISelector = map[string]string{ - "aws-ids": "ami-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - - ant.Spec.AMISelector = map[string]string{ - "aws::ids": "ami-123", - "foo": "bar", - } - Expect(ant.Validate(ctx)).ToNot(Succeed()) - }) - }) - Context("UserData", func() { - It("should succeed if user data is empty", func() { - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should fail if launch template is also specified", func() { - ant.Spec.LaunchTemplateName = ptr.String("someLaunchTemplate") - ant.Spec.UserData = ptr.String("someUserData") - Expect(ant.Validate(ctx)).To(Not(Succeed())) - }) - }) - Context("Tags", func() { - It("should succeed when tags are empty", func() { - ant.Spec.Tags = map[string]string{} - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should succeed if tags aren't in restricted tag keys", func() { - ant.Spec.Tags = map[string]string{ - "karpenter.sh/custom-key": "value", - "karpenter.sh/managed": "true", - "kubernetes.io/role/key": "value", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should succeed by validating that regex is properly escaped", func() { - ant.Spec.Tags = map[string]string{ - "karpenterzsh/provisioner-name": "value", - } - Expect(ant.Validate(ctx)).To(Succeed()) - ant.Spec.Tags = map[string]string{ - "kubernetesbio/cluster/test": "value", - } - Expect(ant.Validate(ctx)).To(Succeed()) - ant.Spec.Tags = map[string]string{ - "karpenterzsh/managed-by": "test", - } - Expect(ant.Validate(ctx)).To(Succeed()) - }) - It("should fail if tags contain a restricted domain key", func() { - ant.Spec.Tags = map[string]string{ - "karpenter.sh/provisioner-name": "value", - } - Expect(ant.Validate(ctx)).To(Not(Succeed())) - ant.Spec.Tags = map[string]string{ - "kubernetes.io/cluster/test": "value", - } - Expect(ant.Validate(ctx)).To(Not(Succeed())) - ant.Spec.Tags = map[string]string{ - "karpenter.sh/managed-by": "test", - } - Expect(ant.Validate(ctx)).To(Not(Succeed())) - }) - }) - var _ = Describe("AWSNodeTemplate Hash", func() { - var awsnodetemplatespec v1alpha1.AWSNodeTemplateSpec - var awsnodetemplate *v1alpha1.AWSNodeTemplate - const awsnodetemplateStaticHash = "8218109239399812816" - - BeforeEach(func() { - awsnodetemplatespec = v1alpha1.AWSNodeTemplateSpec{ - AWS: v1alpha1.AWS{ - AMIFamily: aws.String(v1alpha1.AMIFamilyAL2), - Context: aws.String("context-1"), - InstanceProfile: aws.String("profile-1"), - Tags: map[string]string{ - "keyTag-1": "valueTag-1", - "keyTag-2": "valueTag-2", - }, - LaunchTemplate: v1alpha1.LaunchTemplate{ - MetadataOptions: &v1alpha1.MetadataOptions{ - HTTPEndpoint: aws.String("test-metadata-1"), - }, - BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{ - { - DeviceName: aws.String("map-device-1"), - }, - { - DeviceName: aws.String("map-device-2"), - }, - }, - }, - }, - UserData: aws.String("userdata-test-1"), - DetailedMonitoring: aws.Bool(false), - } - awsnodetemplate = test.AWSNodeTemplate(awsnodetemplatespec) - }) - DescribeTable( - "should match static hash", - func(hash string, specs ...v1alpha1.AWSNodeTemplateSpec) { - specs = append([]v1alpha1.AWSNodeTemplateSpec{awsnodetemplatespec}, specs...) - nodeTemplate := test.AWSNodeTemplate(specs...) - Expect(nodeTemplate.Hash()).To(Equal(hash)) - }, - Entry("Base AWSNodeTemplate", awsnodetemplateStaticHash), - - // Static fields, expect changed hash from base - Entry("InstanceProfile Drift", "7151640568926200147", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{InstanceProfile: aws.String("profile-2")}}), - Entry("UserData Drift", "7125936663475632400", v1alpha1.AWSNodeTemplateSpec{UserData: aws.String("userdata-test-2")}), - Entry("Tags Drift", "7008297732848636107", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Tags: map[string]string{"keyTag-test-3": "valueTag-test-3"}}}), - Entry("MetadataOptions Drift", "3771503890852427396", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{MetadataOptions: &v1alpha1.MetadataOptions{HTTPEndpoint: aws.String("test-metadata-2")}}}}), - Entry("BlockDeviceMappings Drift", "13540813918064174930", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{{DeviceName: aws.String("map-device-test-3")}}}}}), - Entry("Context Drift", "14848954101731282288", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Context: aws.String("context-2")}}), - Entry("DetailedMonitoring Drift", "1327478230553204075", v1alpha1.AWSNodeTemplateSpec{DetailedMonitoring: aws.Bool(true)}), - Entry("AMIFamily Drift", "11757951095500780022", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{AMIFamily: aws.String(v1alpha1.AMIFamilyBottlerocket)}}), - Entry("Reorder Tags", "8218109239399812816", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Tags: map[string]string{"keyTag-2": "valueTag-2", "keyTag-1": "valueTag-1"}}}), - Entry("Reorder BlockDeviceMapping", "8218109239399812816", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{{DeviceName: aws.String("map-device-2")}, {DeviceName: aws.String("map-device-1")}}}}}), - - // Behavior / Dynamic fields, expect same hash as base - Entry("Modified AMISelector", awsnodetemplateStaticHash, v1alpha1.AWSNodeTemplateSpec{AMISelector: map[string]string{"ami-test-key": "ami-test-value"}}), - Entry("Modified SubnetSelector", awsnodetemplateStaticHash, v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{SubnetSelector: map[string]string{"subnet-test-key": "subnet-test-value"}}}), - Entry("Modified SecurityGroupSelector", awsnodetemplateStaticHash, v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{SecurityGroupSelector: map[string]string{"security-group-test-key": "security-group-test-value"}}}), - Entry("Modified LaunchTemplateName", awsnodetemplateStaticHash, v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{LaunchTemplateName: aws.String("foobar")}}}), - ) - DescribeTable("should change hash when static fields are updated", func(awsnodetemplatespec v1alpha1.AWSNodeTemplateSpec) { - expectedHash := awsnodetemplate.Hash() - updatedAWSNodeTemplate := test.AWSNodeTemplate(*awsnodetemplatespec.DeepCopy(), awsnodetemplatespec) - actualHash := updatedAWSNodeTemplate.Hash() - Expect(actualHash).ToNot(Equal(fmt.Sprint(expectedHash))) - }, - Entry("InstanceProfile Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{InstanceProfile: aws.String("profile-2")}}), - Entry("UserData Drift", v1alpha1.AWSNodeTemplateSpec{UserData: aws.String("userdata-test-2")}), - Entry("Tags Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Tags: map[string]string{"keyTag-test-3": "valueTag-test-3"}}}), - Entry("MetadataOptions Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{MetadataOptions: &v1alpha1.MetadataOptions{HTTPEndpoint: aws.String("test-metadata-2")}}}}), - Entry("BlockDeviceMappings Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{{DeviceName: aws.String("map-device-test-3")}}}}}), - Entry("Context Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Context: aws.String("context-2")}}), - Entry("DetailedMonitoring Drift", v1alpha1.AWSNodeTemplateSpec{DetailedMonitoring: aws.Bool(true)}), - Entry("AMIFamily Drift", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{AMIFamily: aws.String(v1alpha1.AMIFamilyBottlerocket)}}), - Entry("Reorder Tags", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{Tags: map[string]string{"keyTag-2": "valueTag-2", "keyTag-1": "valueTag-1"}}}), - Entry("Reorder BlockDeviceMapping", v1alpha1.AWSNodeTemplateSpec{AWS: v1alpha1.AWS{LaunchTemplate: v1alpha1.LaunchTemplate{BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{{DeviceName: aws.String("map-device-2")}, {DeviceName: aws.String("map-device-1")}}}}}), - ) - It("should not change hash when behavior/dynamic fields are updated", func() { - actualHash := awsnodetemplate.Hash() - - expectedHash, err := hashstructure.Hash(awsnodetemplate.Spec, hashstructure.FormatV2, &hashstructure.HashOptions{ - SlicesAsSets: true, - IgnoreZeroValue: true, - ZeroNil: true, - }) - Expect(err).ToNot(HaveOccurred()) - Expect(actualHash).To(Equal(fmt.Sprint(expectedHash))) - - // Update a behavior/dynamic field - awsnodetemplate.Spec.SubnetSelector = map[string]string{"subnet-test-key": "subnet-test-value"} - awsnodetemplate.Spec.SecurityGroupSelector = map[string]string{"sg-test-key": "sg-test-value"} - awsnodetemplate.Spec.AMISelector = map[string]string{"ami-test-key": "ami-test-value"} - - actualHash = awsnodetemplate.Hash() - Expect(err).ToNot(HaveOccurred()) - Expect(actualHash).To(Equal(fmt.Sprint(expectedHash))) - }) - It("should expect two provisioner with the same spec to have the same provisioner hash", func() { - awsnodetemplateTwo := &v1alpha1.AWSNodeTemplate{ - ObjectMeta: metav1.ObjectMeta{Name: strings.ToLower(randomdata.SillyName())}, - } - awsnodetemplateTwo.Spec = awsnodetemplatespec - - Expect(awsnodetemplate.Hash()).To(Equal(awsnodetemplateTwo.Hash())) - }) - }) -}) diff --git a/pkg/apis/v1alpha1/zz_generated.deepcopy.go b/pkg/apis/v1alpha1/zz_generated.deepcopy.go deleted file mode 100644 index 14309d1b7389..000000000000 --- a/pkg/apis/v1alpha1/zz_generated.deepcopy.go +++ /dev/null @@ -1,412 +0,0 @@ -//go:build !ignore_autogenerated - -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -// Code generated by controller-gen. DO NOT EDIT. - -package v1alpha1 - -import ( - "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/runtime" -) - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AMI) DeepCopyInto(out *AMI) { - *out = *in - if in.Requirements != nil { - in, out := &in.Requirements, &out.Requirements - *out = make([]v1.NodeSelectorRequirement, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AMI. -func (in *AMI) DeepCopy() *AMI { - if in == nil { - return nil - } - out := new(AMI) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AWS) DeepCopyInto(out *AWS) { - *out = *in - out.TypeMeta = in.TypeMeta - if in.AMIFamily != nil { - in, out := &in.AMIFamily, &out.AMIFamily - *out = new(string) - **out = **in - } - if in.Context != nil { - in, out := &in.Context, &out.Context - *out = new(string) - **out = **in - } - if in.InstanceProfile != nil { - in, out := &in.InstanceProfile, &out.InstanceProfile - *out = new(string) - **out = **in - } - if in.SubnetSelector != nil { - in, out := &in.SubnetSelector, &out.SubnetSelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - if in.SecurityGroupSelector != nil { - in, out := &in.SecurityGroupSelector, &out.SecurityGroupSelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - if in.Tags != nil { - in, out := &in.Tags, &out.Tags - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - in.LaunchTemplate.DeepCopyInto(&out.LaunchTemplate) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AWS. -func (in *AWS) DeepCopy() *AWS { - if in == nil { - return nil - } - out := new(AWS) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *AWS) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AWSNodeTemplate) DeepCopyInto(out *AWSNodeTemplate) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AWSNodeTemplate. -func (in *AWSNodeTemplate) DeepCopy() *AWSNodeTemplate { - if in == nil { - return nil - } - out := new(AWSNodeTemplate) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *AWSNodeTemplate) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AWSNodeTemplateList) DeepCopyInto(out *AWSNodeTemplateList) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ListMeta.DeepCopyInto(&out.ListMeta) - if in.Items != nil { - in, out := &in.Items, &out.Items - *out = make([]AWSNodeTemplate, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AWSNodeTemplateList. -func (in *AWSNodeTemplateList) DeepCopy() *AWSNodeTemplateList { - if in == nil { - return nil - } - out := new(AWSNodeTemplateList) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *AWSNodeTemplateList) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AWSNodeTemplateSpec) DeepCopyInto(out *AWSNodeTemplateSpec) { - *out = *in - if in.UserData != nil { - in, out := &in.UserData, &out.UserData - *out = new(string) - **out = **in - } - in.AWS.DeepCopyInto(&out.AWS) - if in.AMISelector != nil { - in, out := &in.AMISelector, &out.AMISelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - if in.DetailedMonitoring != nil { - in, out := &in.DetailedMonitoring, &out.DetailedMonitoring - *out = new(bool) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AWSNodeTemplateSpec. -func (in *AWSNodeTemplateSpec) DeepCopy() *AWSNodeTemplateSpec { - if in == nil { - return nil - } - out := new(AWSNodeTemplateSpec) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *AWSNodeTemplateStatus) DeepCopyInto(out *AWSNodeTemplateStatus) { - *out = *in - if in.Subnets != nil { - in, out := &in.Subnets, &out.Subnets - *out = make([]Subnet, len(*in)) - copy(*out, *in) - } - if in.SecurityGroups != nil { - in, out := &in.SecurityGroups, &out.SecurityGroups - *out = make([]SecurityGroup, len(*in)) - copy(*out, *in) - } - if in.AMIs != nil { - in, out := &in.AMIs, &out.AMIs - *out = make([]AMI, len(*in)) - for i := range *in { - (*in)[i].DeepCopyInto(&(*out)[i]) - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new AWSNodeTemplateStatus. -func (in *AWSNodeTemplateStatus) DeepCopy() *AWSNodeTemplateStatus { - if in == nil { - return nil - } - out := new(AWSNodeTemplateStatus) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *BlockDevice) DeepCopyInto(out *BlockDevice) { - *out = *in - if in.DeleteOnTermination != nil { - in, out := &in.DeleteOnTermination, &out.DeleteOnTermination - *out = new(bool) - **out = **in - } - if in.Encrypted != nil { - in, out := &in.Encrypted, &out.Encrypted - *out = new(bool) - **out = **in - } - if in.IOPS != nil { - in, out := &in.IOPS, &out.IOPS - *out = new(int64) - **out = **in - } - if in.KMSKeyID != nil { - in, out := &in.KMSKeyID, &out.KMSKeyID - *out = new(string) - **out = **in - } - if in.SnapshotID != nil { - in, out := &in.SnapshotID, &out.SnapshotID - *out = new(string) - **out = **in - } - if in.Throughput != nil { - in, out := &in.Throughput, &out.Throughput - *out = new(int64) - **out = **in - } - if in.VolumeSize != nil { - in, out := &in.VolumeSize, &out.VolumeSize - x := (*in).DeepCopy() - *out = &x - } - if in.VolumeType != nil { - in, out := &in.VolumeType, &out.VolumeType - *out = new(string) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BlockDevice. -func (in *BlockDevice) DeepCopy() *BlockDevice { - if in == nil { - return nil - } - out := new(BlockDevice) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *BlockDeviceMapping) DeepCopyInto(out *BlockDeviceMapping) { - *out = *in - if in.DeviceName != nil { - in, out := &in.DeviceName, &out.DeviceName - *out = new(string) - **out = **in - } - if in.EBS != nil { - in, out := &in.EBS, &out.EBS - *out = new(BlockDevice) - (*in).DeepCopyInto(*out) - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BlockDeviceMapping. -func (in *BlockDeviceMapping) DeepCopy() *BlockDeviceMapping { - if in == nil { - return nil - } - out := new(BlockDeviceMapping) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *LaunchTemplate) DeepCopyInto(out *LaunchTemplate) { - *out = *in - if in.LaunchTemplateName != nil { - in, out := &in.LaunchTemplateName, &out.LaunchTemplateName - *out = new(string) - **out = **in - } - if in.MetadataOptions != nil { - in, out := &in.MetadataOptions, &out.MetadataOptions - *out = new(MetadataOptions) - (*in).DeepCopyInto(*out) - } - if in.BlockDeviceMappings != nil { - in, out := &in.BlockDeviceMappings, &out.BlockDeviceMappings - *out = make([]*BlockDeviceMapping, len(*in)) - for i := range *in { - if (*in)[i] != nil { - in, out := &(*in)[i], &(*out)[i] - *out = new(BlockDeviceMapping) - (*in).DeepCopyInto(*out) - } - } - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new LaunchTemplate. -func (in *LaunchTemplate) DeepCopy() *LaunchTemplate { - if in == nil { - return nil - } - out := new(LaunchTemplate) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *MetadataOptions) DeepCopyInto(out *MetadataOptions) { - *out = *in - if in.HTTPEndpoint != nil { - in, out := &in.HTTPEndpoint, &out.HTTPEndpoint - *out = new(string) - **out = **in - } - if in.HTTPProtocolIPv6 != nil { - in, out := &in.HTTPProtocolIPv6, &out.HTTPProtocolIPv6 - *out = new(string) - **out = **in - } - if in.HTTPPutResponseHopLimit != nil { - in, out := &in.HTTPPutResponseHopLimit, &out.HTTPPutResponseHopLimit - *out = new(int64) - **out = **in - } - if in.HTTPTokens != nil { - in, out := &in.HTTPTokens, &out.HTTPTokens - *out = new(string) - **out = **in - } -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new MetadataOptions. -func (in *MetadataOptions) DeepCopy() *MetadataOptions { - if in == nil { - return nil - } - out := new(MetadataOptions) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *SecurityGroup) DeepCopyInto(out *SecurityGroup) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new SecurityGroup. -func (in *SecurityGroup) DeepCopy() *SecurityGroup { - if in == nil { - return nil - } - out := new(SecurityGroup) - in.DeepCopyInto(out) - return out -} - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *Subnet) DeepCopyInto(out *Subnet) { - *out = *in -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Subnet. -func (in *Subnet) DeepCopy() *Subnet { - if in == nil { - return nil - } - out := new(Subnet) - in.DeepCopyInto(out) - return out -} diff --git a/pkg/apis/v1alpha5/provisioner.go b/pkg/apis/v1alpha5/provisioner.go deleted file mode 100644 index 2c85bb830a0e..000000000000 --- a/pkg/apis/v1alpha5/provisioner.go +++ /dev/null @@ -1,87 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha5 - -import ( - "context" - - "github.com/aws/aws-sdk-go/service/ec2" - admissionregistrationv1 "k8s.io/api/admissionregistration/v1" - v1 "k8s.io/api/core/v1" - "knative.dev/pkg/apis" - - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" - "sigs.k8s.io/karpenter/pkg/scheduling" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" -) - -// Provisioner is an alias type for additional validation -// +kubebuilder:object:root=true -type Provisioner v1alpha5.Provisioner - -func (p *Provisioner) SupportedVerbs() []admissionregistrationv1.OperationType { - return []admissionregistrationv1.OperationType{ - admissionregistrationv1.Create, - admissionregistrationv1.Update, - } -} - -func (p *Provisioner) Validate(_ context.Context) (errs *apis.FieldError) { - if p.Spec.Provider == nil { - return nil - } - provider, err := v1alpha1.DeserializeProvider(p.Spec.Provider.Raw) - if err != nil { - return apis.ErrGeneric(err.Error()) - } - return provider.Validate() -} - -func (p *Provisioner) SetDefaults(_ context.Context) { - requirements := scheduling.NewNodeSelectorRequirements(p.Spec.Requirements...) - - // default to linux OS - if !requirements.Has(v1.LabelOSStable) { - p.Spec.Requirements = append(p.Spec.Requirements, v1.NodeSelectorRequirement{ - Key: v1.LabelOSStable, Operator: v1.NodeSelectorOpIn, Values: []string{string(v1.Linux)}, - }) - } - - // default to amd64 - if !requirements.Has(v1.LabelArchStable) { - p.Spec.Requirements = append(p.Spec.Requirements, v1.NodeSelectorRequirement{ - Key: v1.LabelArchStable, Operator: v1.NodeSelectorOpIn, Values: []string{v1alpha5.ArchitectureAmd64}, - }) - } - - // default to on-demand - if !requirements.Has(v1alpha5.LabelCapacityType) { - p.Spec.Requirements = append(p.Spec.Requirements, v1.NodeSelectorRequirement{ - Key: v1alpha5.LabelCapacityType, Operator: v1.NodeSelectorOpIn, Values: []string{ec2.DefaultTargetCapacityTypeOnDemand}, - }) - } - - // default to C, M, R categories if no instance type constraints are specified - if !requirements.Has(v1.LabelInstanceTypeStable) && - !requirements.Has(v1alpha1.LabelInstanceFamily) && - !requirements.Has(v1alpha1.LabelInstanceCategory) && - !requirements.Has(v1alpha1.LabelInstanceGeneration) { - p.Spec.Requirements = append(p.Spec.Requirements, []v1.NodeSelectorRequirement{ - {Key: v1alpha1.LabelInstanceCategory, Operator: v1.NodeSelectorOpIn, Values: []string{"c", "m", "r"}}, - {Key: v1alpha1.LabelInstanceGeneration, Operator: v1.NodeSelectorOpGt, Values: []string{"2"}}, - }...) - } -} diff --git a/pkg/apis/v1alpha5/suite_test.go b/pkg/apis/v1alpha5/suite_test.go deleted file mode 100644 index 68eb0649607b..000000000000 --- a/pkg/apis/v1alpha5/suite_test.go +++ /dev/null @@ -1,414 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package v1alpha5_test - -import ( - "context" - "math" - "testing" - - "github.com/Pallinder/go-randomdata" - "github.com/aws/aws-sdk-go/aws" - "github.com/aws/aws-sdk-go/service/ec2" - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - "github.com/samber/lo" - "go.uber.org/multierr" - v1 "k8s.io/api/core/v1" - "k8s.io/apimachinery/pkg/api/resource" - . "knative.dev/pkg/logging/testing" - - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" - "sigs.k8s.io/karpenter/pkg/scheduling" - "sigs.k8s.io/karpenter/pkg/test" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - apisv1alpha5 "github.com/aws/karpenter/pkg/apis/v1alpha5" -) - -var ctx context.Context - -func TestV1Alpha5(t *testing.T) { - RegisterFailHandler(Fail) - RunSpecs(t, "v1alpha5") - ctx = TestContextWithLogger(t) -} - -var _ = Describe("Provisioner", func() { - var provisioner *v1alpha5.Provisioner - - BeforeEach(func() { - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: &v1alpha1.AWS{ - SubnetSelector: map[string]string{"*": "*"}, - SecurityGroupSelector: map[string]string{"*": "*"}, - }}) - }) - - Context("SetDefaults", func() { - It("should default OS to linux", func() { - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1.LabelOSStable)). - To(Equal(scheduling.NewRequirement(v1.LabelOSStable, v1.NodeSelectorOpIn, string(v1.Linux)))) - }) - It("should not default OS if set", func() { - provisioner.Spec.Requirements = append(provisioner.Spec.Requirements, - v1.NodeSelectorRequirement{Key: v1.LabelOSStable, Operator: v1.NodeSelectorOpDoesNotExist}) - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1.LabelOSStable)). - To(Equal(scheduling.NewRequirement(v1.LabelOSStable, v1.NodeSelectorOpDoesNotExist))) - }) - It("should default architecture to amd64", func() { - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1.LabelArchStable)). - To(Equal(scheduling.NewRequirement(v1.LabelArchStable, v1.NodeSelectorOpIn, v1alpha5.ArchitectureAmd64))) - }) - It("should not default architecture if set", func() { - provisioner.Spec.Requirements = append(provisioner.Spec.Requirements, - v1.NodeSelectorRequirement{Key: v1.LabelArchStable, Operator: v1.NodeSelectorOpDoesNotExist}) - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1.LabelArchStable)). - To(Equal(scheduling.NewRequirement(v1.LabelArchStable, v1.NodeSelectorOpDoesNotExist))) - }) - It("should default capacity-type to on-demand", func() { - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha5.LabelCapacityType)). - To(Equal(scheduling.NewRequirement(v1alpha5.LabelCapacityType, v1.NodeSelectorOpIn, v1alpha1.CapacityTypeOnDemand))) - }) - It("should not default capacity-type if set", func() { - provisioner.Spec.Requirements = append(provisioner.Spec.Requirements, - v1.NodeSelectorRequirement{Key: v1alpha5.LabelCapacityType, Operator: v1.NodeSelectorOpDoesNotExist}) - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha5.LabelCapacityType)). - To(Equal(scheduling.NewRequirement(v1alpha5.LabelCapacityType, v1.NodeSelectorOpDoesNotExist))) - }) - It("should default instance-category, generation to c m r, gen>1", func() { - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceCategory)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceCategory, v1.NodeSelectorOpIn, "c", "m", "r"))) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceGeneration)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceGeneration, v1.NodeSelectorOpGt, "2"))) - }) - It("should not default instance-category, generation if set", func() { - provisioner.Spec.Requirements = append(provisioner.Spec.Requirements, - v1.NodeSelectorRequirement{Key: v1alpha1.LabelInstanceCategory, Operator: v1.NodeSelectorOpExists}) - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceCategory)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceCategory, v1.NodeSelectorOpExists))) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceGeneration)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceGeneration, v1.NodeSelectorOpExists))) - }) - It("should not default instance-category if any instance label is set", func() { - for _, label := range []string{v1.LabelInstanceTypeStable, v1alpha1.LabelInstanceFamily} { - provisioner.Spec.Requirements = []v1.NodeSelectorRequirement{{Key: label, Operator: v1.NodeSelectorOpIn, Values: []string{"test"}}} - SetDefaults(ctx, provisioner) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(label)). - To(Equal(scheduling.NewRequirement(label, v1.NodeSelectorOpIn, "test"))) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceCategory)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceCategory, v1.NodeSelectorOpExists))) - Expect(scheduling.NewNodeSelectorRequirements(provisioner.Spec.Requirements...).Get(v1alpha1.LabelInstanceGeneration)). - To(Equal(scheduling.NewRequirement(v1alpha1.LabelInstanceGeneration, v1.NodeSelectorOpExists))) - } - }) - }) - - Context("Validate", func() { - - It("should validate", func() { - Expect(provisioner.Validate(ctx)).To(Succeed()) - }) - It("should succeed if provider undefined", func() { - provisioner.Spec.Provider = nil - provisioner.Spec.ProviderRef = &v1alpha5.MachineTemplateRef{ - Kind: "AWSNodeTemplate", - Name: "default", - } - Expect(provisioner.Validate(ctx)).To(Succeed()) - }) - - Context("SubnetSelector", func() { - It("should not allow empty string keys or values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - for key, value := range map[string]string{ - "": "value", - "key": "", - } { - provider.SubnetSelector = map[string]string{key: value} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - } - }) - }) - Context("SecurityGroupSelector", func() { - It("should not allow with a custom launch template", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.LaunchTemplateName = aws.String("my-lt") - provider.SecurityGroupSelector = map[string]string{"key": "value"} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow empty string keys or values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - for key, value := range map[string]string{ - "": "value", - "key": "", - } { - provider.SecurityGroupSelector = map[string]string{key: value} - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - } - }) - }) - - Context("Labels", func() { - It("should not allow unrecognized labels with the aws label prefix", func() { - provisioner.Spec.Labels = map[string]string{v1alpha1.LabelDomain + "/" + randomdata.SillyName(): randomdata.SillyName()} - Expect(Validate(ctx, provisioner)).ToNot(Succeed()) - }) - It("should support well known labels", func() { - for _, label := range []string{ - v1alpha1.LabelInstanceHypervisor, - v1alpha1.LabelInstanceFamily, - v1alpha1.LabelInstanceSize, - v1alpha1.LabelInstanceCPU, - v1alpha1.LabelInstanceMemory, - v1alpha1.LabelInstanceGPUName, - v1alpha1.LabelInstanceGPUManufacturer, - v1alpha1.LabelInstanceGPUCount, - v1alpha1.LabelInstanceGPUMemory, - v1alpha1.LabelInstanceAcceleratorName, - v1alpha1.LabelInstanceAcceleratorManufacturer, - v1alpha1.LabelInstanceAcceleratorCount, - } { - provisioner.Spec.Labels = map[string]string{label: randomdata.SillyName()} - Expect(provisioner.Validate(ctx)).To(Succeed()) - } - }) - }) - Context("MetadataOptions", func() { - It("should not allow with a custom launch template", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.LaunchTemplateName = aws.String("my-lt") - provider.MetadataOptions = &v1alpha1.MetadataOptions{} - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should allow missing values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{} - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(provisioner.Validate(ctx)).To(Succeed()) - }) - Context("HTTPEndpoint", func() { - It("should allow enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - for i := range ec2.LaunchTemplateInstanceMetadataEndpointState_Values() { - value := ec2.LaunchTemplateInstanceMetadataEndpointState_Values()[i] - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPEndpoint: &value, - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(provisioner.Validate(ctx)).To(Succeed()) - } - }) - It("should not allow non-enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPEndpoint: aws.String(randomdata.SillyName()), - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - }) - Context("HTTPProtocolIpv6", func() { - It("should allow enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - for i := range ec2.LaunchTemplateInstanceMetadataProtocolIpv6_Values() { - value := ec2.LaunchTemplateInstanceMetadataProtocolIpv6_Values()[i] - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPProtocolIPv6: &value, - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(provisioner.Validate(ctx)).To(Succeed()) - } - }) - It("should not allow non-enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPProtocolIPv6: aws.String(randomdata.SillyName()), - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - }) - Context("HTTPPutResponseHopLimit", func() { - It("should validate inside accepted range", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPPutResponseHopLimit: aws.Int64(int64(randomdata.Number(1, 65))), - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(provisioner.Validate(ctx)).To(Succeed()) - }) - It("should not validate outside accepted range", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{} - // We expect to be able to invalidate any hop limit between - // [math.MinInt64, 1). But, to avoid a panic here, we can't - // exceed math.MaxInt for the difference between bounds of - // the random number range. So we divide the range - // approximately in half and test on both halves. - provider.MetadataOptions.HTTPPutResponseHopLimit = aws.Int64(int64(randomdata.Number(math.MinInt64, math.MinInt64/2))) - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - provider.MetadataOptions.HTTPPutResponseHopLimit = aws.Int64(int64(randomdata.Number(math.MinInt64/2, 1))) - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - - provider.MetadataOptions.HTTPPutResponseHopLimit = aws.Int64(int64(randomdata.Number(65, math.MaxInt64))) - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - }) - Context("HTTPTokens", func() { - It("should allow enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - for _, value := range ec2.LaunchTemplateHttpTokensState_Values() { - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPTokens: aws.String(value), - } - provisioner = test.Provisioner(test.ProvisionerOptions{Provider: provider}) - Expect(provisioner.Validate(ctx)).To(Succeed()) - } - }) - It("should not allow non-enum values", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.MetadataOptions = &v1alpha1.MetadataOptions{ - HTTPTokens: aws.String(randomdata.SillyName()), - } - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - }) - Context("BlockDeviceMappings", func() { - It("should not allow with a custom launch template", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.LaunchTemplateName = aws.String("my-lt") - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{ - VolumeSize: resource.NewScaledQuantity(1, resource.Giga), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should validate minimal device mapping", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{ - VolumeSize: resource.NewScaledQuantity(1, resource.Giga), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should validate ebs device mapping with snapshotID only", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{ - SnapshotID: aws.String("snap-0123456789"), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow volume size below minimum", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{ - VolumeSize: resource.NewScaledQuantity(100, resource.Mega), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow volume size above max", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{ - VolumeSize: resource.NewScaledQuantity(65, resource.Tera), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow nil device name", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - EBS: &v1alpha1.BlockDevice{ - VolumeSize: resource.NewScaledQuantity(65, resource.Tera), - }, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow nil volume size", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - EBS: &v1alpha1.BlockDevice{}, - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - It("should not allow empty ebs block", func() { - provider, err := v1alpha1.DeserializeProvider(provisioner.Spec.Provider.Raw) - Expect(err).ToNot(HaveOccurred()) - provider.BlockDeviceMappings = []*v1alpha1.BlockDeviceMapping{{ - DeviceName: aws.String("/dev/xvda"), - }} - Expect(Validate(ctx, test.Provisioner(test.ProvisionerOptions{Provider: provider}))).ToNot(Succeed()) - }) - }) - }) - }) -}) - -func SetDefaults(ctx context.Context, provisioner *v1alpha5.Provisioner) { - prov := apisv1alpha5.Provisioner(*provisioner) - prov.SetDefaults(ctx) - *provisioner = v1alpha5.Provisioner(prov) -} - -func Validate(ctx context.Context, provisioner *v1alpha5.Provisioner) error { - return multierr.Combine( - lo.ToPtr(apisv1alpha5.Provisioner(*provisioner)).Validate(ctx), - provisioner.Validate(ctx), - ) -} diff --git a/pkg/apis/v1alpha5/zz_generated.deepcopy.go b/pkg/apis/v1alpha5/zz_generated.deepcopy.go deleted file mode 100644 index 722b9872175f..000000000000 --- a/pkg/apis/v1alpha5/zz_generated.deepcopy.go +++ /dev/null @@ -1,50 +0,0 @@ -//go:build !ignore_autogenerated - -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -// Code generated by controller-gen. DO NOT EDIT. - -package v1alpha5 - -import ( - runtime "k8s.io/apimachinery/pkg/runtime" -) - -// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. -func (in *Provisioner) DeepCopyInto(out *Provisioner) { - *out = *in - out.TypeMeta = in.TypeMeta - in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) - in.Spec.DeepCopyInto(&out.Spec) - in.Status.DeepCopyInto(&out.Status) -} - -// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new Provisioner. -func (in *Provisioner) DeepCopy() *Provisioner { - if in == nil { - return nil - } - out := new(Provisioner) - in.DeepCopyInto(out) - return out -} - -// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. -func (in *Provisioner) DeepCopyObject() runtime.Object { - if c := in.DeepCopy(); c != nil { - return c - } - return nil -} diff --git a/pkg/apis/v1beta1/ec2nodeclass.go b/pkg/apis/v1beta1/ec2nodeclass.go index 3a0238ce50c9..054bca2d867d 100644 --- a/pkg/apis/v1beta1/ec2nodeclass.go +++ b/pkg/apis/v1beta1/ec2nodeclass.go @@ -109,27 +109,6 @@ type EC2NodeClassSpec struct { // https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_CreateFleet.html // +optional Context *string `json:"context,omitempty"` - // TODO @joinnis: Remove this field when v1alpha5 is unsupported in a future version of Karpenter - // LaunchTemplateName for the node. If not specified, a launch template will be generated. - // NOTE: This field is for specifying a custom launch template and is exposed in the Spec - // as `launchTemplate` for backwards compatibility. - // +optional - LaunchTemplateName *string `json:"-" hash:"ignore"` - // TODO @joinnis: Remove this field when v1alpha5 is unsupported in a future version of Karpenter - // OriginalSubnetSelector is the original subnet selector that was used by the v1alpha5 representation of this API. - // DO NOT USE THIS VALUE when performing business logic in code - // +optional - OriginalSubnetSelector map[string]string `json:"-" hash:"ignore"` - // TODO @joinnis: Remove this field when v1alpha5 is unsupported in a future version of Karpenter - // OriginalSecurityGroupSelector is the original security group selector that was used by the v1alpha5 representation of this API. - // DO NOT USE THIS VALUE when performing business logic in code - // +optional - OriginalSecurityGroupSelector map[string]string `json:"-" hash:"ignore"` - // TODO @joinnis: Remove this field when v1alpha5 is unsupported in a future version of Karpenter - // OriginalAMISelector is the original ami selector that was used by the v1alpha5 representation of this API. - // DO NOT USE THIS VALUE when performing business logic in code - // +optional - OriginalAMISelector map[string]string `json:"-" hash:"ignore"` } // SubnetSelectorTerm defines selection logic for a subnet used by Karpenter to launch nodes. diff --git a/pkg/apis/v1beta1/ec2nodeclass_hash_test.go b/pkg/apis/v1beta1/ec2nodeclass_hash_test.go index 668fa47a7d87..1e7eb9ae0342 100644 --- a/pkg/apis/v1beta1/ec2nodeclass_hash_test.go +++ b/pkg/apis/v1beta1/ec2nodeclass_hash_test.go @@ -22,7 +22,6 @@ import ( . "github.com/onsi/ginkgo/v2" . "github.com/onsi/gomega" - "github.com/aws/karpenter/pkg/apis/v1alpha1" "github.com/aws/karpenter/pkg/apis/v1beta1" "github.com/aws/karpenter/pkg/test" ) @@ -33,7 +32,7 @@ var _ = Describe("Hash", func() { BeforeEach(func() { nodeClass = test.EC2NodeClass(v1beta1.EC2NodeClass{ Spec: v1beta1.EC2NodeClassSpec{ - AMIFamily: aws.String(v1alpha1.AMIFamilyAL2), + AMIFamily: aws.String(v1beta1.AMIFamilyAL2), Context: aws.String("context-1"), Role: "role-1", Tags: map[string]string{ @@ -70,7 +69,7 @@ var _ = Describe("Hash", func() { Entry("BlockDeviceMappings Drift", "436753305915039702", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{BlockDeviceMappings: []*v1beta1.BlockDeviceMapping{{DeviceName: aws.String("map-device-test-3")}}}}), Entry("Context Drift", "3729470655588343019", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{Context: aws.String("context-2")}}), Entry("DetailedMonitoring Drift", "17892305444040067573", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{DetailedMonitoring: aws.Bool(true)}}), - Entry("AMIFamily Drift", "9493798894326942407", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{AMIFamily: aws.String(v1alpha1.AMIFamilyBottlerocket)}}), + Entry("AMIFamily Drift", "9493798894326942407", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{AMIFamily: aws.String(v1beta1.AMIFamilyBottlerocket)}}), Entry("Reorder Tags", staticHash, v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{Tags: map[string]string{"keyTag-2": "valueTag-2", "keyTag-1": "valueTag-1"}}}), Entry("Reorder BlockDeviceMapping", staticHash, v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{BlockDeviceMappings: []*v1beta1.BlockDeviceMapping{{DeviceName: aws.String("map-device-2")}, {DeviceName: aws.String("map-device-1")}}}}), @@ -96,7 +95,7 @@ var _ = Describe("Hash", func() { Entry("BlockDeviceMappings Drift", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{BlockDeviceMappings: []*v1beta1.BlockDeviceMapping{{DeviceName: aws.String("map-device-test-3")}}}}), Entry("Context Drift", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{Context: aws.String("context-2")}}), Entry("DetailedMonitoring Drift", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{DetailedMonitoring: aws.Bool(true)}}), - Entry("AMIFamily Drift", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{AMIFamily: aws.String(v1alpha1.AMIFamilyBottlerocket)}}), + Entry("AMIFamily Drift", v1beta1.EC2NodeClass{Spec: v1beta1.EC2NodeClassSpec{AMIFamily: aws.String(v1beta1.AMIFamilyBottlerocket)}}), ) It("should change hash when instanceProfile is updated", func() { nodeClass.Spec.Role = "" @@ -137,7 +136,7 @@ var _ = Describe("Hash", func() { updatedHash := nodeClass.Hash() Expect(hash).To(Equal(updatedHash)) }) - It("should expect two EC2NodeClasses with the same spec to have the same provisioner hash", func() { + It("should expect two EC2NodeClasses with the same spec to have the same hash", func() { otherNodeClass := test.EC2NodeClass(v1beta1.EC2NodeClass{ Spec: nodeClass.Spec, }) diff --git a/pkg/apis/v1beta1/ec2nodeclass_validation_cel_test.go b/pkg/apis/v1beta1/ec2nodeclass_validation_cel_test.go index 9b952aa3d334..214b4ae3b155 100644 --- a/pkg/apis/v1beta1/ec2nodeclass_validation_cel_test.go +++ b/pkg/apis/v1beta1/ec2nodeclass_validation_cel_test.go @@ -20,8 +20,8 @@ import ( . "github.com/onsi/gomega" "github.com/samber/lo" "k8s.io/apimachinery/pkg/api/resource" + corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" - "github.com/aws/karpenter/pkg/apis/v1alpha1" "github.com/aws/karpenter/pkg/apis/v1beta1" "github.com/aws/karpenter/pkg/test" ) @@ -71,7 +71,7 @@ var _ = Describe("CEL/Validation", func() { }) It("should fail if tags contain a restricted domain key", func() { nc.Spec.Tags = map[string]string{ - "karpenter.sh/provisioner-name": "value", + corev1beta1.NodePoolLabelKey: "value", } Expect(env.Client.Create(ctx, nc)).To(Not(Succeed())) nc.Spec.Tags = map[string]string{ @@ -79,7 +79,7 @@ var _ = Describe("CEL/Validation", func() { } Expect(env.Client.Create(ctx, nc)).To(Not(Succeed())) nc.Spec.Tags = map[string]string{ - "karpenter.sh/managed-by": "test", + corev1beta1.ManagedByAnnotationKey: "test", } Expect(env.Client.Create(ctx, nc)).To(Not(Succeed())) }) @@ -444,7 +444,7 @@ var _ = Describe("CEL/Validation", func() { Expect(env.Client.Create(ctx, nc)).ToNot(Succeed()) }) It("should fail when AMIFamily is Custom and not AMISelectorTerms", func() { - nc.Spec.AMIFamily = &v1alpha1.AMIFamilyCustom + nc.Spec.AMIFamily = &v1beta1.AMIFamilyCustom Expect(env.Client.Create(ctx, nc)).ToNot(Succeed()) }) }) diff --git a/pkg/apis/v1beta1/ec2nodeclass_validation_webhook_test.go b/pkg/apis/v1beta1/ec2nodeclass_validation_webhook_test.go index cfae6cb576ce..83f90b530dc6 100644 --- a/pkg/apis/v1beta1/ec2nodeclass_validation_webhook_test.go +++ b/pkg/apis/v1beta1/ec2nodeclass_validation_webhook_test.go @@ -69,7 +69,7 @@ var _ = Describe("Webhook/Validation", func() { }) It("should succeed by validating that regex is properly escaped", func() { nc.Spec.Tags = map[string]string{ - "karpenterzsh/provisioner-name": "value", + "karpenterzsh/nodepool": "value", } Expect(nc.Validate(ctx)).To(Succeed()) nc.Spec.Tags = map[string]string{ diff --git a/pkg/apis/v1beta1/zz_generated.deepcopy.go b/pkg/apis/v1beta1/zz_generated.deepcopy.go index 68d395d4b246..ab75e45097bd 100644 --- a/pkg/apis/v1beta1/zz_generated.deepcopy.go +++ b/pkg/apis/v1beta1/zz_generated.deepcopy.go @@ -278,32 +278,6 @@ func (in *EC2NodeClassSpec) DeepCopyInto(out *EC2NodeClassSpec) { *out = new(string) **out = **in } - if in.LaunchTemplateName != nil { - in, out := &in.LaunchTemplateName, &out.LaunchTemplateName - *out = new(string) - **out = **in - } - if in.OriginalSubnetSelector != nil { - in, out := &in.OriginalSubnetSelector, &out.OriginalSubnetSelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - if in.OriginalSecurityGroupSelector != nil { - in, out := &in.OriginalSecurityGroupSelector, &out.OriginalSecurityGroupSelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } - if in.OriginalAMISelector != nil { - in, out := &in.OriginalAMISelector, &out.OriginalAMISelector - *out = make(map[string]string, len(*in)) - for key, val := range *in { - (*out)[key] = val - } - } } // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new EC2NodeClassSpec. diff --git a/pkg/cloudprovider/cloudprovider.go b/pkg/cloudprovider/cloudprovider.go index 701cf8a132cc..863fd6f5b8d7 100644 --- a/pkg/cloudprovider/cloudprovider.go +++ b/pkg/cloudprovider/cloudprovider.go @@ -27,15 +27,12 @@ import ( corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" "sigs.k8s.io/karpenter/pkg/events" + "sigs.k8s.io/karpenter/pkg/scheduling" "sigs.k8s.io/karpenter/pkg/utils/functional" - nodepoolutil "sigs.k8s.io/karpenter/pkg/utils/nodepool" + "sigs.k8s.io/karpenter/pkg/utils/resources" "github.com/aws/karpenter/pkg/apis/v1beta1" "github.com/aws/karpenter/pkg/utils" - nodeclassutil "github.com/aws/karpenter/pkg/utils/nodeclass" - - "sigs.k8s.io/karpenter/pkg/scheduling" - "sigs.k8s.io/karpenter/pkg/utils/resources" "github.com/samber/lo" v1 "k8s.io/api/core/v1" @@ -51,7 +48,6 @@ import ( "github.com/aws/karpenter/pkg/providers/securitygroup" "github.com/aws/karpenter/pkg/providers/subnet" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" "sigs.k8s.io/karpenter/pkg/cloudprovider" ) @@ -105,21 +101,10 @@ func (c *CloudProvider) Create(ctx context.Context, nodeClaim *corev1beta1.NodeC return i.Name == instance.Type }) nc := c.instanceToNodeClaim(instance, instanceType) - nc.Annotations = lo.Assign(nc.Annotations, nodeclassutil.HashAnnotation(nodeClass)) + nc.Annotations = lo.Assign(nc.Annotations, map[string]string{v1beta1.AnnotationEC2NodeClassHash: nodeClass.Hash()}) return nc, nil } -// Link adds a tag to the cloudprovider NodeClaim to tell the cloudprovider that it's now owned by a NodeClaim -func (c *CloudProvider) Link(ctx context.Context, nodeClaim *corev1beta1.NodeClaim) error { - ctx = logging.WithLogger(ctx, logging.FromContext(ctx).With("nodeclaim", nodeClaim.Name)) - id, err := utils.ParseInstanceID(nodeClaim.Status.ProviderID) - if err != nil { - return fmt.Errorf("getting instance ID, %w", err) - } - ctx = logging.WithLogger(ctx, logging.FromContext(ctx).With("id", id)) - return c.instanceProvider.Link(ctx, id, nodeClaim.Labels[v1alpha5.ProvisionerNameLabelKey]) -} - func (c *CloudProvider) List(ctx context.Context) ([]*corev1beta1.NodeClaim, error) { instances, err := c.instanceProvider.List(ctx) if err != nil { @@ -183,8 +168,7 @@ func (c *CloudProvider) GetInstanceTypes(ctx context.Context, nodePool *corev1be func (c *CloudProvider) Delete(ctx context.Context, nodeClaim *corev1beta1.NodeClaim) error { ctx = logging.WithLogger(ctx, logging.FromContext(ctx).With("nodeclaim", nodeClaim.Name)) - providerID := lo.Ternary(nodeClaim.Status.ProviderID != "", nodeClaim.Status.ProviderID, nodeClaim.Annotations[v1alpha5.MachineLinkedAnnotationKey]) - id, err := utils.ParseInstanceID(providerID) + id, err := utils.ParseInstanceID(nodeClaim.Status.ProviderID) if err != nil { return fmt.Errorf("getting instance ID, %w", err) } @@ -262,15 +246,15 @@ func (c *CloudProvider) resolveInstanceTypes(ctx context.Context, nodeClaim *cor } func (c *CloudProvider) resolveInstanceTypeFromInstance(ctx context.Context, instance *instance.Instance) (*cloudprovider.InstanceType, error) { - provisioner, err := c.resolveNodePoolFromInstance(ctx, instance) + nodePool, err := c.resolveNodePoolFromInstance(ctx, instance) if err != nil { - // If we can't resolve the provisioner, we fall back to not getting instance type info - return nil, client.IgnoreNotFound(fmt.Errorf("resolving provisioner, %w", err)) + // If we can't resolve the NodePool, we fall back to not getting instance type info + return nil, client.IgnoreNotFound(fmt.Errorf("resolving nodepool, %w", err)) } - instanceTypes, err := c.GetInstanceTypes(ctx, provisioner) + instanceTypes, err := c.GetInstanceTypes(ctx, nodePool) if err != nil { - // If we can't resolve the provisioner, we fall back to not getting instance type info - return nil, client.IgnoreNotFound(fmt.Errorf("resolving node template, %w", err)) + // If we can't resolve the NodePool, we fall back to not getting instance type info + return nil, client.IgnoreNotFound(fmt.Errorf("resolving nodeclass, %w", err)) } instanceType, _ := lo.Find(instanceTypes, func(i *cloudprovider.InstanceType) bool { return i.Name == instance.Type @@ -279,25 +263,14 @@ func (c *CloudProvider) resolveInstanceTypeFromInstance(ctx context.Context, ins } func (c *CloudProvider) resolveNodePoolFromInstance(ctx context.Context, instance *instance.Instance) (*corev1beta1.NodePool, error) { - provisionerName := instance.Tags[v1alpha5.ProvisionerNameLabelKey] - nodePoolName := instance.Tags[corev1beta1.NodePoolLabelKey] - - switch { - case nodePoolName != "": + if nodePoolName, ok := instance.Tags[corev1beta1.NodePoolLabelKey]; ok { nodePool := &corev1beta1.NodePool{} if err := c.kubeClient.Get(ctx, types.NamespacedName{Name: nodePoolName}, nodePool); err != nil { return nil, err } return nodePool, nil - case provisionerName != "": - provisioner := &v1alpha5.Provisioner{} - if err := c.kubeClient.Get(ctx, types.NamespacedName{Name: provisionerName}, provisioner); err != nil { - return nil, err - } - return nodepoolutil.New(provisioner), nil - default: - return nil, errors.NewNotFound(schema.GroupResource{Group: corev1beta1.Group, Resource: "NodePool"}, "") } + return nil, errors.NewNotFound(schema.GroupResource{Group: corev1beta1.Group, Resource: "NodePool"}, "") } func (c *CloudProvider) instanceToNodeClaim(i *instance.Instance, instanceType *cloudprovider.InstanceType) *corev1beta1.NodeClaim { diff --git a/pkg/cloudprovider/drift.go b/pkg/cloudprovider/drift.go index 57d19d314032..94cd8703c9d3 100644 --- a/pkg/cloudprovider/drift.go +++ b/pkg/cloudprovider/drift.go @@ -35,7 +35,6 @@ const ( AMIDrift cloudprovider.DriftReason = "AMIDrift" SubnetDrift cloudprovider.DriftReason = "SubnetDrift" SecurityGroupDrift cloudprovider.DriftReason = "SecurityGroupDrift" - NodeTemplateDrift cloudprovider.DriftReason = "NodeTemplateDrift" NodeClassDrift cloudprovider.DriftReason = "NodeClassDrift" ) @@ -78,9 +77,6 @@ func (c *CloudProvider) isAMIDrifted(ctx context.Context, nodeClaim *corev1beta1 if !found { return "", fmt.Errorf(`finding node instance type "%s"`, nodeClaim.Labels[v1.LabelInstanceTypeStable]) } - if nodeClass.Spec.LaunchTemplateName != nil { - return "", nil - } amis, err := c.amiProvider.Get(ctx, nodeClass, &amifamily.Options{}) if err != nil { return "", fmt.Errorf("getting amis, %w", err) @@ -114,11 +110,6 @@ func (c *CloudProvider) isSubnetDrifted(instance *instance.Instance, nodeClass * // Checks if the security groups are drifted, by comparing the EC2NodeClass.Status.SecurityGroups // to the ec2 instance security groups func (c *CloudProvider) areSecurityGroupsDrifted(ec2Instance *instance.Instance, nodeClass *v1beta1.EC2NodeClass) (cloudprovider.DriftReason, error) { - // nodeClass.Spec.SecurityGroupSelector can be nil if the user is using a launchTemplateName to define SecurityGroups - // Karpenter will not drift on changes to securitygroup in the launchTemplateName - if nodeClass.Spec.LaunchTemplateName != nil { - return "", nil - } securityGroupIds := sets.New(lo.Map(nodeClass.Status.SecurityGroups, func(sg v1beta1.SecurityGroup, _ int) string { return sg.ID })...) if len(securityGroupIds) == 0 { return "", fmt.Errorf("no security groups exist in status") diff --git a/pkg/controllers/interruption/controller.go b/pkg/controllers/interruption/controller.go index c5a75425f178..2edec6b35dbb 100644 --- a/pkg/controllers/interruption/controller.go +++ b/pkg/controllers/interruption/controller.go @@ -33,7 +33,6 @@ import ( "sigs.k8s.io/karpenter/pkg/apis/v1beta1" "sigs.k8s.io/karpenter/pkg/utils/pretty" - "github.com/aws/karpenter/pkg/apis/v1alpha1" "github.com/aws/karpenter/pkg/cache" interruptionevents "github.com/aws/karpenter/pkg/controllers/interruption/events" "github.com/aws/karpenter/pkg/controllers/interruption/messages" @@ -142,7 +141,7 @@ func (c *Controller) parseMessage(raw *sqsapi.Message) (messages.Message, error) return msg, nil } -// handleMessage takes an action against every node involved in the message that is owned by a Provisioner +// handleMessage takes an action against every node involved in the message that is owned by a NodePool func (c *Controller) handleMessage(ctx context.Context, nodeClaimInstanceIDMap map[string]*v1beta1.NodeClaim, nodeInstanceIDMap map[string]*v1.Node, msg messages.Message) (err error) { @@ -195,7 +194,7 @@ func (c *Controller) handleNodeClaim(ctx context.Context, msg messages.Message, zone := nodeClaim.Labels[v1.LabelTopologyZone] instanceType := nodeClaim.Labels[v1.LabelInstanceTypeStable] if zone != "" && instanceType != "" { - c.unavailableOfferingsCache.MarkUnavailable(ctx, string(msg.Kind()), instanceType, zone, v1alpha1.CapacityTypeSpot) + c.unavailableOfferingsCache.MarkUnavailable(ctx, string(msg.Kind()), instanceType, zone, v1beta1.CapacityTypeSpot) } } if action != NoAction { diff --git a/pkg/controllers/interruption/interruption_benchmark_test.go b/pkg/controllers/interruption/interruption_benchmark_test.go index 650eb628b88b..54730bb336c9 100644 --- a/pkg/controllers/interruption/interruption_benchmark_test.go +++ b/pkg/controllers/interruption/interruption_benchmark_test.go @@ -41,6 +41,9 @@ import ( "knative.dev/pkg/logging" controllerruntime "sigs.k8s.io/controller-runtime" "sigs.k8s.io/controller-runtime/pkg/client" + "sigs.k8s.io/karpenter/pkg/apis/v1beta1" + + "sigs.k8s.io/karpenter/pkg/operator/scheme" awscache "github.com/aws/karpenter/pkg/cache" "github.com/aws/karpenter/pkg/controllers/interruption" @@ -49,9 +52,7 @@ import ( "github.com/aws/karpenter/pkg/operator/options" "github.com/aws/karpenter/pkg/providers/sqs" "github.com/aws/karpenter/pkg/test" - "sigs.k8s.io/karpenter/pkg/operator/scheme" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" coreoptions "sigs.k8s.io/karpenter/pkg/operator/options" coretest "sigs.k8s.io/karpenter/pkg/test" ) @@ -274,7 +275,7 @@ func makeScheduledChangeMessagesAndNodes(count int) ([]interface{}, []*v1.Node) nodes = append(nodes, coretest.Node(coretest.NodeOptions{ ObjectMeta: metav1.ObjectMeta{ Labels: map[string]string{ - v1alpha5.ProvisionerNameLabelKey: "default", + v1beta1.NodePoolLabelKey: "default", }, }, ProviderID: fake.ProviderID(instanceID), @@ -293,7 +294,7 @@ func makeStateChangeMessagesAndNodes(count int, states []string) ([]interface{}, nodes = append(nodes, coretest.Node(coretest.NodeOptions{ ObjectMeta: metav1.ObjectMeta{ Labels: map[string]string{ - v1alpha5.ProvisionerNameLabelKey: "default", + v1beta1.NodePoolLabelKey: "default", }, }, ProviderID: fake.ProviderID(instanceID), @@ -311,7 +312,7 @@ func makeSpotInterruptionMessagesAndNodes(count int) ([]interface{}, []*v1.Node) nodes = append(nodes, coretest.Node(coretest.NodeOptions{ ObjectMeta: metav1.ObjectMeta{ Labels: map[string]string{ - v1alpha5.ProvisionerNameLabelKey: "default", + v1beta1.NodePoolLabelKey: "default", }, }, ProviderID: fake.ProviderID(instanceID), diff --git a/pkg/controllers/nodeclass/controller.go b/pkg/controllers/nodeclass/controller.go index 13395896e9ee..4f18c7d2f107 100644 --- a/pkg/controllers/nodeclass/controller.go +++ b/pkg/controllers/nodeclass/controller.go @@ -47,7 +47,6 @@ import ( "github.com/aws/karpenter/pkg/providers/instanceprofile" "github.com/aws/karpenter/pkg/providers/securitygroup" "github.com/aws/karpenter/pkg/providers/subnet" - nodeclassutil "github.com/aws/karpenter/pkg/utils/nodeclass" ) type Controller struct { @@ -74,7 +73,7 @@ func NewController(kubeClient client.Client, recorder events.Recorder, subnetPro func (c *Controller) Reconcile(ctx context.Context, nodeClass *v1beta1.EC2NodeClass) (reconcile.Result, error) { stored := nodeClass.DeepCopy() controllerutil.AddFinalizer(nodeClass, v1beta1.TerminationFinalizer) - nodeClass.Annotations = lo.Assign(nodeClass.Annotations, nodeclassutil.HashAnnotation(nodeClass)) + nodeClass.Annotations = lo.Assign(nodeClass.Annotations, map[string]string{v1beta1.AnnotationEC2NodeClassHash: nodeClass.Hash()}) err := multierr.Combine( c.resolveSubnets(ctx, nodeClass), c.resolveSecurityGroups(ctx, nodeClass), @@ -83,10 +82,10 @@ func (c *Controller) Reconcile(ctx context.Context, nodeClass *v1beta1.EC2NodeCl ) if !equality.Semantic.DeepEqual(stored, nodeClass) { statusCopy := nodeClass.DeepCopy() - if patchErr := nodeclassutil.Patch(ctx, c.kubeClient, stored, nodeClass); patchErr != nil { + if patchErr := c.kubeClient.Patch(ctx, nodeClass, client.MergeFrom(stored)); err != nil { err = multierr.Append(err, client.IgnoreNotFound(patchErr)) } - if patchErr := nodeclassutil.PatchStatus(ctx, c.kubeClient, stored, statusCopy); patchErr != nil { + if patchErr := c.kubeClient.Status().Patch(ctx, statusCopy, client.MergeFrom(stored)); err != nil { err = multierr.Append(err, client.IgnoreNotFound(patchErr)) } } @@ -116,7 +115,7 @@ func (c *Controller) Finalize(ctx context.Context, nodeClass *v1beta1.EC2NodeCla } controllerutil.RemoveFinalizer(nodeClass, v1beta1.TerminationFinalizer) if !equality.Semantic.DeepEqual(stored, nodeClass) { - if err := nodeclassutil.Patch(ctx, c.kubeClient, stored, nodeClass); err != nil { + if err := c.kubeClient.Patch(ctx, nodeClass, client.MergeFrom(stored)); err != nil { return reconcile.Result{}, client.IgnoreNotFound(fmt.Errorf("removing termination finalizer, %w", err)) } } diff --git a/pkg/fake/ec2api.go b/pkg/fake/ec2api.go index f7ec604b5d58..8b7ca6dbb113 100644 --- a/pkg/fake/ec2api.go +++ b/pkg/fake/ec2api.go @@ -30,8 +30,7 @@ import ( "github.com/aws/aws-sdk-go/service/ec2/ec2iface" "github.com/samber/lo" "k8s.io/apimachinery/pkg/util/sets" - - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" + corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" "sigs.k8s.io/karpenter/pkg/test" "sigs.k8s.io/karpenter/pkg/utils/atomic" @@ -118,7 +117,7 @@ func (e *EC2API) CreateFleetWithContext(_ context.Context, input *ec2.CreateFlee var skippedPools []CapacityPool var spotInstanceRequestID *string - if aws.StringValue(input.TargetCapacitySpecification.DefaultTargetCapacityType) == v1alpha5.CapacityTypeSpot { + if aws.StringValue(input.TargetCapacitySpecification.DefaultTargetCapacityType) == corev1beta1.CapacityTypeSpot { spotInstanceRequestID = aws.String(test.RandomName()) } diff --git a/pkg/operator/operator.go b/pkg/operator/operator.go index cee1cf297ba9..63e580a73196 100644 --- a/pkg/operator/operator.go +++ b/pkg/operator/operator.go @@ -48,7 +48,6 @@ import ( "knative.dev/pkg/logging" "knative.dev/pkg/ptr" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" "sigs.k8s.io/karpenter/pkg/operator" "sigs.k8s.io/karpenter/pkg/operator/scheme" @@ -69,7 +68,6 @@ import ( func init() { lo.Must0(apis.AddToScheme(scheme.Scheme)) - v1alpha5.NormalizedLabels = lo.Assign(v1alpha5.NormalizedLabels, map[string]string{"topology.ebs.csi.aws.com/zone": corev1.LabelTopologyZone}) corev1beta1.NormalizedLabels = lo.Assign(corev1beta1.NormalizedLabels, map[string]string{"topology.ebs.csi.aws.com/zone": corev1.LabelTopologyZone}) } diff --git a/pkg/providers/amifamily/bootstrap/bottlerocket.go b/pkg/providers/amifamily/bootstrap/bottlerocket.go index 3e94e2d7b721..c12b39f9ce04 100644 --- a/pkg/providers/amifamily/bootstrap/bottlerocket.go +++ b/pkg/providers/amifamily/bootstrap/bottlerocket.go @@ -40,7 +40,7 @@ func (b Bottlerocket) Script() (string, error) { return "", fmt.Errorf("invalid UserData %w", err) } // Karpenter will overwrite settings present inside custom UserData - // based on other fields specified in the provisioner + // based on other fields specified in the NodePool s.Settings.Kubernetes.ClusterName = &b.ClusterName s.Settings.Kubernetes.APIServer = &b.ClusterEndpoint s.Settings.Kubernetes.ClusterCertificate = b.CABundle diff --git a/pkg/providers/amifamily/windows.go b/pkg/providers/amifamily/windows.go index 8b4ddb41d550..dcfba85b2796 100644 --- a/pkg/providers/amifamily/windows.go +++ b/pkg/providers/amifamily/windows.go @@ -32,8 +32,6 @@ import ( v1 "k8s.io/api/core/v1" "sigs.k8s.io/karpenter/pkg/cloudprovider" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" ) type Windows struct { @@ -46,7 +44,7 @@ type Windows struct { func (w Windows) DefaultAMIs(version string) []DefaultAMIOutput { return []DefaultAMIOutput{ { - Query: fmt.Sprintf("/aws/service/ami-windows-latest/Windows_Server-%s-English-%s-EKS_Optimized-%s/image_id", w.Version, v1alpha1.WindowsCore, version), + Query: fmt.Sprintf("/aws/service/ami-windows-latest/Windows_Server-%s-English-%s-EKS_Optimized-%s/image_id", w.Version, v1beta1.WindowsCore, version), Requirements: scheduling.NewRequirements( scheduling.NewRequirement(v1.LabelArchStable, v1.NodeSelectorOpIn, corev1beta1.ArchitectureAmd64), scheduling.NewRequirement(v1.LabelOSStable, v1.NodeSelectorOpIn, string(v1.Windows)), diff --git a/pkg/providers/instance/instance.go b/pkg/providers/instance/instance.go index 47d4b6fba0b0..fa933a3ccbae 100644 --- a/pkg/providers/instance/instance.go +++ b/pkg/providers/instance/instance.go @@ -45,7 +45,6 @@ import ( "github.com/aws/karpenter/pkg/providers/subnet" "github.com/aws/karpenter/pkg/utils" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" "sigs.k8s.io/karpenter/pkg/cloudprovider" "sigs.k8s.io/karpenter/pkg/scheduling" ) @@ -104,16 +103,6 @@ func (p *Provider) Create(ctx context.Context, nodeClass *v1beta1.EC2NodeClass, return NewInstanceFromFleet(fleetInstance, tags, efaEnabled), nil } -func (p *Provider) Link(ctx context.Context, id, provisionerName string) error { - if err := p.CreateTags(ctx, id, map[string]string{ - v1alpha5.MachineManagedByAnnotationKey: options.FromContext(ctx).ClusterName, - v1alpha5.ProvisionerNameLabelKey: provisionerName, - }); err != nil { - return fmt.Errorf("linking tags, %w", err) - } - return nil -} - func (p *Provider) Get(ctx context.Context, id string) (*Instance, error) { out, err := p.ec2Batcher.DescribeInstances(ctx, &ec2.DescribeInstancesInput{ InstanceIds: aws.StringSlice([]string{id}), @@ -141,7 +130,7 @@ func (p *Provider) List(ctx context.Context) ([]*Instance, error) { Filters: []*ec2.Filter{ { Name: aws.String("tag-key"), - Values: aws.StringSlice([]string{v1alpha5.ProvisionerNameLabelKey, corev1beta1.NodePoolLabelKey}), + Values: aws.StringSlice([]string{corev1beta1.NodePoolLabelKey}), }, { Name: aws.String("tag-key"), @@ -411,7 +400,7 @@ func (p *Provider) filterInstanceTypes(nodeClaim *corev1beta1.NodeClaim, instanc return instanceTypes } -// isMixedCapacityLaunch returns true if provisioners and available offerings could potentially allow either a spot or +// isMixedCapacityLaunch returns true if nodepools and available offerings could potentially allow either a spot or // and on-demand node to launch func (p *Provider) isMixedCapacityLaunch(nodeClaim *corev1beta1.NodeClaim, instanceTypes []*cloudprovider.InstanceType) bool { requirements := scheduling.NewNodeSelectorRequirements(nodeClaim.Spec.Requirements...) diff --git a/pkg/providers/instance/suite_test.go b/pkg/providers/instance/suite_test.go index f18e45f1e0f2..c72e2610152b 100644 --- a/pkg/providers/instance/suite_test.go +++ b/pkg/providers/instance/suite_test.go @@ -30,7 +30,6 @@ import ( "k8s.io/client-go/tools/record" . "knative.dev/pkg/logging/testing" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" corecloudprovider "sigs.k8s.io/karpenter/pkg/cloudprovider" "sigs.k8s.io/karpenter/pkg/events" @@ -111,10 +110,10 @@ var _ = Describe("InstanceProvider", func() { It("should return an ICE error when all attempted instance types return an ICE error", func() { ExpectApplied(ctx, env.Client, nodeClaim, nodePool, nodeClass) awsEnv.EC2API.InsufficientCapacityPools.Set([]fake.CapacityPool{ - {CapacityType: v1alpha5.CapacityTypeOnDemand, InstanceType: "m5.xlarge", Zone: "test-zone-1a"}, - {CapacityType: v1alpha5.CapacityTypeOnDemand, InstanceType: "m5.xlarge", Zone: "test-zone-1b"}, - {CapacityType: v1alpha5.CapacityTypeSpot, InstanceType: "m5.xlarge", Zone: "test-zone-1a"}, - {CapacityType: v1alpha5.CapacityTypeSpot, InstanceType: "m5.xlarge", Zone: "test-zone-1b"}, + {CapacityType: corev1beta1.CapacityTypeOnDemand, InstanceType: "m5.xlarge", Zone: "test-zone-1a"}, + {CapacityType: corev1beta1.CapacityTypeOnDemand, InstanceType: "m5.xlarge", Zone: "test-zone-1b"}, + {CapacityType: corev1beta1.CapacityTypeSpot, InstanceType: "m5.xlarge", Zone: "test-zone-1a"}, + {CapacityType: corev1beta1.CapacityTypeSpot, InstanceType: "m5.xlarge", Zone: "test-zone-1b"}, }) instanceTypes, err := cloudProvider.GetInstanceTypes(ctx, nodePool) Expect(err).ToNot(HaveOccurred()) diff --git a/pkg/providers/instancetype/instancetype.go b/pkg/providers/instancetype/instancetype.go index 968b7790d399..750f40cca3b7 100644 --- a/pkg/providers/instancetype/instancetype.go +++ b/pkg/providers/instancetype/instancetype.go @@ -60,7 +60,7 @@ type Provider struct { // Has one cache entry for all the zones for each subnet selector (key: InstanceTypesZonesCacheKeyPrefix:) // Values cached *before* considering insufficient capacity errors from the unavailableOfferings cache. // Fully initialized Instance Types are also cached based on the set of all instance types, zones, unavailableOfferings cache, - // node template, and kubelet configuration from the provisioner + // EC2NodeClass, and kubelet configuration from the NodePool mu sync.Mutex cache *cache.Cache diff --git a/pkg/providers/instancetype/suite_test.go b/pkg/providers/instancetype/suite_test.go index 7a83ccc53ee4..f6ca44bf43f1 100644 --- a/pkg/providers/instancetype/suite_test.go +++ b/pkg/providers/instancetype/suite_test.go @@ -37,7 +37,6 @@ import ( . "knative.dev/pkg/logging/testing" "knative.dev/pkg/ptr" - "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" corecloudprovider "sigs.k8s.io/karpenter/pkg/cloudprovider" "sigs.k8s.io/karpenter/pkg/controllers/provisioning" @@ -1586,13 +1585,13 @@ func generateSpotPricing(cp *cloudprovider.CloudProvider, nodePool *corev1beta1. instanceType := it onDemandPrice := 1.00 for _, o := range it.Offerings { - if o.CapacityType == v1alpha5.CapacityTypeOnDemand { + if o.CapacityType == corev1beta1.CapacityTypeOnDemand { onDemandPrice = o.Price } } for _, o := range instanceType.Offerings { o := o - if o.CapacityType != v1alpha5.CapacityTypeSpot { + if o.CapacityType != corev1beta1.CapacityTypeSpot { continue } spotPrice := fmt.Sprintf("%0.3f", onDemandPrice*0.5) diff --git a/pkg/providers/launchtemplate/launchtemplate.go b/pkg/providers/launchtemplate/launchtemplate.go index 56a0a4978881..077e6968fe0a 100644 --- a/pkg/providers/launchtemplate/launchtemplate.go +++ b/pkg/providers/launchtemplate/launchtemplate.go @@ -33,8 +33,6 @@ import ( v1 "k8s.io/api/core/v1" "k8s.io/apimachinery/pkg/api/resource" "knative.dev/pkg/logging" - "knative.dev/pkg/ptr" - corev1beta1 "sigs.k8s.io/karpenter/pkg/apis/v1beta1" "github.com/aws/karpenter/pkg/apis/v1beta1" @@ -108,10 +106,6 @@ func (p *Provider) EnsureAll(ctx context.Context, nodeClass *v1beta1.EC2NodeClas p.Lock() defer p.Unlock() - // If Launch Template is directly specified then just use it - if nodeClass.Spec.LaunchTemplateName != nil { - return []*LaunchTemplate{{Name: ptr.StringValue(nodeClass.Spec.LaunchTemplateName), InstanceTypes: instanceTypes}}, nil - } options, err := p.createAMIOptions(ctx, nodeClass, lo.Assign(nodeClaim.Labels, map[string]string{corev1beta1.CapacityTypeLabelKey: capacityType}), tags) if err != nil { diff --git a/pkg/providers/launchtemplate/suite_test.go b/pkg/providers/launchtemplate/suite_test.go index e933a86b270a..d27b73b815be 100644 --- a/pkg/providers/launchtemplate/suite_test.go +++ b/pkg/providers/launchtemplate/suite_test.go @@ -792,7 +792,7 @@ var _ = Describe("LaunchTemplates", func() { ExpectScheduled(ctx, env.Client, pod) ExpectLaunchTemplatesCreatedWithUserDataContaining("--use-max-pods false") }) - It("should specify --use-max-pods=false and --max-pods user value when user specifies maxPods in Provisioner", func() { + It("should specify --use-max-pods=false and --max-pods user value when user specifies maxPods in NodePool", func() { nodePool.Spec.Template.Spec.Kubelet = &corev1beta1.KubeletConfiguration{MaxPods: aws.Int32(10)} ExpectApplied(ctx, env.Client, nodePool, nodeClass) pod := coretest.UnschedulablePod() @@ -1112,12 +1112,12 @@ var _ = Describe("LaunchTemplates", func() { Expect(err).To(BeNil()) ExpectLaunchTemplatesCreatedWithUserData(fmt.Sprintf(string(content), corev1beta1.NodePoolLabelKey, nodePool.Name)) }) - It("should not bootstrap when provider ref points to a non-existent resource", func() { + It("should not bootstrap when provider ref points to a non-existent EC2NodeClass resource", func() { nodePool.Spec.Template.Spec.NodeClassRef = &corev1beta1.NodeClassReference{Name: "doesnotexist"} ExpectApplied(ctx, env.Client, nodePool) pod := coretest.UnschedulablePod() ExpectProvisioned(ctx, env.Client, cluster, cloudProvider, prov, pod) - // This will not be scheduled since we were pointed to a non-existent awsnodetemplate resource. + // This will not be scheduled since we were pointed to a non-existent EC2NodeClass resource. ExpectNotScheduled(ctx, env.Client, pod) }) It("should not bootstrap on invalid toml user data", func() { @@ -1351,7 +1351,7 @@ var _ = Describe("LaunchTemplates", func() { }) }) Context("Custom AMI Selector", func() { - It("should use ami selector specified in AWSNodeTemplate", func() { + It("should use ami selector specified in EC2NodeClass", func() { nodeClass.Spec.AMISelectorTerms = []v1beta1.AMISelectorTerm{{Tags: map[string]string{"*": "*"}}} awsEnv.EC2API.DescribeImagesOutput.Set(&ec2.DescribeImagesOutput{Images: []*ec2.Image{ { @@ -1388,7 +1388,7 @@ var _ = Describe("LaunchTemplates", func() { ExpectScheduled(ctx, env.Client, pod) ExpectLaunchTemplatesCreatedWithUserData("special user data") }) - It("should correctly use ami selector with specific IDs in AWSNodeTemplate", func() { + It("should correctly use ami selector with specific IDs in EC2NodeClass", func() { nodeClass.Spec.AMISelectorTerms = []v1beta1.AMISelectorTerm{{ID: "ami-123"}, {ID: "ami-456"}} awsEnv.EC2API.DescribeImagesOutput.Set(&ec2.DescribeImagesOutput{Images: []*ec2.Image{ { @@ -1508,7 +1508,7 @@ var _ = Describe("LaunchTemplates", func() { ExpectNotScheduled(ctx, env.Client, pod) Expect(awsEnv.EC2API.CalledWithCreateLaunchTemplateInput.Len()).To(Equal(0)) }) - It("should choose amis from SSM if no selector specified in AWSNodeTemplate", func() { + It("should choose amis from SSM if no selector specified in EC2NodeClass", func() { version := lo.Must(awsEnv.VersionProvider.Get(ctx)) awsEnv.SSMAPI.Parameters = map[string]string{ fmt.Sprintf("/aws/service/eks/optimized-ami/%s/amazon-linux-2/recommended/image_id", version): "test-ami-123", diff --git a/pkg/providers/securitygroup/securitygroup.go b/pkg/providers/securitygroup/securitygroup.go index 6287b85df1ea..2ad8c0ea457e 100644 --- a/pkg/providers/securitygroup/securitygroup.go +++ b/pkg/providers/securitygroup/securitygroup.go @@ -46,7 +46,7 @@ func NewProvider(ec2api ec2iface.EC2API, cache *cache.Cache) *Provider { return &Provider{ ec2api: ec2api, cm: pretty.NewChangeMonitor(), - // TODO: Remove cache for v1beta1, utilize resolved security groups from the AWSNodeTemplate.status + // TODO: Remove cache cache when we utilize the security groups from the EC2NodeClass.status cache: cache, } } @@ -54,13 +54,9 @@ func NewProvider(ec2api ec2iface.EC2API, cache *cache.Cache) *Provider { func (p *Provider) List(ctx context.Context, nodeClass *v1beta1.EC2NodeClass) ([]*ec2.SecurityGroup, error) { p.Lock() defer p.Unlock() + // Get SecurityGroups - // TODO: When removing custom launchTemplates for v1beta1, security groups will be required. - // The check will not be necessary filterSets := getFilterSets(nodeClass.Spec.SecurityGroupSelectorTerms) - if len(filterSets) == 0 { - return []*ec2.SecurityGroup{}, nil - } securityGroups, err := p.getSecurityGroups(ctx, filterSets) if err != nil { return nil, err diff --git a/pkg/providers/subnet/subnet.go b/pkg/providers/subnet/subnet.go index a1c3d17a70c1..128981f95bab 100644 --- a/pkg/providers/subnet/subnet.go +++ b/pkg/providers/subnet/subnet.go @@ -47,7 +47,7 @@ func NewProvider(ec2api ec2iface.EC2API, cache *cache.Cache) *Provider { return &Provider{ ec2api: ec2api, cm: pretty.NewChangeMonitor(), - // TODO: Remove cache for v1beta1, utilize resolved subnet from the AWSNodeTemplate.status + // TODO: Remove cache when we utilize the resolved subnets from the EC2NodeClass.status // Subnets are sorted on AvailableIpAddressCount, descending order cache: cache, // inflightIPs is used to track IPs from known launched instances diff --git a/pkg/test/awsnodetemplate.go b/pkg/test/awsnodetemplate.go deleted file mode 100644 index f10cda9b9249..000000000000 --- a/pkg/test/awsnodetemplate.go +++ /dev/null @@ -1,47 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package test - -import ( - "fmt" - - "github.com/imdario/mergo" - - "sigs.k8s.io/karpenter/pkg/test" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" -) - -func AWSNodeTemplate(overrides ...v1alpha1.AWSNodeTemplateSpec) *v1alpha1.AWSNodeTemplate { - options := v1alpha1.AWSNodeTemplateSpec{} - for _, override := range overrides { - if err := mergo.Merge(&options, override, mergo.WithOverride); err != nil { - panic(fmt.Sprintf("Failed to merge settings: %s", err)) - } - } - - if options.AWS.SecurityGroupSelector == nil { - options.AWS.SecurityGroupSelector = map[string]string{"*": "*"} - } - - if options.AWS.SubnetSelector == nil { - options.AWS.SubnetSelector = map[string]string{"*": "*"} - } - - return &v1alpha1.AWSNodeTemplate{ - ObjectMeta: test.ObjectMeta(), - Spec: options, - } -} diff --git a/pkg/test/expectations/expectations.go b/pkg/test/expectations/expectations.go deleted file mode 100644 index 4c9556a988a3..000000000000 --- a/pkg/test/expectations/expectations.go +++ /dev/null @@ -1,88 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package expectations - -import ( - . "github.com/onsi/gomega" - "github.com/samber/lo" - v1 "k8s.io/api/core/v1" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1beta1" -) - -func ExpectBlockDeviceMappingsEqual(bdm1 []*v1alpha1.BlockDeviceMapping, bdm2 []*v1beta1.BlockDeviceMapping) { - // Expect that all BlockDeviceMappings are present and the same - // Ensure that they are the same by ensuring a consistent ordering - Expect(bdm1).To(HaveLen(len(bdm2))) - for i := range bdm1 { - Expect(lo.FromPtr(bdm1[i].DeviceName)).To(Equal(lo.FromPtr(bdm2[i].DeviceName))) - ExpectBlockDevicesEqual(bdm1[i].EBS, bdm2[i].EBS) - } -} - -func ExpectBlockDevicesEqual(bd1 *v1alpha1.BlockDevice, bd2 *v1beta1.BlockDevice) { - Expect(bd1 == nil).To(Equal(bd2 == nil)) - if bd1 != nil { - Expect(lo.FromPtr(bd1.DeleteOnTermination)).To(Equal(lo.FromPtr(bd2.VolumeType))) - Expect(lo.FromPtr(bd1.Encrypted)).To(Equal(lo.FromPtr(bd2.Encrypted))) - Expect(lo.FromPtr(bd1.IOPS)).To(Equal(lo.FromPtr(bd2.IOPS))) - Expect(lo.FromPtr(bd1.KMSKeyID)).To(Equal(lo.FromPtr(bd2.KMSKeyID))) - Expect(lo.FromPtr(bd1.SnapshotID)).To(Equal(lo.FromPtr(bd2.SnapshotID))) - Expect(lo.FromPtr(bd1.Throughput)).To(Equal(lo.FromPtr(bd2.Throughput))) - Expect(lo.FromPtr(bd1.VolumeSize)).To(Equal(lo.FromPtr(bd2.VolumeSize))) - Expect(lo.FromPtr(bd1.VolumeType)).To(Equal(lo.FromPtr(bd2.VolumeType))) - } -} - -func ExpectMetadataOptionsEqual(mo1 *v1alpha1.MetadataOptions, mo2 *v1beta1.MetadataOptions) { - Expect(mo1 == nil).To(Equal(mo2 == nil)) - if mo1 != nil { - Expect(lo.FromPtr(mo1.HTTPEndpoint)).To(Equal(lo.FromPtr(mo2.HTTPEndpoint))) - Expect(lo.FromPtr(mo1.HTTPProtocolIPv6)).To(Equal(lo.FromPtr(mo2.HTTPProtocolIPv6))) - Expect(lo.FromPtr(mo1.HTTPPutResponseHopLimit)).To(Equal(lo.FromPtr(mo2.HTTPPutResponseHopLimit))) - Expect(lo.FromPtr(mo1.HTTPTokens)).To(Equal(lo.FromPtr(mo2.HTTPTokens))) - } -} - -func ExpectSubnetStatusEqual(subnets1 []v1alpha1.Subnet, subnets2 []v1beta1.Subnet) { - // Expect that all Subnet Status entries are present and the same - // Ensure that they are the same by ensuring a consistent ordering - Expect(subnets1).To(HaveLen(len(subnets2))) - for i := range subnets1 { - Expect(subnets1[i].ID).To(Equal(subnets2[i].ID)) - Expect(subnets1[i].Zone).To(Equal(subnets2[i].Zone)) - } -} - -func ExpectSecurityGroupStatusEqual(securityGroups1 []v1alpha1.SecurityGroup, securityGroups2 []v1beta1.SecurityGroup) { - // Expect that all SecurityGroup Status entries are present and the same - // Ensure that they are the same by ensuring a consistent ordering - Expect(securityGroups1).To(HaveLen(len(securityGroups2))) - for i := range securityGroups1 { - Expect(securityGroups1[i].ID).To(Equal(securityGroups2[i].ID)) - Expect(securityGroups1[i].Name).To(Equal(securityGroups2[i].Name)) - } -} - -func ExpectAMIStatusEqual(amis1 []v1alpha1.AMI, amis2 []v1beta1.AMI) { - // Expect that all AMI Status entries are present and the same - Expect(amis1).To(HaveLen(len(amis2))) - for i := range amis1 { - Expect(amis1[i].ID).To(Equal(amis2[i].ID)) - Expect(amis1[i].Name).To(Equal(amis2[i].Name)) - Expect(amis1[i].Requirements).To(ConsistOf(lo.Map(amis2[i].Requirements, func(r v1.NodeSelectorRequirement, _ int) interface{} { return BeEquivalentTo(r) })...)) - } -} diff --git a/pkg/test/provisioner.go b/pkg/test/provisioner.go deleted file mode 100644 index 8995c91db439..000000000000 --- a/pkg/test/provisioner.go +++ /dev/null @@ -1,32 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package test - -import ( - "context" - - "github.com/samber/lo" - - corev1alpha5 "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" - "sigs.k8s.io/karpenter/pkg/test" - - "github.com/aws/karpenter/pkg/apis/v1alpha5" -) - -func Provisioner(options test.ProvisionerOptions) *corev1alpha5.Provisioner { - provisioner := v1alpha5.Provisioner(lo.FromPtr(test.Provisioner(options))) - provisioner.SetDefaults(context.Background()) - return lo.ToPtr(corev1alpha5.Provisioner(provisioner)) -} diff --git a/pkg/utils/nodeclass/nodeclass.go b/pkg/utils/nodeclass/nodeclass.go deleted file mode 100644 index d8d34bdf7fca..000000000000 --- a/pkg/utils/nodeclass/nodeclass.go +++ /dev/null @@ -1,247 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package nodeclass - -import ( - "context" - "strings" - - "github.com/samber/lo" - "k8s.io/apimachinery/pkg/types" - "sigs.k8s.io/controller-runtime/pkg/client" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1beta1" -) - -func New(nodeTemplate *v1alpha1.AWSNodeTemplate) *v1beta1.EC2NodeClass { - return &v1beta1.EC2NodeClass{ - TypeMeta: nodeTemplate.TypeMeta, - ObjectMeta: nodeTemplate.ObjectMeta, - Spec: v1beta1.EC2NodeClassSpec{ - SubnetSelectorTerms: NewSubnetSelectorTerms(nodeTemplate.Spec.SubnetSelector), - OriginalSubnetSelector: nodeTemplate.Spec.SubnetSelector, - SecurityGroupSelectorTerms: NewSecurityGroupSelectorTerms(nodeTemplate.Spec.SecurityGroupSelector), - OriginalSecurityGroupSelector: nodeTemplate.Spec.SecurityGroupSelector, - AMISelectorTerms: NewAMISelectorTerms(nodeTemplate.Spec.AMISelector), - OriginalAMISelector: nodeTemplate.Spec.AMISelector, - AMIFamily: nodeTemplate.Spec.AMIFamily, - UserData: nodeTemplate.Spec.UserData, - Tags: nodeTemplate.Spec.Tags, - BlockDeviceMappings: NewBlockDeviceMappings(nodeTemplate.Spec.BlockDeviceMappings), - DetailedMonitoring: nodeTemplate.Spec.DetailedMonitoring, - MetadataOptions: NewMetadataOptions(nodeTemplate.Spec.MetadataOptions), - Context: nodeTemplate.Spec.Context, - LaunchTemplateName: nodeTemplate.Spec.LaunchTemplateName, - InstanceProfile: nodeTemplate.Spec.InstanceProfile, - }, - Status: v1beta1.EC2NodeClassStatus{ - Subnets: NewSubnets(nodeTemplate.Status.Subnets), - SecurityGroups: NewSecurityGroups(nodeTemplate.Status.SecurityGroups), - AMIs: NewAMIs(nodeTemplate.Status.AMIs), - }, - } -} - -func NewSubnetSelectorTerms(subnetSelector map[string]string) (terms []v1beta1.SubnetSelectorTerm) { - if len(subnetSelector) == 0 { - return nil - } - // Each of these slices needs to be pre-populated with the "0" element so that we can properly generate permutations - ids := []string{""} - tags := map[string]string{} - for k, v := range subnetSelector { - switch k { - case "aws-ids", "aws::ids": - ids = lo.Map(strings.Split(v, ","), func(s string, _ int) string { return strings.Trim(s, " ") }) - default: - tags[k] = v - } - } - // If there are some "special" keys used, we have to represent the old selector as multiple terms - for _, id := range ids { - terms = append(terms, v1beta1.SubnetSelectorTerm{ - Tags: tags, - ID: id, - }) - } - return terms -} - -func NewSecurityGroupSelectorTerms(securityGroupSelector map[string]string) (terms []v1beta1.SecurityGroupSelectorTerm) { - if len(securityGroupSelector) == 0 { - return nil - } - // Each of these slices needs to be pre-populated with the "0" element so that we can properly generate permutations - ids := []string{""} - tags := map[string]string{} - for k, v := range securityGroupSelector { - switch k { - case "aws-ids", "aws::ids": - ids = lo.Map(strings.Split(v, ","), func(s string, _ int) string { return strings.Trim(s, " ") }) - default: - tags[k] = v - } - } - // If there are some "special" keys used, we have to represent the old selector as multiple terms - for _, id := range ids { - terms = append(terms, v1beta1.SecurityGroupSelectorTerm{ - Tags: tags, - ID: id, - }) - } - return terms -} - -func NewAMISelectorTerms(amiSelector map[string]string) (terms []v1beta1.AMISelectorTerm) { - if len(amiSelector) == 0 { - return nil - } - // Each of these slices needs to be pre-populated with the "0" element so that we can properly generate permutations - ids := []string{""} - names := []string{""} - owners := []string{""} - tags := map[string]string{} - for k, v := range amiSelector { - switch k { - case "aws-ids", "aws::ids": - ids = lo.Map(strings.Split(v, ","), func(s string, _ int) string { return strings.Trim(s, " ") }) - case "aws::name": - names = lo.Map(strings.Split(v, ","), func(s string, _ int) string { return strings.Trim(s, " ") }) - case "aws::owners": - owners = lo.Map(strings.Split(v, ","), func(s string, _ int) string { return strings.Trim(s, " ") }) - default: - tags[k] = v - } - } - // If there are some "special" keys used, we have to represent the old selector as multiple terms - for _, owner := range owners { - for _, id := range ids { - for _, name := range names { - terms = append(terms, v1beta1.AMISelectorTerm{ - Tags: tags, - ID: id, - Name: name, - Owner: owner, - }) - } - } - } - return terms -} - -func NewBlockDeviceMappings(bdms []*v1alpha1.BlockDeviceMapping) []*v1beta1.BlockDeviceMapping { - if bdms == nil { - return nil - } - return lo.Map(bdms, func(bdm *v1alpha1.BlockDeviceMapping, _ int) *v1beta1.BlockDeviceMapping { - return NewBlockDeviceMapping(bdm) - }) -} - -func NewBlockDeviceMapping(bdm *v1alpha1.BlockDeviceMapping) *v1beta1.BlockDeviceMapping { - if bdm == nil { - return nil - } - return &v1beta1.BlockDeviceMapping{ - DeviceName: bdm.DeviceName, - EBS: NewBlockDevice(bdm.EBS), - } -} - -func NewBlockDevice(bd *v1alpha1.BlockDevice) *v1beta1.BlockDevice { - if bd == nil { - return nil - } - return &v1beta1.BlockDevice{ - DeleteOnTermination: bd.DeleteOnTermination, - Encrypted: bd.Encrypted, - IOPS: bd.IOPS, - KMSKeyID: bd.KMSKeyID, - SnapshotID: bd.SnapshotID, - Throughput: bd.Throughput, - VolumeSize: bd.VolumeSize, - VolumeType: bd.VolumeType, - } -} - -func NewMetadataOptions(mo *v1alpha1.MetadataOptions) *v1beta1.MetadataOptions { - if mo == nil { - return nil - } - return &v1beta1.MetadataOptions{ - HTTPEndpoint: mo.HTTPEndpoint, - HTTPProtocolIPv6: mo.HTTPProtocolIPv6, - HTTPPutResponseHopLimit: mo.HTTPPutResponseHopLimit, - HTTPTokens: mo.HTTPTokens, - } -} - -func NewSubnets(subnets []v1alpha1.Subnet) []v1beta1.Subnet { - if subnets == nil { - return nil - } - return lo.Map(subnets, func(s v1alpha1.Subnet, _ int) v1beta1.Subnet { - return v1beta1.Subnet{ - ID: s.ID, - Zone: s.Zone, - } - }) -} - -func NewSecurityGroups(securityGroups []v1alpha1.SecurityGroup) []v1beta1.SecurityGroup { - if securityGroups == nil { - return nil - } - return lo.Map(securityGroups, func(s v1alpha1.SecurityGroup, _ int) v1beta1.SecurityGroup { - return v1beta1.SecurityGroup{ - ID: s.ID, - Name: s.Name, - } - }) -} - -func NewAMIs(amis []v1alpha1.AMI) []v1beta1.AMI { - if amis == nil { - return nil - } - return lo.Map(amis, func(a v1alpha1.AMI, _ int) v1beta1.AMI { - return v1beta1.AMI{ - ID: a.ID, - Name: a.Name, - Requirements: a.Requirements, - } - }) -} - -func Get(ctx context.Context, c client.Client, name string) (*v1beta1.EC2NodeClass, error) { - nodeClass := &v1beta1.EC2NodeClass{} - if err := c.Get(ctx, types.NamespacedName{Name: name}, nodeClass); err != nil { - return nil, err - } - return nodeClass, nil -} - -func Patch(ctx context.Context, c client.Client, stored, nodeClass *v1beta1.EC2NodeClass) error { - return c.Patch(ctx, nodeClass, client.MergeFrom(stored)) -} - -func PatchStatus(ctx context.Context, c client.Client, stored, nodeClass *v1beta1.EC2NodeClass) error { - return c.Status().Patch(ctx, nodeClass, client.MergeFrom(stored)) -} - -func HashAnnotation(nodeClass *v1beta1.EC2NodeClass) map[string]string { - return map[string]string{v1beta1.AnnotationEC2NodeClassHash: nodeClass.Hash()} -} diff --git a/pkg/utils/nodeclass/suite_test.go b/pkg/utils/nodeclass/suite_test.go deleted file mode 100644 index 287bbb90bd81..000000000000 --- a/pkg/utils/nodeclass/suite_test.go +++ /dev/null @@ -1,545 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package nodeclass_test - -import ( - "context" - "testing" - - "github.com/aws/aws-sdk-go/aws" - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - - "github.com/samber/lo" - v1 "k8s.io/api/core/v1" - . "knative.dev/pkg/logging/testing" - - "sigs.k8s.io/karpenter/pkg/operator/scheme" - . "sigs.k8s.io/karpenter/pkg/test/expectations" - - "github.com/aws/karpenter/pkg/apis" - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1beta1" - . "github.com/aws/karpenter/pkg/test/expectations" - - coretest "sigs.k8s.io/karpenter/pkg/test" - - "github.com/aws/karpenter/pkg/test" - nodeclassutil "github.com/aws/karpenter/pkg/utils/nodeclass" - nodetemplateutil "github.com/aws/karpenter/pkg/utils/nodetemplate" -) - -func init() { - lo.Must0(apis.AddToScheme(scheme.Scheme)) -} - -var ctx context.Context -var env *coretest.Environment - -func TestAPIs(t *testing.T) { - ctx = TestContextWithLogger(t) - RegisterFailHandler(Fail) - RunSpecs(t, "NodeClaimUtils") -} - -var _ = BeforeSuite(func() { - env = coretest.NewEnvironment(scheme.Scheme, coretest.WithCRDs(apis.CRDs...)) -}) - -var _ = AfterSuite(func() { - Expect(env.Stop()).To(Succeed(), "Failed to stop environment") -}) - -var _ = AfterEach(func() { - ExpectCleanedUp(ctx, env.Client) -}) - -var _ = Describe("NodeClassUtils", func() { - var nodeTemplate *v1alpha1.AWSNodeTemplate - BeforeEach(func() { - nodeTemplate = test.AWSNodeTemplate(v1alpha1.AWSNodeTemplateSpec{ - AWS: v1alpha1.AWS{ - AMIFamily: aws.String(v1alpha1.AMIFamilyAL2), - Context: aws.String("context-1"), - InstanceProfile: aws.String("profile-1"), - Tags: map[string]string{ - "keyTag-1": "valueTag-1", - "keyTag-2": "valueTag-2", - }, - SubnetSelector: map[string]string{ - "test-subnet-key": "test-subnet-value", - }, - SecurityGroupSelector: map[string]string{ - "test-security-group-key": "test-security-group-value", - }, - LaunchTemplate: v1alpha1.LaunchTemplate{ - MetadataOptions: &v1alpha1.MetadataOptions{ - HTTPEndpoint: aws.String("test-metadata-1"), - }, - BlockDeviceMappings: []*v1alpha1.BlockDeviceMapping{ - { - DeviceName: aws.String("map-device-1"), - }, - { - DeviceName: aws.String("map-device-2"), - }, - }, - }, - }, - UserData: aws.String("userdata-test-1"), - DetailedMonitoring: aws.Bool(false), - AMISelector: map[string]string{ - "test-ami-key": "test-ami-value", - }, - }) - nodeTemplate.Status = v1alpha1.AWSNodeTemplateStatus{ - Subnets: []v1alpha1.Subnet{ - { - ID: "test-subnet-id", - Zone: "test-zone-1a", - }, - { - ID: "test-subnet-id2", - Zone: "test-zone-1b", - }, - }, - SecurityGroups: []v1alpha1.SecurityGroup{ - { - ID: "test-security-group-id", - Name: "test-security-group-name", - }, - { - ID: "test-security-group-id2", - Name: "test-security-group-name2", - }, - }, - AMIs: []v1alpha1.AMI{ - { - ID: "test-ami-id", - Name: "test-ami-name", - Requirements: []v1.NodeSelectorRequirement{ - { - Key: v1.LabelArchStable, - Operator: v1.NodeSelectorOpIn, - Values: []string{"amd64"}, - }, - }, - }, - { - ID: "test-ami-id2", - Name: "test-ami-name2", - Requirements: []v1.NodeSelectorRequirement{ - { - Key: v1.LabelArchStable, - Operator: v1.NodeSelectorOpIn, - Values: []string{"arm64"}, - }, - }, - }, - }, - } - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass", func() { - nodeClass := nodeclassutil.New(nodeTemplate) - - for k, v := range nodeTemplate.Annotations { - Expect(nodeClass.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeTemplate.Labels { - Expect(nodeClass.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeClass.Spec.SubnetSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SubnetSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - Expect(nodeClass.Spec.AMISelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.AMISelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.AMISelector)) - Expect(nodeClass.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(nodeClass.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(nodeClass.Spec.Role).To(BeEmpty()) - Expect(nodeClass.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - Expect(nodeClass.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - Expect(nodeClass.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(nodeClass.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(nodeClass.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass (with AMISelector name and owner values set)", func() { - nodeTemplate.Spec.AMISelector = map[string]string{ - "aws::name": "ami-name1,ami-name2", - "aws::owners": "self,amazon,123456789", - } - nodeClass := nodeclassutil.New(nodeTemplate) - - for k, v := range nodeTemplate.Annotations { - Expect(nodeClass.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeTemplate.Labels { - Expect(nodeClass.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeClass.Spec.SubnetSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SubnetSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - - // Expect AMISelectorTerms to be exactly what we would expect from the filtering above - Expect(nodeClass.Spec.AMISelectorTerms).To(HaveLen(6)) - Expect(nodeClass.Spec.AMISelectorTerms).To(ConsistOf( - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "self", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "amazon", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "123456789", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "self", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "amazon", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "123456789", - Tags: map[string]string{}, - }, - )) - - Expect(nodeClass.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(nodeClass.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(nodeClass.Spec.Role).To(BeEmpty()) - Expect(nodeClass.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - Expect(nodeClass.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - Expect(nodeClass.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(nodeClass.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(nodeClass.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass (with AMISelector name and owner values set) with spaces", func() { - nodeTemplate.Spec.AMISelector = map[string]string{ - "aws::name": "ami-name1, ami-name2, test name", - "aws::owners": "self, amazon, 123456789, test owner", - } - nodeClass := nodeclassutil.New(nodeTemplate) - - for k, v := range nodeTemplate.Annotations { - Expect(nodeClass.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeTemplate.Labels { - Expect(nodeClass.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeClass.Spec.SubnetSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SubnetSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - - // Expect AMISelectorTerms to be exactly what we would expect from the filtering above - Expect(nodeClass.Spec.AMISelectorTerms).To(HaveLen(12)) - Expect(nodeClass.Spec.AMISelectorTerms).To(ConsistOf( - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "self", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "amazon", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "123456789", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "test owner", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "self", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "amazon", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "123456789", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "test owner", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "test name", - Owner: "self", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "test name", - Owner: "amazon", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "test name", - Owner: "123456789", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - Name: "test name", - Owner: "test owner", - Tags: map[string]string{}, - }, - )) - - Expect(nodeClass.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(nodeClass.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(nodeClass.Spec.Role).To(BeEmpty()) - Expect(nodeClass.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - Expect(nodeClass.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - Expect(nodeClass.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(nodeClass.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(nodeClass.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass (with AMISelector id set)", func() { - nodeTemplate.Spec.AMISelector = map[string]string{ - "aws::ids": "ami-1234,ami-5678,ami-custom-id", - } - nodeClass := nodeclassutil.New(nodeTemplate) - - for k, v := range nodeTemplate.Annotations { - Expect(nodeClass.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeTemplate.Labels { - Expect(nodeClass.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeClass.Spec.SubnetSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SubnetSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - - // Expect AMISelectorTerms to be exactly what we would expect from the filtering above - Expect(nodeClass.Spec.AMISelectorTerms).To(HaveLen(3)) - Expect(nodeClass.Spec.AMISelectorTerms).To(ConsistOf( - v1beta1.AMISelectorTerm{ - ID: "ami-1234", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - ID: "ami-5678", - Tags: map[string]string{}, - }, - v1beta1.AMISelectorTerm{ - ID: "ami-custom-id", - Tags: map[string]string{}, - }, - )) - - Expect(nodeClass.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(nodeClass.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(nodeClass.Spec.Role).To(BeEmpty()) - Expect(nodeClass.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - Expect(nodeClass.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - Expect(nodeClass.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(nodeClass.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(nodeClass.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass (with AMISelector name, owner, id, and tags set)", func() { - nodeTemplate.Spec.AMISelector = map[string]string{ - "aws::name": "ami-name1,ami-name2", - "aws::owners": "self,amazon", - "aws::ids": "ami-1234,ami-5678", - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - } - nodeClass := nodeclassutil.New(nodeTemplate) - - for k, v := range nodeTemplate.Annotations { - Expect(nodeClass.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeTemplate.Labels { - Expect(nodeClass.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeClass.Spec.SubnetSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SubnetSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms).To(HaveLen(1)) - Expect(nodeClass.Spec.SecurityGroupSelectorTerms[0].Tags).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - - // Expect AMISelectorTerms to be exactly what we would expect from the filtering above - // This should include all permutations of the filters that could be used by this selector mechanism - Expect(nodeClass.Spec.AMISelectorTerms).To(HaveLen(8)) - Expect(nodeClass.Spec.AMISelectorTerms).To(ConsistOf( - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "self", - ID: "ami-1234", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "self", - ID: "ami-5678", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "amazon", - ID: "ami-1234", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name1", - Owner: "amazon", - ID: "ami-5678", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "self", - ID: "ami-1234", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "self", - ID: "ami-5678", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "amazon", - ID: "ami-1234", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - v1beta1.AMISelectorTerm{ - Name: "ami-name2", - Owner: "amazon", - ID: "ami-5678", - Tags: map[string]string{ - "custom-tag": "custom-value", - "custom-tag2": "custom-value2", - }, - }, - )) - - Expect(nodeClass.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(nodeClass.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(nodeClass.Spec.Role).To(BeEmpty()) - Expect(nodeClass.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - Expect(nodeClass.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - Expect(nodeClass.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(nodeClass.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(nodeClass.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) - It("should convert a AWSNodeTemplate to a EC2NodeClass and back and still retain all original data", func() { - convertedNodeTemplate := nodetemplateutil.New(nodeclassutil.New(nodeTemplate)) - - Expect(convertedNodeTemplate.Name).To(Equal(nodeTemplate.Name)) - Expect(convertedNodeTemplate.Annotations).To(Equal(nodeTemplate.Annotations)) - Expect(convertedNodeTemplate.Labels).To(Equal(nodeTemplate.Labels)) - - Expect(convertedNodeTemplate.Spec.UserData).To(Equal(nodeTemplate.Spec.UserData)) - Expect(convertedNodeTemplate.Spec.AMISelector).To(Equal(nodeTemplate.Spec.AMISelector)) - Expect(convertedNodeTemplate.Spec.DetailedMonitoring).To(Equal(nodeTemplate.Spec.DetailedMonitoring)) - Expect(convertedNodeTemplate.Spec.AMIFamily).To(Equal(nodeTemplate.Spec.AMIFamily)) - Expect(convertedNodeTemplate.Spec.Context).To(Equal(nodeTemplate.Spec.Context)) - Expect(convertedNodeTemplate.Spec.InstanceProfile).To(Equal(nodeTemplate.Spec.InstanceProfile)) - Expect(convertedNodeTemplate.Spec.SubnetSelector).To(Equal(nodeTemplate.Spec.SubnetSelector)) - Expect(convertedNodeTemplate.Spec.SecurityGroupSelector).To(Equal(nodeTemplate.Spec.SecurityGroupSelector)) - Expect(convertedNodeTemplate.Spec.Tags).To(Equal(nodeTemplate.Spec.Tags)) - Expect(convertedNodeTemplate.Spec.LaunchTemplateName).To(Equal(nodeTemplate.Spec.LaunchTemplateName)) - Expect(convertedNodeTemplate.Spec.MetadataOptions).To(Equal(nodeTemplate.Spec.MetadataOptions)) - Expect(convertedNodeTemplate.Spec.BlockDeviceMappings).To(Equal(nodeTemplate.Spec.BlockDeviceMappings)) - - Expect(convertedNodeTemplate.Status.SecurityGroups).To(Equal(nodeTemplate.Status.SecurityGroups)) - Expect(convertedNodeTemplate.Status.Subnets).To(Equal(nodeTemplate.Status.Subnets)) - Expect(convertedNodeTemplate.Status.AMIs).To(Equal(nodeTemplate.Status.AMIs)) - }) - It("should retrieve a EC2NodeClass with a get call", func() { - nodeClass := test.EC2NodeClass() - ExpectApplied(ctx, env.Client, nodeClass) - - retrieved, err := nodeclassutil.Get(ctx, env.Client, nodeClass.Name) - Expect(err).ToNot(HaveOccurred()) - Expect(retrieved.Name).To(Equal(nodeClass.Name)) - }) -}) diff --git a/pkg/utils/nodetemplate/nodetemplate.go b/pkg/utils/nodetemplate/nodetemplate.go deleted file mode 100644 index 3112a04b51e9..000000000000 --- a/pkg/utils/nodetemplate/nodetemplate.go +++ /dev/null @@ -1,136 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package nodetemplate - -import ( - "github.com/samber/lo" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1beta1" -) - -func New(nodeClass *v1beta1.EC2NodeClass) *v1alpha1.AWSNodeTemplate { - return &v1alpha1.AWSNodeTemplate{ - TypeMeta: nodeClass.TypeMeta, - ObjectMeta: nodeClass.ObjectMeta, - Spec: v1alpha1.AWSNodeTemplateSpec{ - UserData: nodeClass.Spec.UserData, - AWS: v1alpha1.AWS{ - AMIFamily: nodeClass.Spec.AMIFamily, - Context: nodeClass.Spec.Context, - InstanceProfile: nodeClass.Spec.InstanceProfile, - SubnetSelector: nodeClass.Spec.OriginalSubnetSelector, - SecurityGroupSelector: nodeClass.Spec.OriginalSecurityGroupSelector, - Tags: nodeClass.Spec.Tags, - LaunchTemplate: v1alpha1.LaunchTemplate{ - LaunchTemplateName: nodeClass.Spec.LaunchTemplateName, - MetadataOptions: NewMetadataOptions(nodeClass.Spec.MetadataOptions), - BlockDeviceMappings: NewBlockDeviceMappings(nodeClass.Spec.BlockDeviceMappings), - }, - }, - AMISelector: nodeClass.Spec.OriginalAMISelector, - DetailedMonitoring: nodeClass.Spec.DetailedMonitoring, - }, - Status: v1alpha1.AWSNodeTemplateStatus{ - Subnets: NewSubnets(nodeClass.Status.Subnets), - SecurityGroups: NewSecurityGroups(nodeClass.Status.SecurityGroups), - AMIs: NewAMIs(nodeClass.Status.AMIs), - }, - } -} - -func NewBlockDeviceMappings(bdms []*v1beta1.BlockDeviceMapping) []*v1alpha1.BlockDeviceMapping { - if bdms == nil { - return nil - } - return lo.Map(bdms, func(bdm *v1beta1.BlockDeviceMapping, _ int) *v1alpha1.BlockDeviceMapping { - return NewBlockDeviceMapping(bdm) - }) -} - -func NewBlockDeviceMapping(bdm *v1beta1.BlockDeviceMapping) *v1alpha1.BlockDeviceMapping { - if bdm == nil { - return nil - } - return &v1alpha1.BlockDeviceMapping{ - DeviceName: bdm.DeviceName, - EBS: NewBlockDevice(bdm.EBS), - } -} - -func NewBlockDevice(bd *v1beta1.BlockDevice) *v1alpha1.BlockDevice { - if bd == nil { - return nil - } - return &v1alpha1.BlockDevice{ - DeleteOnTermination: bd.DeleteOnTermination, - Encrypted: bd.Encrypted, - IOPS: bd.IOPS, - KMSKeyID: bd.KMSKeyID, - SnapshotID: bd.SnapshotID, - Throughput: bd.Throughput, - VolumeSize: bd.VolumeSize, - VolumeType: bd.VolumeType, - } -} - -func NewMetadataOptions(mo *v1beta1.MetadataOptions) *v1alpha1.MetadataOptions { - if mo == nil { - return nil - } - return &v1alpha1.MetadataOptions{ - HTTPEndpoint: mo.HTTPEndpoint, - HTTPProtocolIPv6: mo.HTTPProtocolIPv6, - HTTPPutResponseHopLimit: mo.HTTPPutResponseHopLimit, - HTTPTokens: mo.HTTPTokens, - } -} - -func NewSubnets(subnets []v1beta1.Subnet) []v1alpha1.Subnet { - if subnets == nil { - return nil - } - return lo.Map(subnets, func(s v1beta1.Subnet, _ int) v1alpha1.Subnet { - return v1alpha1.Subnet{ - ID: s.ID, - Zone: s.Zone, - } - }) -} - -func NewSecurityGroups(securityGroups []v1beta1.SecurityGroup) []v1alpha1.SecurityGroup { - if securityGroups == nil { - return nil - } - return lo.Map(securityGroups, func(s v1beta1.SecurityGroup, _ int) v1alpha1.SecurityGroup { - return v1alpha1.SecurityGroup{ - ID: s.ID, - Name: s.Name, - } - }) -} - -func NewAMIs(amis []v1beta1.AMI) []v1alpha1.AMI { - if amis == nil { - return nil - } - return lo.Map(amis, func(a v1beta1.AMI, _ int) v1alpha1.AMI { - return v1alpha1.AMI{ - ID: a.ID, - Name: a.Name, - Requirements: a.Requirements, - } - }) -} diff --git a/pkg/utils/nodetemplate/suite_test.go b/pkg/utils/nodetemplate/suite_test.go deleted file mode 100644 index 5811b8eafa04..000000000000 --- a/pkg/utils/nodetemplate/suite_test.go +++ /dev/null @@ -1,174 +0,0 @@ -/* -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at - - http://www.apache.org/licenses/LICENSE-2.0 - -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. -*/ - -package nodetemplate_test - -import ( - "context" - "testing" - - "github.com/aws/aws-sdk-go/aws" - . "github.com/onsi/ginkgo/v2" - . "github.com/onsi/gomega" - "github.com/samber/lo" - v1 "k8s.io/api/core/v1" - . "knative.dev/pkg/logging/testing" - - "sigs.k8s.io/karpenter/pkg/operator/scheme" - coretest "sigs.k8s.io/karpenter/pkg/test" - . "sigs.k8s.io/karpenter/pkg/test/expectations" - - "github.com/aws/karpenter/pkg/apis" - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1beta1" - "github.com/aws/karpenter/pkg/test" - . "github.com/aws/karpenter/pkg/test/expectations" - nodetemplateutil "github.com/aws/karpenter/pkg/utils/nodetemplate" -) - -func init() { - lo.Must0(apis.AddToScheme(scheme.Scheme)) -} - -var ctx context.Context -var env *coretest.Environment - -func TestAPIs(t *testing.T) { - ctx = TestContextWithLogger(t) - RegisterFailHandler(Fail) - RunSpecs(t, "NodeClaimUtils") -} - -var _ = BeforeSuite(func() { - env = coretest.NewEnvironment(scheme.Scheme, coretest.WithCRDs(apis.CRDs...)) -}) - -var _ = AfterSuite(func() { - Expect(env.Stop()).To(Succeed(), "Failed to stop environment") -}) - -var _ = AfterEach(func() { - ExpectCleanedUp(ctx, env.Client) -}) - -var _ = Describe("NodeTemplateUtils", func() { - var nodeClass *v1beta1.EC2NodeClass - BeforeEach(func() { - nodeClass = test.EC2NodeClass(v1beta1.EC2NodeClass{ - Spec: v1beta1.EC2NodeClassSpec{ - AMIFamily: aws.String(v1alpha1.AMIFamilyAL2), - Context: aws.String("context-1"), - InstanceProfile: aws.String("profile-1"), - Tags: map[string]string{ - "keyTag-1": "valueTag-1", - "keyTag-2": "valueTag-2", - }, - OriginalSubnetSelector: map[string]string{ - "test-subnet-key": "test-subnet-value", - }, - OriginalSecurityGroupSelector: map[string]string{ - "test-security-group-key": "test-security-group-value", - }, - MetadataOptions: &v1beta1.MetadataOptions{ - HTTPEndpoint: aws.String("test-metadata-1"), - }, - BlockDeviceMappings: []*v1beta1.BlockDeviceMapping{ - { - DeviceName: aws.String("map-device-1"), - }, - { - DeviceName: aws.String("map-device-2"), - }, - }, - UserData: aws.String("userdata-test-1"), - DetailedMonitoring: aws.Bool(false), - OriginalAMISelector: map[string]string{ - "test-ami-key": "test-ami-value", - }, - }, - }) - nodeClass.Status = v1beta1.EC2NodeClassStatus{ - Subnets: []v1beta1.Subnet{ - { - ID: "test-subnet-id", - Zone: "test-zone-1a", - }, - { - ID: "test-subnet-id2", - Zone: "test-zone-1b", - }, - }, - SecurityGroups: []v1beta1.SecurityGroup{ - { - ID: "test-security-group-id", - Name: "test-security-group-name", - }, - { - ID: "test-security-group-id2", - Name: "test-security-group-name2", - }, - }, - AMIs: []v1beta1.AMI{ - { - ID: "test-ami-id", - Name: "test-ami-name", - Requirements: []v1.NodeSelectorRequirement{ - { - Key: v1.LabelArchStable, - Operator: v1.NodeSelectorOpIn, - Values: []string{"amd64"}, - }, - }, - }, - { - ID: "test-ami-id2", - Name: "test-ami-name2", - Requirements: []v1.NodeSelectorRequirement{ - { - Key: v1.LabelArchStable, - Operator: v1.NodeSelectorOpIn, - Values: []string{"arm64"}, - }, - }, - }, - }, - } - }) - It("should convert a EC2NodeClass to an AWSNodeTemplate", func() { - nodeTemplate := nodetemplateutil.New(nodeClass) - - for k, v := range nodeClass.Annotations { - Expect(nodeTemplate.Annotations).To(HaveKeyWithValue(k, v)) - } - for k, v := range nodeClass.Labels { - Expect(nodeTemplate.Labels).To(HaveKeyWithValue(k, v)) - } - Expect(nodeTemplate.Spec.SubnetSelector).To(Equal(nodeClass.Spec.OriginalSubnetSelector)) - Expect(nodeTemplate.Spec.SecurityGroupSelector).To(Equal(nodeClass.Spec.OriginalSecurityGroupSelector)) - Expect(nodeTemplate.Spec.AMISelector).To(Equal(nodeClass.Spec.OriginalAMISelector)) - Expect(nodeTemplate.Spec.AMIFamily).To(Equal(nodeClass.Spec.AMIFamily)) - Expect(nodeTemplate.Spec.Context).To(Equal(nodeClass.Spec.Context)) - Expect(nodeTemplate.Spec.InstanceProfile).To(Equal(nodeClass.Spec.InstanceProfile)) - Expect(nodeTemplate.Spec.UserData).To(Equal(nodeClass.Spec.UserData)) - Expect(nodeTemplate.Spec.Tags).To(Equal(nodeClass.Spec.Tags)) - Expect(nodeTemplate.Spec.DetailedMonitoring).To(Equal(nodeClass.Spec.DetailedMonitoring)) - Expect(nodeTemplate.Spec.LaunchTemplateName).To(Equal(nodeClass.Spec.LaunchTemplateName)) - - ExpectBlockDeviceMappingsEqual(nodeTemplate.Spec.BlockDeviceMappings, nodeClass.Spec.BlockDeviceMappings) - ExpectMetadataOptionsEqual(nodeTemplate.Spec.MetadataOptions, nodeClass.Spec.MetadataOptions) - ExpectSubnetStatusEqual(nodeTemplate.Status.Subnets, nodeClass.Status.Subnets) - ExpectSecurityGroupStatusEqual(nodeTemplate.Status.SecurityGroups, nodeClass.Status.SecurityGroups) - ExpectAMIStatusEqual(nodeTemplate.Status.AMIs, nodeClass.Status.AMIs) - }) -}) diff --git a/pkg/webhooks/webhooks.go b/pkg/webhooks/webhooks.go index 0cb02f878222..392129940d5a 100644 --- a/pkg/webhooks/webhooks.go +++ b/pkg/webhooks/webhooks.go @@ -25,10 +25,6 @@ import ( "knative.dev/pkg/webhook/resourcesemantics/defaulting" "knative.dev/pkg/webhook/resourcesemantics/validation" - corev1alpha5 "sigs.k8s.io/karpenter/pkg/apis/v1alpha5" - - "github.com/aws/karpenter/pkg/apis/v1alpha1" - "github.com/aws/karpenter/pkg/apis/v1alpha5" "github.com/aws/karpenter/pkg/apis/v1beta1" ) @@ -60,7 +56,5 @@ func NewCRDValidationWebhook(ctx context.Context, _ configmap.Watcher) *controll } var Resources = map[schema.GroupVersionKind]resourcesemantics.GenericCRD{ - v1alpha1.SchemeGroupVersion.WithKind("AWSNodeTemplate"): &v1alpha1.AWSNodeTemplate{}, - corev1alpha5.SchemeGroupVersion.WithKind("Provisioner"): &v1alpha5.Provisioner{}, - v1beta1.SchemeGroupVersion.WithKind("EC2NodeClass"): &v1beta1.EC2NodeClass{}, + v1beta1.SchemeGroupVersion.WithKind("EC2NodeClass"): &v1beta1.EC2NodeClass{}, } diff --git a/test/pkg/environment/common/expectations.go b/test/pkg/environment/common/expectations.go index 37f23f8b3513..bc0cfae5a7ca 100644 --- a/test/pkg/environment/common/expectations.go +++ b/test/pkg/environment/common/expectations.go @@ -679,11 +679,11 @@ func (env *Environment) GetDaemonSetCount(np *corev1beta1.NodePool) int { return lo.CountBy(daemonSetList.Items, func(d appsv1.DaemonSet) bool { p := &v1.Pod{Spec: d.Spec.Template.Spec} - nodeTemplate := pscheduling.NewNodeClaimTemplate(np) - if err := scheduling.Taints(nodeTemplate.Spec.Taints).Tolerates(p); err != nil { + nodeClaimTemplate := pscheduling.NewNodeClaimTemplate(np) + if err := scheduling.Taints(nodeClaimTemplate.Spec.Taints).Tolerates(p); err != nil { return false } - if err := nodeTemplate.Requirements.Compatible(scheduling.NewPodRequirements(p), scheduling.AllowUndefinedWellKnownLabels); err != nil { + if err := nodeClaimTemplate.Requirements.Compatible(scheduling.NewPodRequirements(p), scheduling.AllowUndefinedWellKnownLabels); err != nil { return false } return true diff --git a/tools/allocatable-diff/go.mod b/tools/allocatable-diff/go.mod index c04c8db3ca1a..e212f0ae58cb 100644 --- a/tools/allocatable-diff/go.mod +++ b/tools/allocatable-diff/go.mod @@ -3,19 +3,19 @@ module github.com/aws/karpenter/tools/allocatable-diff go 1.21 require ( - github.com/aws/karpenter v0.32.2 - github.com/aws/karpenter-core v0.32.2 - github.com/samber/lo v1.38.1 + github.com/aws/karpenter v0.32.2-0.20231205002141-fbdb621104b4 + github.com/samber/lo v1.39.0 k8s.io/api v0.28.4 k8s.io/apimachinery v0.28.4 k8s.io/client-go v0.28.4 sigs.k8s.io/controller-runtime v0.16.3 + sigs.k8s.io/karpenter v0.32.2-0.20231205004259-6a5c00a354b3 ) require ( contrib.go.opencensus.io/exporter/ocagent v0.7.1-0.20200907061046-05415f1de66d // indirect contrib.go.opencensus.io/exporter/prometheus v0.4.2 // indirect - github.com/aws/aws-sdk-go v1.48.0 // indirect + github.com/aws/aws-sdk-go v1.48.11 // indirect github.com/beorn7/perks v1.0.1 // indirect github.com/blang/semver/v4 v4.0.0 // indirect github.com/blendle/zapdriver v1.3.1 // indirect @@ -71,11 +71,11 @@ require ( golang.org/x/exp v0.0.0-20231006140011-7918f672742d // indirect golang.org/x/net v0.17.0 // indirect golang.org/x/oauth2 v0.13.0 // indirect - golang.org/x/sync v0.4.0 // indirect - golang.org/x/sys v0.13.0 // indirect + golang.org/x/sync v0.5.0 // indirect + golang.org/x/sys v0.14.0 // indirect golang.org/x/term v0.13.0 // indirect golang.org/x/text v0.14.0 // indirect - golang.org/x/time v0.3.0 // indirect + golang.org/x/time v0.5.0 // indirect gomodules.xyz/jsonpatch/v2 v2.4.0 // indirect google.golang.org/api v0.146.0 // indirect google.golang.org/appengine v1.6.8 // indirect @@ -86,7 +86,7 @@ require ( gopkg.in/inf.v0 v0.9.1 // indirect gopkg.in/yaml.v2 v2.4.0 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect - k8s.io/apiextensions-apiserver v0.28.3 // indirect + k8s.io/apiextensions-apiserver v0.28.4 // indirect k8s.io/cloud-provider v0.28.4 // indirect k8s.io/component-base v0.28.4 // indirect k8s.io/csi-translation-lib v0.28.4 // indirect diff --git a/tools/allocatable-diff/go.sum b/tools/allocatable-diff/go.sum index 577956f47682..5b9272309e8e 100644 --- a/tools/allocatable-diff/go.sum +++ b/tools/allocatable-diff/go.sum @@ -48,12 +48,10 @@ github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8V github.com/antihax/optional v1.0.0/go.mod h1:uupD/76wgC+ih3iEmQUL+0Ugr19nfwCT1kdvxnR2qWY= github.com/avast/retry-go v3.0.0+incompatible h1:4SOWQ7Qs+oroOTQOYnAHqelpCO0biHSxpiH9JdtuBj0= github.com/avast/retry-go v3.0.0+incompatible/go.mod h1:XtSnn+n/sHqQIpZ10K1qAevBhOOCWBLXXy3hyiqqBrY= -github.com/aws/aws-sdk-go v1.48.0 h1:1SeJ8agckRDQvnSCt1dGZYAwUaoD2Ixj6IaXB4LCv8Q= -github.com/aws/aws-sdk-go v1.48.0/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk= -github.com/aws/karpenter v0.32.2 h1:Yrr/kvkYdhh11594BtaJJwgCq5Jf2rVl5KeMYb95CTQ= -github.com/aws/karpenter v0.32.2/go.mod h1:NPAqHJItiXAfBVAIVGFbyCse79UzX2dpbjZRBtWYX1E= -github.com/aws/karpenter-core v0.32.2 h1:jUKhC2qrk+/ZO9okx4iIE9Ju6f0gFG90eSPppJtqbDk= -github.com/aws/karpenter-core v0.32.2/go.mod h1:RNih2g6qCiah8rFaZ7HkmClIK66Hjj38z3DbWnWGM2w= +github.com/aws/aws-sdk-go v1.48.11 h1:9YbiSbaF/jWi+qLRl+J5dEhr2mcbDYHmKg2V7RBcD5M= +github.com/aws/aws-sdk-go v1.48.11/go.mod h1:LF8svs817+Nz+DmiMQKTO3ubZ/6IaTpq3TjupRn3Eqk= +github.com/aws/karpenter v0.32.2-0.20231205002141-fbdb621104b4 h1:GbFqWkP2rUNKEtzAqJ8HJTCLQAVdlUdyu8XZrEpjEFU= +github.com/aws/karpenter v0.32.2-0.20231205002141-fbdb621104b4/go.mod h1:7tIW1X81+Q+b177/BnypK6Ir7GiL5KKtR6pU6GphnAo= github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q= github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= @@ -259,10 +257,10 @@ github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ= github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= github.com/mwitkow/go-conntrack v0.0.0-20190716064945-2f068394615f/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U= -github.com/onsi/ginkgo/v2 v2.13.0 h1:0jY9lJquiL8fcf3M4LAXN5aMlS/b2BV86HFFPCPMgE4= -github.com/onsi/ginkgo/v2 v2.13.0/go.mod h1:TE309ZR8s5FsKKpuB1YAQYBzCaAfUgatB/xlT/ETL/o= -github.com/onsi/gomega v1.29.0 h1:KIA/t2t5UBzoirT4H9tsML45GEbo3ouUnBHsCfD2tVg= -github.com/onsi/gomega v1.29.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ= +github.com/onsi/ginkgo/v2 v2.13.2 h1:Bi2gGVkfn6gQcjNjZJVO8Gf0FHzMPf2phUei9tejVMs= +github.com/onsi/ginkgo/v2 v2.13.2/go.mod h1:XStQ8QcGwLyF4HdfcZB8SFOS/MWCgDuXMSBe6zrvLgM= +github.com/onsi/gomega v1.30.0 h1:hvMK7xYz4D3HapigLTeGdId/NcfQx1VHMJc60ew99+8= +github.com/onsi/gomega v1.30.0/go.mod h1:9sxs+SwGrKI0+PWe4Fxa9tFQQBG5xSsSbMXOI8PPpoQ= github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc= github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ= github.com/pelletier/go-toml/v2 v2.1.0 h1:FnwAJ4oYMvbT/34k9zzHuZNrhlz48GB3/s6at6/MHO4= @@ -314,8 +312,8 @@ github.com/rogpeppe/go-internal v1.3.0/go.mod h1:M8bDsm7K2OlrFYOpmOWEs/qY81heoFR github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ= github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog= github.com/russross/blackfriday/v2 v2.1.0/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM= -github.com/samber/lo v1.38.1 h1:j2XEAqXKb09Am4ebOg31SpvzUTTs6EN3VfgeLUhPdXM= -github.com/samber/lo v1.38.1/go.mod h1:+m/ZKRl6ClXCE2Lgf3MsQlWfh4bn1bz6CXEOxnEXnEA= +github.com/samber/lo v1.39.0 h1:4gTz1wUhNYLhFSKl6O+8peW0v2F4BCY034GRpU9WnuA= +github.com/samber/lo v1.39.0/go.mod h1:+m/ZKRl6ClXCE2Lgf3MsQlWfh4bn1bz6CXEOxnEXnEA= github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo= github.com/sirupsen/logrus v1.4.2/go.mod h1:tLMulIdttU9McNUspp0xgXVQah82FyeX6MwdIuYE2rE= github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88= @@ -462,8 +460,8 @@ golang.org/x/sync v0.0.0-20201020160332-67f06af15bc9/go.mod h1:RxMgew5VJxzue5/jJ golang.org/x/sync v0.0.0-20201207232520-09787c993a3a/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220601150217-0de741cfad7f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= golang.org/x/sync v0.0.0-20220722155255-886fb9371eb4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM= -golang.org/x/sync v0.4.0 h1:zxkM55ReGkDlKSM+Fu41A+zmbZuaPVbGMzvvdUPznYQ= -golang.org/x/sync v0.4.0/go.mod h1:FU7BRWz2tNW+3quACPkgCx/L+uEAv1htQ0V83Z9Rj+Y= +golang.org/x/sync v0.5.0 h1:60k92dhOjHxJkrqnwsfl8KuaHbn/5dl0lUPUklKo3qE= +golang.org/x/sync v0.5.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk= golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY= @@ -507,8 +505,8 @@ golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a/go.mod h1:oPkhp1MJrh7nUepCBc golang.org/x/sys v0.0.0-20220708085239-5a0f0661e09d/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220722155257-8c9f86f7a55f/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220908164124-27713097b956/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= -golang.org/x/sys v0.13.0 h1:Af8nKPmuFypiUBjVoU9V20FiaFXOcuZI21p0ycVYYGE= -golang.org/x/sys v0.13.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= +golang.org/x/sys v0.14.0 h1:Vz7Qs629MkJkGyHxUlRHizWJRG2j8fbQKjELVSNhy7Q= +golang.org/x/sys v0.14.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo= golang.org/x/term v0.0.0-20210927222741-03fcf44c2211/go.mod h1:jbD1KX2456YbFQfuXm/mYQcufACuNUgVhRMnK/tPxf8= golang.org/x/term v0.13.0 h1:bb+I9cTfFazGW51MZqBVmZy7+JEJMouUHTUSKVQLBek= @@ -526,8 +524,8 @@ golang.org/x/text v0.14.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU= golang.org/x/time v0.0.0-20181108054448-85acf8d2951c/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= golang.org/x/time v0.0.0-20191024005414-555d28b269f0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= -golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4= -golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ= +golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk= +golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM= golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ= golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY= @@ -704,8 +702,8 @@ honnef.co/go/tools v0.0.1-2020.1.3/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9 honnef.co/go/tools v0.0.1-2020.1.4/go.mod h1:X/FiERA/W4tHapMX5mGpAtMSVEeEUOyHaw9vFzvIQ3k= k8s.io/api v0.28.4 h1:8ZBrLjwosLl/NYgv1P7EQLqoO8MGQApnbgH8tu3BMzY= k8s.io/api v0.28.4/go.mod h1:axWTGrY88s/5YE+JSt4uUi6NMM+gur1en2REMR7IRj0= -k8s.io/apiextensions-apiserver v0.28.3 h1:Od7DEnhXHnHPZG+W9I97/fSQkVpVPQx2diy+2EtmY08= -k8s.io/apiextensions-apiserver v0.28.3/go.mod h1:NE1XJZ4On0hS11aWWJUTNkmVB03j9LM7gJSisbRt8Lc= +k8s.io/apiextensions-apiserver v0.28.4 h1:AZpKY/7wQ8n+ZYDtNHbAJBb+N4AXXJvyZx6ww6yAJvU= +k8s.io/apiextensions-apiserver v0.28.4/go.mod h1:pgQIZ1U8eJSMQcENew/0ShUTlePcSGFq6dxSxf2mwPM= k8s.io/apimachinery v0.28.4 h1:zOSJe1mc+GxuMnFzD4Z/U1wst50X28ZNsn5bhgIIao8= k8s.io/apimachinery v0.28.4/go.mod h1:wI37ncBvfAoswfq626yPTe6Bz1c22L7uaJ8dho83mgg= k8s.io/client-go v0.28.4 h1:Np5ocjlZcTrkyRJ3+T3PkXDpe4UpatQxj85+xjaD2wY= @@ -731,6 +729,8 @@ sigs.k8s.io/controller-runtime v0.16.3 h1:2TuvuokmfXvDUamSx1SuAOO3eTyye+47mJCigw sigs.k8s.io/controller-runtime v0.16.3/go.mod h1:j7bialYoSn142nv9sCOJmQgDXQXxnroFU4VnX/brVJ0= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd h1:EDPBXCAspyGV4jQlpZSudPeMmr1bNJefnuqLsRAsHZo= sigs.k8s.io/json v0.0.0-20221116044647-bc3834ca7abd/go.mod h1:B8JuhiUyNFVKdsE8h686QcCxMaH6HrOAZj4vswFpcB0= +sigs.k8s.io/karpenter v0.32.2-0.20231205004259-6a5c00a354b3 h1:iTCm2GfvpG+x8GedLc2NIr5DtV6AbDf5V5NgV9CtcA0= +sigs.k8s.io/karpenter v0.32.2-0.20231205004259-6a5c00a354b3/go.mod h1:7hPB7kdImFHAFp7pqPUqgBDOrh8GEcRnHT5uIlsXMKA= sigs.k8s.io/structured-merge-diff/v4 v4.3.0 h1:UZbZAZfX0wV2zr7YZorDz6GXROfDFj6LvqCRm4VUVKk= sigs.k8s.io/structured-merge-diff/v4 v4.3.0/go.mod h1:N8hJocpFajUSSeSJ9bOZ77VzejKZaXsTtZo4/u7Io08= sigs.k8s.io/yaml v1.3.0 h1:a2VclLzOGrwOHDiV8EfBGhvjHvP46CtW5j6POvhYGGo= diff --git a/tools/allocatable-diff/main.go b/tools/allocatable-diff/main.go index c95c18c88309..62945085981a 100644 --- a/tools/allocatable-diff/main.go +++ b/tools/allocatable-diff/main.go @@ -32,10 +32,9 @@ import ( "sigs.k8s.io/controller-runtime/pkg/client/config" "sigs.k8s.io/controller-runtime/pkg/manager" - "github.com/aws/karpenter-core/pkg/apis/v1alpha5" - corecloudprovider "github.com/aws/karpenter-core/pkg/cloudprovider" - coreoperator "github.com/aws/karpenter-core/pkg/operator" - nodepoolutil "github.com/aws/karpenter-core/pkg/utils/nodepool" + corecloudprovider "sigs.k8s.io/karpenter/pkg/cloudprovider" + coreoperator "sigs.k8s.io/karpenter/pkg/operator" + "github.com/aws/karpenter/pkg/apis/v1alpha1" "github.com/aws/karpenter/pkg/cloudprovider" "github.com/aws/karpenter/pkg/operator" @@ -90,11 +89,7 @@ func main() { "karpenter.sh/discovery": clusterName, }, })))) - instanceTypes := lo.Must(cloudProvider.GetInstanceTypes(ctx, nodepoolutil.New(&v1alpha5.Provisioner{ - Spec: v1alpha5.ProvisionerSpec{ - Provider: raw, - }, - }))) + instanceTypes := lo.Must(cloudProvider.GetInstanceTypes(ctx, nil)) // Write the header information into the CSV lo.Must0(w.Write([]string{"Instance Type", "Expected Capacity", "", "", "Expected Allocatable", "", "", "Actual Capacity", "", "", "Actual Allocatable", ""})) diff --git a/website/content/en/preview/reference/metrics.md b/website/content/en/preview/reference/metrics.md index d1405fccfe67..cdb7e7fff63c 100644 --- a/website/content/en/preview/reference/metrics.md +++ b/website/content/en/preview/reference/metrics.md @@ -90,14 +90,6 @@ Number of nodeclaims registered in total by Karpenter. Labeled by the owning nod ### `karpenter_nodeclaims_terminated` Number of nodeclaims terminated in total by Karpenter. Labeled by reason the nodeclaim was terminated and the owning nodepool. -## Nodepool Metrics - -### `karpenter_nodepool_limit` -The nodepool limits are the limits specified on the provisioner that restrict the quantity of resources provisioned. Labeled by nodepool name and resource type. - -### `karpenter_nodepool_usage` -The nodepool usage is the amount of resources that have been provisioned by a particular nodepool. Labeled by nodepool name and resource type. - ## Provisioner Metrics ### `karpenter_provisioner_scheduling_duration_seconds` @@ -106,13 +98,21 @@ Duration of scheduling process in seconds. ### `karpenter_provisioner_scheduling_simulation_duration_seconds` Duration of scheduling simulations used for deprovisioning and provisioning in seconds. +## Nodepool Metrics + +### `karpenter_nodepool_limit` +The nodepool limits are the limits specified on the nodepool that restrict the quantity of resources provisioned. Labeled by nodepool name and resource type. + +### `karpenter_nodepool_usage` +The nodepool usage is the amount of resources that have been provisioned by a particular nodepool. Labeled by nodepool name and resource type. + ## Nodes Metrics ### `karpenter_nodes_allocatable` Node allocatable are the resources allocatable by nodes. ### `karpenter_nodes_created` -Number of nodes created in total by Karpenter. Labeled by owning provisioner. +Number of nodes created in total by Karpenter. Labeled by owning nodepool. ### `karpenter_nodes_leases_deleted` Number of deleted leaked leases. @@ -121,7 +121,7 @@ Number of deleted leaked leases. Node system daemon overhead are the resources reserved for system overhead, the difference between the node's capacity and allocatable values are reported by the status. ### `karpenter_nodes_terminated` -Number of nodes terminated in total by Karpenter. Labeled by owning provisioner. +Number of nodes terminated in total by Karpenter. Labeled by owning nodepool. ### `karpenter_nodes_termination_time_seconds` The time taken between a node's deletion request and the removal of its finalizer @@ -144,7 +144,7 @@ Node total pod requests are the resources requested by non-DaemonSet pods bound The time from pod creation until the pod is running. ### `karpenter_pods_state` -Pod state is the current state of pods. This metric can be used several ways as it is labeled by the pod name, namespace, owner, node, provisioner name, zone, architecture, capacity type, instance type and pod phase. +Pod state is the current state of pods. This metric can be used several ways as it is labeled by the pod name, namespace, owner, node, nodepool name, zone, architecture, capacity type, instance type and pod phase. ## Cloudprovider Metrics