Have a way to select provision instances based on CPU type (AMD vs. Intel)

[sidprak]

Tell us about your request

Some of our workloads perform very poorly on AMD CPUs. We would love a way to exclude all AMD CPU instances from provisioning. Example:

- key: karpenter.k8s.aws/cpu-brand
      operator: NotIn
      values: [amd]

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?

Today, we have to do something like this and list all the instance families.

- key: karpenter.k8s.aws/instance-family
      operator: NotIn
      values: [c6a, m6a, r6a, c5a, m5a, r5a, c6ad, m6ad, r6ad, m5ad, r5ad, r5ad]

Are you currently working around this issue?

Above.

Additional Context

No response

Attachments

No response

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[jonathan-innis]

@sidprak Are you aware of a way to programmatically grab these details beyond just looking up the CPU details on the instance type description? From what I can tell, describe-instance-types doesn't specify this in any way so how have you become aware of which instance families support which CPU type

[sidprak]

I don't know of a way either 😞 I was just thinking of using the instance type and looking for the a somewhere before the ..

[jonathan-innis]

Oh! Got it. I didn't realize there was consistent naming across the instance types which always contained an "a" at the end if it's an AMD cpu type. That's helpful.

For this one, we're in a bit of flux right now due to a lot of other changes that are happening in the project so we're trying to reduce API changes right now until we get past #2964 and #2944. With that in mind, we'll keep tracking this issue but expanding the label selector set isn't something that's going to get added immediately.

It seems like you have a reasonable workaround for the time being.

[sidprak]

Makes sense! Yes, I'm able to workaround it. Just wanted to suggest this usability improvement.

I actually think we can generalize this further. Another very similar issue I have is automatically selecting the high-network throughput instances and that can be detected by looking for the n suffix before the dot. A similar thing can be said for d with local storage, z for high frequency CPUs, etc.

[NetanelK]

@sidprak
It looks like AWS API does have an option to add CPU manufacturer requirement.

Maybe it can implemented like GPU manufacturer requirement?

[ilkinmammadzada]

It would be definitely useful! Otherwise we need to track new ec2 type releases to add them to the list :)

[VariableExp0rt]

I took a look a this and thought I could perhaps attempt to implement it.

Oh! Got it. I didn't realize there was consistent naming across the instance types which always contained an "a" at the end if it's an AMD cpu type. That's helpful.

I notice that the newer P5 instance types don't have the a at the end but have AMD EPYC CPU's. Would it be enough to look at the ProcessorInfo.SupportedArchitectures and infer the brands that could satisfy the requirement given an instances' supported architecture? Something like

	// CPU Manufacturer
	if info.ProcessorInfo != nil && len(info.ProcessorInfo.SupportedArchitectures) == 1 {
		arch := aws.StringValue(info.ProcessorInfo.SupportedArchitectures[0])
		switch arch {
		case "x86_64":
			requirements.Get(v1alpha1.LabelInstanceCPUManufacturer).Insert(
				fmt.Sprint(v1alpha1.AMDCPUManufacturer),
				fmt.Sprint(v1alpha1.AWSCPUManufacturer),
				fmt.Sprint(v1alpha1.IntelCPUManufacturer),
			)
		case "arm64":
			requirements.Get(v1alpha1.LabelInstanceCPUManufacturer).Insert(
				fmt.Sprint(v1alpha1.AWSCPUManufacturer),
			)
		}
	}

There are however certain instance types that support 32-bit architecture that do not satisfy the above constraint. I'd be happy to give this a go if anyone has any additional ways in which the brand of the CPU can be set.