[EKS] Karpenter inside control plane

[James-Quigley]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Tell us about your request
It would be really useful if EKS came pre-installed with Karpenter running in the control plane. Currently with Karpenter you have to first setup at least one nodegroup for Karpenter, then Karpenter can manage the rest.

If the control plane ran Karpenter for you, then there wouldn't be a need for any management of nodegroups for customers.

Which service(s) is this request for?
EKS

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
As an EKS cluster manager, when creating a cluster I have to bootstrap it with various tools as well as enough resources to run workloads. Also as a user of Karpenter, this requires setting up a nodegroup (managed), installing Karpenter and limiting it to run on that nodegroup before deploying other workloads.

This also means that as a cluster manager I need to be aware of the two different node setups: managed nodegroups and Karpenter nodes.

Are you currently working around this issue?
Currently we deploy managed nodegroups, then karpenter.

Additional context
Sort of similar to #47

[krmayankk]

Karpenter doesnt use ASG's and hence its availability is of paramount importance (equal to the availability requirement of ASG itself). Is that the main reason you use managed node group to run Karpenter ? You could technically run Karpenter in a node group managed by ASG as well. Would help to understand why that would not work ?
Overall I support it being available as pre-installed and managed by EKS, but looking for how other teams are currently using it ? Also is Karpenter production ready ?

[stevehipwell]

@krmayankk Karpenter has been ready for production for almost a year at this point.

I'd love to see Karpenter running as part of the control plane, otherwise I have to managed separate node groups which don't have the best ergonomics for upgrades.

[runningman84]

I run karpenter using eks fargate which is a good solution until karpenter moves to the control plane itself.

[HenryYanTR]

Same here, running Karpenter in fargate. Feel it's a perfect fit for EKS control plane.

[danielloader]

I'd feel happier if I could use fargate spot instances but it's a compromise.

[abebars]

@mikestef9 Any thoughts on when this is going to become available? Also happy to work on some testing when it's ready for that.

[KuldeepSharmaTR]

@mikestef9 Any thoughts on when this is going to become available? Also happy to work on some testing when it's ready for that.

I believe it will be available after coming re:invent..!!

[kasvith]

I was also looking for this, we were setting up a ng only for karpenter, but if it can run on control plane...the problem is gone

[duxing]

would love to EKS control plan enhancement on supporting dedicated hardware for karpenter control plane without us having to manage fargate or EKS node group.

maturity improvements like this would benefit the adoption for both EKS and karpenter.

[deimosfr]

Hi,

Could we have more information about the implementation? How will it work? What will be the required setup compared to current EKS etc...

Thanks

[zendz]

Hello,

Do you have any update this?.

I need this feature.

[kr3cj]

If we're already managing karpenter ourselves (using Fargate/MNGs/whatever), I'm curious how cutting over to the karpenter-inside-control-plane would look/work.

[FernandoMiguel]

If we're already managing karpenter ourselves (using Fargate/MNGs/whatever), I'm curious how cutting over to the karpenter-inside-control-plane would look/work.

i would expect us to provide our own provisioners, let the managed karpenter create new nodes, and set the existing ones as unscheduled, so workloads move

[Aym3nTN]

Any updates regarding this feature?

[kr3cj]

This new EKS Auto Mode looks related: https://aws.amazon.com/about-aws/whats-new/2024/12/amazon-eks-auto-mode/ since it supports NodeClass and NodePool.

[stevehipwell]

@kr3cj related but you have to give up a lot of control to get it.

[jalaziz]

This new EKS Auto Mode looks related: https://aws.amazon.com/about-aws/whats-new/2024/12/amazon-eks-auto-mode/ since it supports NodeClass and NodePool.

Seems like it is but Auto Mode is pretty restrictive compared to Karpenter because it uses BottleRokcet and doesn't allow you to customize the AMI or bootstrapping procedure in any way.

This is particularly a problem for us because we need support for nitro enclaves and it appears it's impossible to use EKS Auto Mode with nitro enclaves.

[stevehipwell]

For a general approach I'd be most concerned about the limited networking in EKS Auto. We already don't use add-ons due to the lack of customization; so IMHO EKS Auto is a move in the wrong direction for power users. But for K8s 101 usage, if we ignore the the networking limitations (no secondary CIDRs), EKS Auto looks like a good bet.

[gazal-k]

The objective of having Karpenter in the control plane as expressed in this feature request is to simplify management of EKS clusters. And Auto Mode does seem to deliver on that. There are trade offs like:

• Less customisable.
• Some added cost.

So, it may not be for everyone. But, it does seem to address this feature request 🤷

[jalaziz]

The objective of having Karpenter in the control plane as expressed in this feature request is to simplify management of EKS clusters. And Auto Mode does seem to deliver on that. There are trade offs like:

• Less customisable.
• Some added cost.

So, it may not be for everyone. But, it does seem to address this feature request 🤷

Sure, but the original ask would solve the same issue but without the restrictions. It very much seems like an over-engineered way to solve what was a narrowly scoped (relatively) feature request.

I appreciate what auto mode does for the simple use case, absolutely. It's just disappointing that AWS didn't poll more customers for actual use cases. Or maybe they did and simply decided it wasn't worth supporting their use cases.

[bbegw]

Definitely agree @jalaziz , I think having a simpler-to-manage version of EKS that's more restricted (Auto Mode) makes sense, but managed Karpenter is a really nice feature and I don't see why it couldn't be extended to non-Auto Mode clusters.