What is the intended Karpenter support path for EKS-optimized AL2027 AMIs?

[tanmaymohan]

The [EKS AL2027 preview discussion](awslabs/amazon-eks-ami#2849) notes that AL2027 uses nodeadm and that the Karpenter AL2023 AMI family can currently generate compatible user data.

I validated this on an existing EKS 1.34 cluster managed by Karpenter using custom AL2027 AMIs and an explicit AMI ID with amiFamily: AL2023.

Tested configurations:

• ARM64: t4g.large
• x86_64 / NVIDIA: g6.xlarge and gr6.4xlarge

All nodes joined successfully, and networking, storage, system DaemonSets, GPU device plugin registration, CUDA workloads, inference, and reboot recovery worked as expected.

As AL2027 progresses toward general availability, what is the intended Karpenter integration path?

• Dedicated AL2027 AMI family and alias for native support?
• Continued use of AL2023 given the shared nodeadm bootstrap mechanism?
• Custom AMI family with explicit AMI selectors and user data?

This is not a bug report; the explicit-AMI approach worked in my testing. I'm looking for guidance on the approach most likely to remain supported as AL2027 evolves.

I'd be happy to help validate or contribute to the integration once the intended direction is clear.

[mikeflame]

I'd keep amiFamily: AL2023 with explicit AMI selection for now. Aliases resolve through EKS SSM parameters, so I wouldn't expect an AL2027 alias before EKS publishes those. I couldn't find an AL2027 issue, PR or code change in this repo yet. What follows is what the code does today (v1.15.1) and how the AL2023 transition went. It is not a maintainer position.

Why AL2023 holds up. amiFamily only controls the user data and the default block devices. It doesn't affect AMI discovery (docs). For AL2023, Karpenter emits an application/node.eks.aws MIME part with a node.eks.aws/v1alpha1 NodeConfig. It holds the cluster name, endpoint, CA and service CIDR, plus your spec.kubelet inline together with maxPods, clusterDNS and registerWithTaints, and --node-labels as a flag (nodeadm.go). The default root volume is 20Gi, gp3, encrypted, on /dev/xvda (al2023.go). AL2027's nodeadm still serves only v1alpha1 (nodeadm/api), and the AL2027 Packer template also puts root on /dev/xvda (template.pkr.hcl). The family's only other special cases are dropping a1.* instance types (compatibility.go) and looking up the cluster CIDR for nodeadm (validation.go). So nothing Karpenter generates is AL2023-only today.

What you give up without an alias. al2023@… reads /aws/service/eks/optimized-ami/<k8s>/amazon-linux-2023/<arch>/<standard|nvidia|neuron>/… from SSM and attaches GPU or accelerator requirements to each AMI (al2023.go). With id, tags, name or ssmParameter terms, Karpenter only knows the architecture (types.go), and it keeps the newest AMI per requirement set (ami.go). So if you put a standard and an NVIDIA AL2027 AMI of the same architecture in one EC2NodeClass, only one of them is used. Use one EC2NodeClass per variant and route GPU instance types with karpenter.k8s.aws/instance-gpu-count, as the docs already advise for custom AMIs (nodeclasses.md).

AL2027 differences that do reach Karpenter:

• Node names. On AL2027, nodeadm names the node after the instance ID unless the InstanceIdNodeName feature gate is set (featuregates.go, kubelet/config.go). Karpenter's NodeConfig doesn't set that gate.
  • Karpenter itself copes, because it links nodes by providerID (core).
  • The node role, though, has to map to system:node:{{SessionName}}, which is the EC2 access entry type in the EKS notice. Karpenter's getting-started mapping uses {{EC2PrivateDNSName}} (step02).
  • If your AL2027 nodes registered as i-…, you already have this mapping. A separate spec.role on the AL2027 EC2NodeClass keeps it apart from the mapping your AL2023 nodes use.
• DNS. On AL2027, nodeadm points kubelet at systemd-resolved's uplink resolv.conf (config.go). Karpenter passes spec.kubelet through as-is, so a resolvConf set there overrides it.
• Accelerated AMIs aren't upstream yet: the Makefile rejects enable_accelerator for al2027 (Makefile). GPU AMIs are self-built for now, as in your test.

Custom. Custom generates no user data and no default block device mappings (custom.go). You'd write the whole NodeConfig yourself, including the karpenter.sh/unregistered:NoExecute taint, and keep it in sync with spec.kubelet by hand (docs). Karpenter also stops adding the karpenter.sh/do-not-sync-taints label, which makes startup taints racy (launchtemplate.go). That's only worth it if the AL2023 output stops working on AL2027.

Alias and GA. al2027@… doesn't pass validation today. The CRD only accepts al2, al2023, bottlerocket and the Windows families (v1.15.1, unchanged on main). By analogy with AL2023, an alias would read something like /aws/service/eks/optimized-ami/<k8s>/amazon-linux-2027/<arch>/<variant>/recommended/image_id. I couldn't find any such parameter published. The preview notice gives a single AMI ID (1.37, x86_64, us-west-2), and the latest AMI release notes (v20261026) don't mention AL2027.

Here's how AL2023 went. This is precedent only, not a commitment:

• The EKS tech preview notice published SSM paths and said to use AL2023 with Karpenter "via a custom AMI" until out-of-box support arrived (amazon-eks-ami#1672). In the meantime a maintainer pointed people at the Custom family (comment), since no existing family generated nodeadm config.
• The first AL2023 EKS AMIs shipped on 2024-02-14. The AL2023 family, with SSM-based discovery, came in #5604 and was released in v0.35.0 on 2024-02-29.
• The al2023@ alias came later, with the v1 API (#6450, v1.0.0).

AL2023 needed its own family because the bootstrap format changed. AL2027 doesn't change it, so whether it gets a new family or just an alias on the existing nodeadm path is the maintainers' call. The docs already describe a similar arrangement for Ubuntu on AL2, with the caveat that "this compatibility isn't guaranteed long term" (nodeclasses.md). I'd treat AL2023-for-AL2027 the same way.

What I'd run in the meantime (valid against the v1.15.1 CRDs):

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: al2027-nvidia
spec:
  amiFamily: AL2023               # nodeadm user data + /dev/xvda root default
  amiSelectorTerms:
    - id: ami-0123456789abcdef0   # your AL2027 NVIDIA x86_64 build
  role: KarpenterNodeRole-al2027  # mapped to system:node:{{SessionName}}
  subnetSelectorTerms:
    - tags:
        karpenter.sh/discovery: my-cluster
  securityGroupSelectorTerms:
    - tags:
        karpenter.sh/discovery: my-cluster
# NodePool pointing at it:
#   requirements:
#     - key: karpenter.k8s.aws/instance-gpu-count
#       operator: Exists

Once EKS publishes AL2027 parameters, you can swap id for ssmParameter: <path>, still with amiFamily: AL2023 and one variant per EC2NodeClass. That picks up new releases without waiting for an alias. Karpenter re-reads custom parameters every 5 minutes (types.go), and a new AMI drifts existing nodes (disruption.md).