Add support for AMIs based on AmazonLinux2023 #7684

TiberiuGC · 2024-03-28T13:00:56Z

Description

This PR aims to make AL2023 based EKS-optimized AMIs supported in eksctl, specifically, adds support for general CRUD operations for both self-managed and EKS-managed nodegroups based on AL2023.

Successful integration test run.

Closes #7634

Checklist

Added tests that cover your change (if possible)
Added/modified documentation as required (such as the README.md, or the userdocs directory)
Manually tested
Made sure the title of the PR is a good description that can go into the release notes
(Core team) Added labels for change area (e.g. area/nodegroup) and kind (e.g. kind/improvement)

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

Backfilled missing tests for code in same general area 🎉
Refactored something and made the world a better place 🌟

thedavidwhiteside · 2024-03-29T23:12:50Z

I am interested in al2023 support as well. Looking forward for this feature.

pkg/apis/eksctl.io/v1alpha5/validation.go

yuxiang-zhang

LGTM 👍 ty for getting this out so quickly ⚡

a-hilaly

Great work on this @TiberiuGC ! Thank you very mcuh! I left a few questions and comments in-line

pkg/ami/auto_resolver.go

pkg/ami/ssm_resolver.go

pkg/apis/eksctl.io/v1alpha5/validation.go

a-hilaly · 2024-04-02T20:50:38Z

pkg/nodebootstrap/managed_al2.go

+			return err
+		}
+	}
+
 	return mw.Close()


Maybe we can just defer mw.Close() at the beginning of the function?

I don't think this is necessary. mw.Close() has mainly two purposes, both of which are not denied by an earlier failure within the function.

set mw.closed = true so that no further writes are allowed. However, the writer is declared within the scope of the function, so even if it fails early, there won't be additional writes.

return the last error that occurred across mv.Write() calls, if any. However, if the function fails earlier, we probably want to return that error instead and not silence it with the error from the deferred function.

I agree that it' unnecessary' given that this variable is local, however, i think it's generally considered a good Go practice to consistently defer Close() calls for reasons like consistency and clarity and minimizing unexpected behaviour.

This is non-blocking and can be addressed separately another day, maybe for all similar cases.

a-hilaly

This is great! Thanks Tibi!
/lgtm

* add support for AL2023 for EKS-managed and self-managed nodes * ensure AL2023 only supports containerd * add GPU related validations + small nits * add support for upgrades * add support for EFA * improve validations * fix lint and unit tests * update docs * add validation error for maxpods limitation * add integration tests for al2023 * improve validation message

* Safely access/mutate fargate coredns pod annotations Prior to this patch, the `pkg/fargate/coredns` package had some bits of code that accessed/mutated pod annotations assuming that they'll always be instantiated correctly. This patch adds utility functions to safely mutate and access fargate pod annotations. Signed-off-by: Amine Hilaly <hilalyamine@gmail.com> * Fix generating presigned URL for K8s authentication With `aws-sdk-go-v2@1.24.1`, API server requests containing URLs presigned by `sts.PresignClient` fail with an `Unauthorized` error. `aws-sdk-go-v2@1.24.1` adds an extra header `amz-sdk-request` to the generated request, but this header is not allow-listed by `aws-iam-authenticator` server running on the control plane. This is likely due to [this change](aws/aws-sdk-go-v2#2438) which reorders the middleware operations to execute `RetryMetricsHeader` before `Signing`. This changelist removes the `RetryMetricsHeader` middleware from the stack when constructing `sts.PresignClient`. * Add release notes for 0.168.0 * Prepare for next development iteration * Handle unordered public endpoint CIDRs from EKS in endpoint updates For some clusters, EKS can return the list of public endpoint CIDRs out of order, and won't allow updates where the incoming and current sets have set equality (i.e. regardless of order of CIDR entries). This change restores the set equality check that was removed in commit 72605fb and adds an additional test case to cover this case. * Fix outdated links * Fix StringLike condition key for ebsCSIController IAM policy The IAM condition key StringLike was used incorrectly in the policy and it doesn't work with wildcard (*) in the key itself. Wildcard is only supported in the value of the key. This fixes issue in cases where a volume dynamically provisioned via the older in-tree CSI plugin is being deleted by the new EBS CSI driver, because such volumes don't have the tags used in the policy. The changes made are inspired from the AWS managed AmazonEBSCSIDriverPolicy. * Fix coredns pdb preventing cluster deletion * Add support for EKS 1.29 * Add release notes for 0.169.0 * Prepare for next development iteration * Update arm-support.md * Expand Karpenter settings.aws block to settings for v0.33.0 and greater * Update stale.yml * docs(fargate): eksctl update command is deprecated * Fix deleting cluster sometimes drain managed nodegroups * Update userdocs on nodegroups * Update release drafter template * Add support for Access Entry Type * Update access entries userdocs * Add release notes for 0.170.0 * Prepare for next development iteration * Improve userdocs layout * Announce eksctl Support Status Update * Add support for Ubuntu 22.04 based EKS images Ubuntu switched to 22.04 (Jammy) based images for EKS >= 1.29 . Add support for that here. * Add release notes for 0.171.0 * Prepare for next development iteration * Changed the error to more understandable when the region code isn't set * Fix checks for updated addon versions When executing `eksctl get addons` an available update is not shown if the only change to the version string is the `eksbuild` number, e.g. `v1.0.0-eksbuild.2` is not shown as an update of `v1.0.0-eksbuild.1`. The `findNewerVersions` func in `pkg/actions/get.go` had code to explicitly ignore anything after the patch number. So `v1.1.0-eksbuild.1` and `v1.1.0-eksbuild.4` were both converted to `1.1.0` and considered equal. This fix removes the code that explicitly ignores the `-eksbuild.x` portion of the version and allows the `semver` package to compare the full version information. Tests have also been updated to better match the version strings used by AWS addons. * Extract common steps setting up build environment from workflows * Disable slack notifications * Bump actions/cache from 3.3.2 to 4.0.0 * Update userdocs dependencies * Update schema for new AMI family * Update go dependencies * Bump k8s dependencies to v0.29 Signed-off-by: Amine Hilaly <hilalyamine@gmail.com> * Replace all deprecated sets.String with generic sets.Set * Update actions dependencies Update go version to 1.21 in actions * Update build image manifest, tag file and workflows * Update build image go version to 1.21 Update build image manifest, tag file and workflows * Revert removing RetryMetricsHeader in presigned requests * Add release notes for 0.172.0 * Revert misdeleted checkout step for publishing release * Add integ test without build target * Remove unused slack token * Remove slackToken * Prepare for next development iteration * Bump dependencies for Dependabot alerts Bump github.com/go-git/go-git/v5 from 5.4.2 to 5.11.0 Bumps [github.com/go-git/go-git/v5](https://github.com/go-git/go-git) from 5.4.2 to 5.11.0. - [Release notes](https://github.com/go-git/go-git/releases) - [Commits](go-git/go-git@v5.4.2...v5.11.0) --- updated-dependencies: - dependency-name: github.com/go-git/go-git/v5 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Bump helm.sh/helm/v3 from 3.14.0 to 3.14.2 Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.0 to 3.14.2. - [Release notes](https://github.com/helm/helm/releases) - [Commits](helm/helm@v3.14.0...v3.14.2) --- updated-dependencies: - dependency-name: helm.sh/helm/v3 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com> * Fix generate-internal-groups.sh permission * Update dependencies * Make EKS 1.29 default * Automate PR with release notes draft * Remove inline ELB and CloudWatch policies from Cluster Role (eksctl-io#7603) * remove inline elb and cloud watch policy eksctl-io#7139 * fixed unit tests for removing extra inline policies * Update docs with Ubuntu 20.04 supported in EKS <= 1.29 (eksctl-io#7618) docs: ubuntu 20.04 supported in EKS <= 1.29 This adds a note that ubuntu focal (20.04) is supported up to EKS 1.29 * Bump github.com/cloudflare/circl from 1.3.3 to 1.3.7 (eksctl-io#7591) Bumps [github.com/cloudflare/circl](https://github.com/cloudflare/circl) from 1.3.3 to 1.3.7. - [Release notes](https://github.com/cloudflare/circl/releases) - [Commits](cloudflare/circl@v1.3.3...v1.3.7) --- updated-dependencies: - dependency-name: github.com/cloudflare/circl dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Add profile flag support for get and delete access entry * Fix caching credentials with assummed role MFA session * Remove dependabot from contributors * Use eksctl-bot token for release draft PR (eksctl-io#7629) * Add release notes for v0.173.0 * Prepare for next development iteration * Bump peter-evans/create-pull-request to fix error in GH API * Update dependabot version updates frequency * Fix EFA network interface device index assignment * Remove goreleaser from build deps Update build image manifest, tag file and workflows * Replace make build-all with goreleaser action * Inline script files in publish release workflows * Rename release workflows * Add release notes for v0.174.0 * Fix typo in publish-release.yaml * Prepare for next development iteration * Upgrade with explicit version if release version is up-to-date * Test Bottlerocket node upgrade and verify version * Add release notes for v0.175.0 (eksctl-io#7669) * Add release notes for v0.175.0 * remove empty acknowledgements section --------- Co-authored-by: yuxiang-zhang <23327251+yuxiang-zhang@users.noreply.github.com> Co-authored-by: Tibi <110664232+TiberiuGC@users.noreply.github.com> * Prepare for next development iteration (eksctl-io#7671) * Bump dependencies (eksctl-io#7668) * bump dependencies * update mocks * fix lint * bump helm * Aim for namespace uniqueness across parallel specs (eksctl-io#7680) ensure namespace uniqueness across parallel specs * Include MixedInstancesPolicy LaunchTemplate for validation * Allow GPU instance types for Windows nodes (eksctl-io#7681) * allow GPU instance type for Windows nodes * update unit test for case gpus:0 * Display full draft release notes in PR description (eksctl-io#7686) Update release-drafter.yaml * Bump mkdocs version (eksctl-io#7696) bump mkdocs version * Add support for AMIs based on AmazonLinux2023 (eksctl-io#7684) * add support for AL2023 for EKS-managed and self-managed nodes * ensure AL2023 only supports containerd * add GPU related validations + small nits * add support for upgrades * add support for EFA * improve validations * fix lint and unit tests * update docs * add validation error for maxpods limitation * add integration tests for al2023 * improve validation message * [EKSCTL create cluster command] Authorise self-managed nodes via `aws-auth configmap` when EKS access entries are disabled (eksctl-io#7698) * Disable access entry creation for self-managed nodes on clusters with CONFIG_MAP only * fix logic for updating aws-auth configmap * Enforce `authenticationMode:CONFIG_MAP` on Outposts (eksctl-io#7699) Make authenticationMode:CONFIG_MAP default on Outposts * Add release notes for v0.176.0 (eksctl-io#7672) Co-authored-by: TiberiuGC <110664232+TiberiuGC@users.noreply.github.com> * Prepare for next development iteration * Bump dependencies Closes eksctl-io#7694 eksctl-io#7693 eksctl-io#7692 eksctl-io#7691 eksctl-io#7690 eksctl-io#7689 eksctl-io#7688 eksctl-io#7687 eksctl-io#7679 eksctl-io#7678 eksctl-io#7676 eksctl-io#7673 eksctl-io#7581 eksctl-io#7579 eksctl-io#7577 eksctl-io#7576 * Update build image tag * Bump dependencies * Fix arn build logic to support different aws partitions * Fix reusing instanceRoleARN for nodegroups authorized with access entries This changelist changes the design of creating access entries for self-managed nodegroups that use a pre-existing instanceRoleARN by creating the access entry resource outside of the CloudFormation stack by making a separate call to the AWS API. When deleting such a nodegroup, it's the user's responsibility to also delete the corresponding access entry when no more nodegroups are associated with it. This is because eksctl cannot tell if an access entry resource is still in use by non-eksctl created self-managed nodegroups. Self-managed nodegroups not using a pre-existing instanceRoleARN will continue to have the access entry resource in the CloudFormation stack, making delete nodegroup an atomic operation for most use cases. Fixes eksctl-io#7502 * Add note about deleting nodegroups * Add integration tests * Fix cluster deletion in tests * Allow nodegroup creation after a cluster subnet is deleted (eksctl-io#7714) * Preserve eksctl commands correctness when user deletes subnets * update error when subnet availability validation fails * address PR comments * Handle K8s service account lifecycle on `eksctl create/delete podidentityassociation` commands (eksctl-io#7706) * Handle K8s service account lifecycle on eksctl create/delete podidentityassociations commands * correct typo Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> --------- Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> * Add support for Ubuntu Pro 22.04 based EKS images (eksctl-io#7711) * feat: Add support for Ubuntu Pro 22.04 based EKS images * update schema.json * test: Add nodegroup with Ubuntu Pro 22.04 * fix integration test --------- Co-authored-by: Tibi <110664232+TiberiuGC@users.noreply.github.com> * Disable IMDSv1 in unowned integration tests * include pre-releases as full releases when drafting release notes * Add utils command to migrate `iamidentitymappings` to EKS access entries (eksctl-io#7710) * Added migrate-to-access-entry cmd structure * Fix Target Authentication mode validation * Added logic to get accessEntries and cmEntries from cluster * Added logic to make unique list of configmap accessEntries, and stack creation logic * Added UpdateAuthentication mode and aeEntries filter logic * Add approve flag check * Added functionality to remove awsauth after switch to API only * Adds logic to fetch FullARN of path stripped IAMIdentityMappings * Updates some info log text * Adds test case and refactors code * Removes comments * Adds taskTree and address PR comments * Refactors code and Adds exception handling for NoSuchEntityException * Resolves go.mod and go.sum conflicts * Doc update for migrate-to-access-entry feature * Fixed minimum iam policies doc to add permission for iam:GetUser * Updated access-entries doc at migrate-to-access-entry section * Fixes failing Migrate To Access Entry Test & go.mod, go.sum * Amends migrate to access entry documentation * improve logs and simplify code logic * add unit tests * ensure target-auth-mode has a valid value --------- Co-authored-by: Pankaj Walke <advaitt@amazon.com> Co-authored-by: Venkat Penmetsa <vpenmets@amazon.com> Co-authored-by: Venkat Penmetsa <vpenmets@gmail.com> Co-authored-by: Tibi <110664232+TiberiuGC@users.noreply.github.com> * Revert "[Release drafter] Treat RCs as full releases when drafting notes" (eksctl-io#7725) * Fix creating pod identities Replaces usage of a per-loop variable with a per-iteration variable. * Fix deleting pod identities * Fix deleting clusters with a non-active status * Add release notes for v0.177.0 * update release notes for 0.177.0 * change purchase type to capacity block when using capacity reservation --------- Signed-off-by: Amine Hilaly <hilalyamine@gmail.com> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: Amine Hilaly <hilalyamine@gmail.com> Co-authored-by: cpu1 <chetan@weave.works> Co-authored-by: Yu Xiang Z <yxz.get@gmail.com> Co-authored-by: Yu Xiang Zhang <yuxz@amazon.com> Co-authored-by: eksctl-bot <53547694+eksctl-bot@users.noreply.github.com> Co-authored-by: Nathaniel Emerson <nathaniel.emerson@skyscanner.net> Co-authored-by: guessi <guessi@gmail.com> Co-authored-by: Raghav Khandelwal <raghavk@zendrive.com> Co-authored-by: Stephen Lang <stephen.lang@grafana.com> Co-authored-by: Thomas Bechtold <thomas.bechtold@canonical.com> Co-authored-by: Shuntaro Azuma <azush@amazon.com> Co-authored-by: Matthew Robinson <matt@zensunni.org> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Osama Bin Junaid <32925504+ibnjunaid@users.noreply.github.com> Co-authored-by: rpocase <rpocase@gmail.com> Co-authored-by: Denys Havrysh <denys.gavrysh@gmail.com> Co-authored-by: yuxiang-zhang <23327251+yuxiang-zhang@users.noreply.github.com> Co-authored-by: Tibi <110664232+TiberiuGC@users.noreply.github.com> Co-authored-by: Weifeng Wang <qclaogui@gmail.com> Co-authored-by: Chetan Patwal <cPu1@users.noreply.github.com> Co-authored-by: cpu1 <patwal.chetan@gmail.com> Co-authored-by: TimAndy <xuchonglei@126.com> Co-authored-by: cPu1 <patwal.chetan@gmail.comm> Co-authored-by: Alberto Contreras <aciba90@gmail.com> Co-authored-by: punkwalker <126026317+punkwalker@users.noreply.github.com> Co-authored-by: Pankaj Walke <advaitt@amazon.com> Co-authored-by: Venkat Penmetsa <vpenmets@amazon.com> Co-authored-by: Venkat Penmetsa <vpenmets@gmail.com> Co-authored-by: eV <evq@brave.com>

TiberiuGC added the kind/feature New feature or request label Mar 28, 2024

TiberiuGC force-pushed the feature/amazon-linux-2023 branch 4 times, most recently from a0349dd to d535ddd Compare March 28, 2024 16:54

TiberiuGC marked this pull request as ready for review March 28, 2024 17:41

TiberiuGC requested review from a-hilaly and yuxiang-zhang March 29, 2024 15:58

yuxiang-zhang reviewed Apr 1, 2024

View reviewed changes

pkg/apis/eksctl.io/v1alpha5/validation.go Outdated Show resolved Hide resolved

pkg/apis/eksctl.io/v1alpha5/validation.go Show resolved Hide resolved

pkg/apis/eksctl.io/v1alpha5/validation.go Show resolved Hide resolved

pkg/apis/eksctl.io/v1alpha5/validation.go Show resolved Hide resolved

yuxiang-zhang approved these changes Apr 2, 2024

View reviewed changes

a-hilaly reviewed Apr 2, 2024

View reviewed changes

TiberiuGC added 11 commits April 3, 2024 12:53

add support for AL2023 for EKS-managed and self-managed nodes

5a9e3e6

ensure AL2023 only supports containerd

b29ad52

add GPU related validations + small nits

7dea971

add support for upgrades

677daf0

add support for EFA

c21c4a6

improve validations

36fa4db

fix lint and unit tests

5e754c2

update docs

f5932a0

add validation error for maxpods limitation

de74c5b

add integration tests for al2023

0f3f294

improve validation message

8965847

TiberiuGC force-pushed the feature/amazon-linux-2023 branch from 271e70b to 8965847 Compare April 3, 2024 09:54

a-hilaly approved these changes Apr 4, 2024

View reviewed changes

TiberiuGC merged commit f9475f8 into eksctl-io:main Apr 4, 2024
10 checks passed

eksctl-bot mentioned this pull request Apr 4, 2024

Add release notes for v0.176.0 #7672

Merged

eksctl-bot mentioned this pull request Apr 23, 2024

Add release notes for v0.177.0 #7721

Merged

hspencer77 mentioned this pull request Apr 30, 2024

Update main to match upstream brave-intl/eksctl#533

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for AMIs based on AmazonLinux2023 #7684

Add support for AMIs based on AmazonLinux2023 #7684

TiberiuGC commented Mar 28, 2024 •

edited

thedavidwhiteside commented Mar 29, 2024 •

edited

yuxiang-zhang left a comment

a-hilaly left a comment

a-hilaly Apr 2, 2024

TiberiuGC Apr 3, 2024

a-hilaly Apr 4, 2024

a-hilaly Apr 4, 2024

a-hilaly left a comment

Add support for AMIs based on AmazonLinux2023 #7684

Add support for AMIs based on AmazonLinux2023 #7684

Conversation

TiberiuGC commented Mar 28, 2024 • edited

Description

Checklist

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

thedavidwhiteside commented Mar 29, 2024 • edited

yuxiang-zhang left a comment

Choose a reason for hiding this comment

a-hilaly left a comment

Choose a reason for hiding this comment

a-hilaly Apr 2, 2024

Choose a reason for hiding this comment

TiberiuGC Apr 3, 2024

Choose a reason for hiding this comment

a-hilaly Apr 4, 2024

Choose a reason for hiding this comment

a-hilaly Apr 4, 2024

Choose a reason for hiding this comment

a-hilaly left a comment

Choose a reason for hiding this comment

TiberiuGC commented Mar 28, 2024 •

edited

thedavidwhiteside commented Mar 29, 2024 •

edited