✨ Migrate ec2 to AWS SDK V2 #5521

yiannistri · 2025-06-04T09:24:22Z

What type of PR is this?

/kind feature
/kind cleanup

What this PR does / why we need it:

Migrates ec2 packages to AWS SDK v2.

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #5403, #5411

Special notes for your reviewer:

Checklist:

Release note:

Migrate EC2 packages to AWS SDK v2

k8s-ci-robot · 2025-06-04T09:24:30Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign ankitasw for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot · 2025-06-04T09:24:32Z

Welcome @yiannistri!

It looks like this is your first PR to kubernetes-sigs/cluster-api-provider-aws 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/cluster-api-provider-aws has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-06-04T09:24:33Z

Hi @yiannistri. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

alexander-demicev · 2025-06-05T12:25:58Z

/ok-to-test

nrb

Thank you for taking on this piece in particular, it's a big one!

Makefile

cmd/clusterawsadm/ami/copy.go

pkg/cloud/services/common/common.go

nrb · 2025-06-20T15:02:23Z

pkg/cloud/services/ec2/instances.go

-			metadataOptions.InstanceMetadataTags = infrav1.InstanceMetadataState(*v.MetadataOptions.InstanceMetadataTags)
-		}
+		metadataOptions.HTTPEndpoint = infrav1.InstanceMetadataState(string(v.MetadataOptions.HttpEndpoint))
+		metadataOptions.HTTPPutResponseHopLimit = int64(*v.MetadataOptions.HttpPutResponseHopLimit)


This one would still need a nil check since it's de-referencing a pointer.

damdo · 2025-06-24T08:02:53Z

/retest

damdo · 2025-06-30T09:00:57Z

@yiannistri would you be able to rebase? Thanks!

punkwalker

@yiannistri
Thank you for working on this.
I am late to party but these changes are important for maintaining functionality between SDK v1 and v2

punkwalker · 2025-06-30T21:17:15Z

pkg/cloud/scope/clients.go

+func NewEC2Client(scopeUser cloud.ScopeUsage, session cloud.Session, logger logger.Wrapper, target runtime.Object) *ec2.Client {
+	cfg := session.SessionV2()
+
+	ec2opts := []func(*ec2.Options){
+		func(o *ec2.Options) {
+			o.Logger = logger.GetAWSLogger()
+			o.ClientLogMode = awslogs.GetAWSLogLevelV2(logger.GetLogger())
+		},
+		ec2.WithAPIOptions(
+			awsmetricsv2.WithMiddlewares(scopeUser.ControllerName(), target),
+			awsmetricsv2.WithCAPAUserAgentMiddleware(),
+		),


@yiannistri Can you create EndpointResolver like this - https://github.com/kubernetes-sigs/cluster-api-provider-aws/blob/main/pkg/cloud/endpointsv2/endpoints.go#L109-L123

Also, the serviceLimiter implementation is also required.

punkwalker · 2025-06-30T21:21:52Z

pkg/utils/utils.go

+// ToInt64Value converts an int32 pointer to an int64 value.
+func ToInt64Value(to *int32) int64 {
+	if to == nil {
+		panic("Cannot convert nil pointer to int64")
+	}
+	return int64(*to)
+}
+


Can we not use aws.Int64 here?

I don't think we can because the aws pointer conversion methods convert to a value of the same bit size as the pointer, i.e. int64 to *int64 and vice versa and in this case we convert from *int32 to int64.

Apologigies, I was suggesting aws.ToInt64.
We could use the same logic with it like aws.ToInt64(int64(*int32)). This way we don't need to this additional code/method and we can avoid panicking.

isn't this code aws.ToInt64(int64(*int32)) going to panic too if *int32 is nil?

punkwalker · 2025-06-30T22:58:50Z

@yiannistri
update: Please use the ServiceLimiterMiddleware from PR #5574. You can find more details in it.

k8s-ci-robot · 2025-07-08T09:40:40Z

@yiannistri: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-cluster-api-provider-aws-e2e-eks	`1f72f29`	link	false	`/test pull-cluster-api-provider-aws-e2e-eks`
pull-cluster-api-provider-aws-e2e	`17c1c53`	link	false	`/test pull-cluster-api-provider-aws-e2e`
pull-cluster-api-provider-aws-e2e-blocking	`969ef40`	link	true	`/test pull-cluster-api-provider-aws-e2e-blocking`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

damdo · 2025-07-16T17:44:32Z

@yiannistri hey would you be able to rebate now that the GC or merged? Thanks!

k8s-ci-robot · 2025-07-16T17:44:41Z

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot added release-note do-not-merge/work-in-progress kind/feature kind/cleanup labels Jun 4, 2025

k8s-ci-robot added cncf-cla: yes needs-priority labels Jun 4, 2025

k8s-ci-robot requested review from richardcase and serngawy June 4, 2025 09:24

k8s-ci-robot added needs-ok-to-test size/XXL labels Jun 4, 2025

yiannistri force-pushed the ec2-sdk-v2-migration branch 16 times, most recently from 52a7c2f to 543a41c Compare June 5, 2025 12:15

yiannistri force-pushed the ec2-sdk-v2-migration branch from 9704c3a to 9711485 Compare June 20, 2025 09:55

k8s-ci-robot added needs-rebase and removed needs-rebase labels Jun 20, 2025

nrb reviewed Jun 20, 2025

View reviewed changes

yiannistri force-pushed the ec2-sdk-v2-migration branch from 9711485 to 4f9ce70 Compare June 23, 2025 08:56

k8s-ci-robot removed the needs-rebase label Jun 23, 2025

k8s-ci-robot added the needs-rebase label Jun 26, 2025

punkwalker suggested changes Jun 30, 2025

View reviewed changes

yiannistri force-pushed the ec2-sdk-v2-migration branch from 4f9ce70 to f7de9ec Compare July 2, 2025 07:00

k8s-ci-robot removed the needs-rebase label Jul 2, 2025

yiannistri force-pushed the ec2-sdk-v2-migration branch from f7de9ec to cc17ae5 Compare July 2, 2025 07:18

k8s-ci-robot added the needs-rebase label Jul 3, 2025

LiangquanLi930 mentioned this pull request Jul 4, 2025

✨ Migrate Node packages to AWS SDK v2 #5584

Open

5 tasks

yiannistri force-pushed the ec2-sdk-v2-migration branch from cc17ae5 to e4897d8 Compare July 7, 2025 16:11

k8s-ci-robot added needs-rebase and removed needs-rebase labels Jul 7, 2025

yiannistri force-pushed the ec2-sdk-v2-migration branch from e4897d8 to 182a570 Compare July 8, 2025 06:52

k8s-ci-robot removed the needs-rebase label Jul 8, 2025

yiannistri force-pushed the ec2-sdk-v2-migration branch 2 times, most recently from 4604776 to e9a8513 Compare July 8, 2025 07:04

Migrate ec2 to AWS SDK V2

969ef40

yiannistri force-pushed the ec2-sdk-v2-migration branch from e9a8513 to 969ef40 Compare July 8, 2025 08:49

nrb mentioned this pull request Jul 16, 2025

Migrate Node packages to AWS SDK v2 #5402

Open

k8s-ci-robot added the needs-rebase label Jul 16, 2025

✨ Migrate ec2 to AWS SDK V2 #5521

Are you sure you want to change the base?

✨ Migrate ec2 to AWS SDK V2 #5521

Conversation

yiannistri commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Jun 4, 2025

Uh oh!

k8s-ci-robot commented Jun 4, 2025

Uh oh!

k8s-ci-robot commented Jun 4, 2025

Uh oh!

alexander-demicev commented Jun 5, 2025

Uh oh!

nrb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

nrb Jun 20, 2025

Choose a reason for hiding this comment

Uh oh!

damdo commented Jun 24, 2025

Uh oh!

damdo commented Jun 30, 2025

Uh oh!

punkwalker left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

punkwalker Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

punkwalker Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

punkwalker Jun 30, 2025

Choose a reason for hiding this comment

Uh oh!

yiannistri Jul 2, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

punkwalker Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alexander-demicev Jul 4, 2025

Choose a reason for hiding this comment

Uh oh!

punkwalker commented Jun 30, 2025

Uh oh!

k8s-ci-robot commented Jul 8, 2025

Uh oh!

damdo commented Jul 16, 2025

Uh oh!

k8s-ci-robot commented Jul 16, 2025

Uh oh!

Uh oh!

yiannistri commented Jun 4, 2025 •

edited

Loading

punkwalker left a comment •

edited

Loading

yiannistri Jul 2, 2025 •

edited

Loading

punkwalker Jul 3, 2025 •

edited

Loading