use subnets of ensured NLBs when update worker node SG rules

[oliviassss]

What type of PR is this?
/kind bug

Uncomment only one, leave it on its own line:

/kind api-change
/kind bug
/kind cleanup
/kind design
/kind documentation
/kind failing-test
/kind feature
/kind flake

What this PR does / why we need it:
The controller gets subnets by the order of subnet ids everytime when it ensures NLB, and modifies the worker node SG rules according to subnet CIDRs. This may cause issue under some use case for existing NLBs. For example, if there's an existing NLB-a on subnet-3 and subnet-4, later the user adds new subnets, say subnet-1 and subnet-2, the controller will get subnet-1 and subnet-2 when ensure NLB-a, remove the SG rule on subnet-3 and subnet-4 and add new SG rule on subnet-1 and subnet-2 for the NLB-a, while the NLB-a is still on subnet-3 and subnet-4.

This PR fixes this issue by checking if there are existing subnets on the NLB, if so, we use the cidrs on the existing subnets for SG rules. otherwise, we use the cidrs of discovered subnets.
Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:
I added some debug lines locally and ran the unit test TestNLBNodeRegistration, noticing that for the newly created LB, the loadbalancer.AvailabilityZones are empty, suspecting it may because for the createRequest, we don't pass any AZ info to it.

I1211 14:22:56.218521   61214 aws_loadbalancer.go:182] createRequest for LB: {  
  Name: "aid",
  SubnetMappings: [{
      SubnetId: "subnet-abc123de"
    }],
  Tags: [{
      Key: "kubernetes.io/service-name",
      Value: "/myservice"
    },{
      Key: "KubernetesCluster",
      Value: "clusterid.test"
    },{
      Key: "kubernetes.io/cluster/clusterid.test",
      Value: "owned"
    }],
  Type: "network"
}
I1211 14:22:56.218575   61214 aws_loadbalancer.go:183] Creating load balancer for /myservice with name: aid
I1211 14:22:56.218580   61214 aws_loadbalancer.go:190] AZs for LB: []
I1211 14:22:56.218592   61214 aws_loadbalancer.go:188] createResponse: {
  LoadBalancers: [{
      LoadBalancerArn: "arn:aws:elasticloadbalancing:us-east-1:123456789:loadbalancer/net/b92d3a4888a0b5fb/74b46815",
      LoadBalancerName: "aid",
      Type: "network",
      VpcId: "vpc-abc123def456abc78"
    }]
}

Does this PR introduce a user-facing change?:

None

[k8s-ci-robot]

This issue is currently awaiting triage.

If cloud-provider-aws contributors determine this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[M00nF1sh]

under which case will this condition be true?
shouldn't lbs always have subnets?

[oliviassss]

I was thinking about if it's a new NLB. but checking through the ensureLoadBalancerv2(), if it's a new NLB it will use the discoveredSubnetIDs. So yeah there should always be subnets. I'll update the PR

Updated: since we got the subnets from loadBalancer.AvailabilityZones, and for newly created LBs, the AZs are none, since the ensureLoadBalancerV2() does not pass AZ info as part to createRequest(): https://github.com/kubernetes/cloud-provider-aws/blob/master/pkg/providers/v1/aws_loadbalancer.go#L157. So I add a check here. If it's for newly created LBs, and in any case the output LB does not have AZ info, we use discovered subnets.

[oliviassss]

/retest

[M00nF1sh]

/lgtm
/approve

[oliviassss]

rebase main to fix the govlun failure

[oliviassss]

errors in e2e tests:

Error: unable to find bucket "provider-aws-test-infra", WebIdentityErr: failed to retrieve credentials

An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: No OpenIDConnect provider found in your account for https://oidc.eks.us-east-2.amazonaws.com/id/F8B73554FE6FBAF9B19569183FB39762

/test pull-cloud-provider-aws-e2e-kubetest2-quick
/test pull-cloud-provider-aws-e2e
/test pull-cloud-provider-aws-e2e-kubetest2

[cartermckinnon]

@dims do you know what's up with the OIDC thing?

Tracking: https://kubernetes.slack.com/archives/CCK68P2Q2/p1702414384604979

[oliviassss]

/retest

[oliviassss]

/test pull-cloud-provider-aws-check

[cartermckinnon]

/lgtm
/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cartermckinnon, M00nF1sh

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [M00nF1sh,cartermckinnon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment