replace hard coded aws region checks with aws sdk calls

[guydog28]

future proof AWS endpoint and IAM Prefix generation to handle new AWS Partitions and future new ones. Offload to AWS SDK

[k8s-ci-robot]

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

---

• If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
• If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
• If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: https://support.linuxfoundation.org/
• Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[k8s-ci-robot]

Hi @guydog28. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[guydog28]

/check-cla

[guydog28]

I signed it

[guydog28]

/check-cla

[guydog28]

/check-cla

[guydog28]

I signed it

[guydog28]

I didn't want to muddy up this pull request, but additionally there are partitions whose regions are not represented in upup/pkg/fi/cloud.go (zonesToCloud). Why not generate this list from the aws-sdk-go? This way as new regions are reported, if the sdk continues to be updated, the new regions will be available to kops users.

[guydog28]

/check-cla

[guydog28]

/check-cla

[guydog28]

/check-cla

[hakman]

Thanks @guydog28.
/ok-to-test

[guydog28]

/test pull-kops-e2e-cni-cilium

[hakman]

@guydog28 Please squash and make the commit message shorter. Will make it easier to track in the future.

[rifelpet]

Regarding zonesToCloud, I agree that using the SDK to fetch an AZ list would be preferable, my concern would be that at the time we use that function, we don't necessarily know which cloud provider the kops create cluster is for, so we'd have to invoke each one and deal with credential failures since the user likely doesn't have credentials setup for cloud providers they're not using.

If we can figure out a lightweight way to do that for any providers that support providing this info via API, I would support that effort 👍🏻

That mapping only exists because we support inferring the cloud provider from the provided --zones. I would strongly prefer that we just make --cloud required and drop this map entirely, but I'm not sure how others feel.

[hakman]

That mapping only exists because we support inferring the cloud provider from the provided --zones. I would strongly prefer that we just make --cloud required and drop this map entirely, but I'm not sure how others feel.

I agree with this.

[guydog28]

Regarding zonesToCloud...

If we can figure out a lightweight way to do that for any providers that support providing this info via API, I would support that effort 👍🏻

That mapping only exists because we support inferring the cloud provider from the provided --zones. I would strongly prefer that we just make --cloud required and drop this map entirely, but I'm not sure how others feel.

@hakman @rifelpet If it just to make a guess, why not remove the map altogether and use a few regexs in the GuessCloudForZone func?

Just test it against each regex to guess the provider instead of that big list?

[guydog28]

I have added the changes to cloud.go to this PR as well, as they seem pertinent to the goal of it.

Additionally, I took the old hardcoded zones and used them to create a unit test to test the regex I created to be sure this method still returns the same results.

[hakman]

@guydog28 Let's keep the 2 things separate. Please create a separate PR for the zones thing and we can discuss there.
The partitions thing seems reasonable, the only thing missing would be some unit tests for the changed functions, if you don't mind. It also seems innocent enough to cherry-pick to 1.19, right @rifelpet?

[rifelpet]

Yes I agree

[guydog28]

@hakman ok I pulled out the cloud.go into its own PR, and added tests for the changes.

[hakman]

Could you also update the commit message and PR title to match the content?

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [hakman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment