Update IAMServiceaccount uses a unique changeset name to allow successive updates

[aclevername]

Description

See #3604 (comment):

2021-04-19 16:59:05 [ℹ]  eksctl version 0.46.0-dev+69c20d09.2021-04-19T16:49:53Z
2021-04-19 16:59:05 [ℹ]  using region us-west-2
2021-04-19 16:59:07 [ℹ]  1 task: { 1 task: { update IAMServiceAccount backend-apps/s3-reader } }
2021-04-19 16:59:07 [ℹ]  updating policies for IAMServiceAccount backend-apps/s3-reader
2021-04-19 16:59:08 [ℹ]  1 error(s) occurred and IAM Role stacks haven't been updated properly, you may wish to check CloudFormation console
2021-04-19 16:59:08 [✖]  creating ChangeSet "updating-policy" for stack "eksctl-jk-addon-iamserviceaccount-backend-apps-s3-reader": AlreadyExistsException: ChangeSet updating-policy cannot be created due to a mismatch with existing attribute Template
        status code: 400, request id: 55715d50-08fe-4637-967e-e6d1d3379434

I was slightly confused at first, but after reproducing I understand. To clarify for my future self: you can update a service account multiple times fine, when you attempt an update that has no change you get:

2021-04-19 16:58:54 [ℹ]  eksctl version 0.46.0-dev+69c20d09.2021-04-19T16:49:53Z
2021-04-19 16:58:54 [ℹ]  using region us-west-2
2021-04-19 16:58:56 [ℹ]  1 task: { 1 task: { update IAMServiceAccount backend-apps/s3-reader } }
2021-04-19 16:58:56 [ℹ]  updating policies for IAMServiceAccount backend-apps/s3-reader
2021-04-19 16:58:56 [ℹ]  waiting for CloudFormation changeset "updating-policy" for stack "eksctl-jk-addon-iamserviceaccount-backend-apps-s3-reader"
2021-04-19 16:58:57 [ℹ]  nothing to update

Which is good, but it results in the ChangeSet being in a failed state in Cloudformation (I didn't realise this). If you then attempt an actual change again, it fails because eksctl attempts to update a changeset that is in a failed state.

1. Create IRSA
2. Update with new policies
3. Update again, with no changes (no-op)
4. Update with new policies, errors.

I considered two alternative approaches:

1. Checking if the template has changed in the code (json comparision) instead of relying on CloudFormation
2. Deleting the changset if it fails

I decided against 1. due to the way cloudformation handles its value templating I don't know if diffing in the code would be reliable. 2. prevents the user from going into the stack to debug the failure.

Checklist

• Added tests that cover your change (if possible)
• Added/modified documentation as required (such as the README.md, or the userdocs directory)
• Manually tested
• Made sure the title of the PR is a good description that can go into the release notes
• (Core team) Added labels for change area (e.g. area/nodegroup) and kind (e.g. kind/improvement)

BONUS POINTS checklist: complete for good vibes and maybe prizes?! 🤯

• Backfilled missing tests for code in same general area 🎉
• Refactored something and made the world a better place 🌟

[nmaupu]

🤔 If I understand well, each no-op update would create a failed changeset with a unique name ? If this is the that case, how the cleaning will be done ?

[aclevername]

If I understand well, each no-op update would create a failed changeset with a unique name ? If this is the that case, how the cleaning will be done ?

Great question! After a successful changeset occurs I observed that cloudformation deletes all the failed changsets. Why it does this I don't know, but it does mean that this will get cleaned up.

[nmaupu]

Great question! After a successful changeset occurs I observed that cloudformation deletes all the failed changsets. Why it does this I don't know, but it does mean that this will get cleaned up.

👍 Weird behavior indeed ! Good to know though.