Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

aws-eks: support --atomic flag for helm commands #22254

Closed
1 of 2 tasks
michaelfedell opened this issue Sep 27, 2022 · 2 comments · Fixed by #29454, rwlxxvii/containers#124 or rwlxxvii/containers#140 · May be fixed by NOUIY/aws-solutions-constructs#99 or NOUIY/aws-solutions-constructs#101
Closed
1 of 2 tasks

aws-eks: support --atomic flag for helm commands #22254

michaelfedell opened this issue Sep 27, 2022 · 2 comments · Fixed by #29454, rwlxxvii/containers#124 or rwlxxvii/containers#140 · May be fixed by NOUIY/aws-solutions-constructs#99 or NOUIY/aws-solutions-constructs#101
Assignees
@shikha372
Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. good first issue Related to contributions. See CONTRIBUTING.md p2

Comments

@michaelfedell
Copy link
Contributor

Describe the feature

Support the Helm CLI --atomic flag as a prop to the aws-cdk.aws-eks.HelmChart construct. This would enable a user to perform their install/upgrade as an atomic operation, rolling back or uninstalling if the operation fails.

Use Case

Currently, if the resources installed by helm do not successfully initialize and the --wait flag is true, then helm will fail to send a response back to the custom resource in time. Once the first invocation fails, subsequent attempts will fail with the following error:

Error: UPGRADE FAILED: another operation (install/upgrade/rollback) is in progress

After the max attempts, the CloudFormation stack will be marked as UPDATE_ROLLBACK_FAILED. Forcing the user to manually rollback the helm release as described in this issue on the helm repo

Proposed Solution

This could be supported with an additional, optional ResourceProperty passed to the custom resource handler and used in the constructed helm command.

aws-cdk/packages/@aws-cdk/aws-eks/lib/kubectl-handler/helm/__init__.py

Line 149 in 877ad71

def helm(verb, release, chart = None, repo = None, file = None, namespace = None, version = None, wait = False, timeout = None, create_namespace = None):

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.28.1

Environment details (OS name and version, etc.)

MacOS, TypeScript
@michaelfedell michaelfedell added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Sep 27, 2022
@github-actions github-actions bot added the @aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service label Sep 27, 2022
@peterwoodworth peterwoodworth added good first issue Related to contributions. See CONTRIBUTING.md p2 effort/small Small work item – less than a day of effort and removed needs-triage This issue or PR still needs to be triaged. labels Sep 27, 2022
@peterwoodworth
Copy link
Contributor

Thanks for the request @michaelfedell,

You're right, this would be a pretty easy implementation done just by introducing a new property and using it similar to the other flags.

I am marking this issue as p2, which means that we are unable to work on this immediately.

Check out our contributing guide if you're interested in contributing yourself - there's a low chance the team will be able to address this soon but we'd be happy to review a PR 🙂

@michaelfedell michaelfedell mentioned this issue Sep 27, 2022
Open
michaelfedell added a commit to michaelfedell/aws-cdk that referenced this issue Sep 27, 2022
@michaelfedell
feat: add atomic flag for aws-eks HelmChart
86025da 
fixes: aws#22254
@michaelfedell michaelfedell mentioned this issue Sep 28, 2022
Closed
4 tasks
@alexandersperling alexandersperling mentioned this issue Mar 12, 2023
Closed
@alexandersperling alexandersperling mentioned this issue Dec 12, 2023
Closed
@shikha372 shikha372 self-assigned this Feb 29, 2024
@shikha372 shikha372 mentioned this issue Mar 12, 2024
Merged
1 task
@mergify mergify bot closed this as completed in #29454 Apr 26, 2024
mergify bot pushed a commit that referenced this issue Apr 26, 2024
@shikha372
feat(eks): add atomic flag for aws-eks Helm Chart (#29454)
666f24f 
### Issue # (if applicable)

Closes #22254.

### Reason for this change

Currently, if chart is installed to the EKS cluster with wait timeout period set and fails initialization, helm will fail to send a response back to custom resource and it will be stuck in pending upgrade state (expected state is failed) . 
Subsequent attempts to update the stack will result in failure while chart is stuck in `pending upgrade` state until manually rolled back or deleted from the cluster. 

### Description of changes

Added feature flag `--atomic` supported by helm currently to mark the operation as atomic which will automatically rolls back the changes in case of upgrade/installation failure.
Reference doc: https://helm.sh/docs/helm/helm_install/#options

### Description of how you validated changes

Added unit tests to check if flag is set as per user input in the template.

### Checklist
- [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md)

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
@github-actions GitHub Actions
Copy link

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

@shikha372 shikha372 mentioned this issue May 8, 2024
Closed
1 task
@NOUIY NOUIY mentioned this issue May 24, 2024
Open
@kazimanzurrashid kazimanzurrashid mentioned this issue May 25, 2024
Closed
@rwlxxvii rwlxxvii mentioned this issue May 27, 2024
Merged
@NOUIY NOUIY mentioned this issue May 30, 2024
Open
@NOUIY NOUIY mentioned this issue Jun 7, 2024
Open
@niravparikh05 niravparikh05 mentioned this issue Jun 13, 2024
Open
@NOUIY NOUIY mentioned this issue Jun 14, 2024
Open
@snyk-io snyk-io bot mentioned this issue Jun 20, 2024
Open
@NOUIY NOUIY mentioned this issue Jun 20, 2024
Open
@snyk-io snyk-io bot mentioned this issue Jun 21, 2024
Open
@rwlxxvii rwlxxvii mentioned this issue Jun 22, 2024
Merged
@NOUIY NOUIY mentioned this issue Jun 22, 2024
Open
@Wayne9981 Wayne9981 mentioned this issue Jun 28, 2024
Open
2 tasks
This was referenced Jun 29, 2024
Open
Open
@NOUIY NOUIY mentioned this issue Jul 12, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@shikha372 shikha372

Labels
@aws-cdk/aws-eks Related to Amazon Elastic Kubernetes Service effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. good first issue Related to contributions. See CONTRIBUTING.md p2
Projects
None yet
Milestone
No milestone
4 participants
@otaviomacedo @shikha372 @michaelfedell @peterwoodworth