Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Configurable minimumHealthyPercent and maximumHealthyPercent for ECS Service Hotswap #29618

Open
1 of 2 tasks
rexrafa opened this issue Mar 26, 2024 · 2 comments
Labels
@aws-cdk/aws-ecs Related to Amazon Elastic Container effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2

Comments

@rexrafa
Copy link

rexrafa commented Mar 26, 2024

Describe the feature

Currently, when using the --hotswap flag for deploying ECS services, the minimumHealthyPercent is hardcoded to 0, which can be disruptive for services with interdependencies. This can result in downtime as the current tasks are stopped before the new ones are fully up and running. To improve the flexibility of hotswap deployments, allowing users to set the minimumHealthyPercent and maximumHealthyPercent parameters would be beneficial.

Use Case

We aim to speed up deployment times in our development environment by using the hotswap feature. However, our services have dependencies on each other, and the current hotswap behavior is too disruptive. We want to keep the current tasks running until the new ones are fully operational. Configuring minimumHealthyPercent and maximumHealthyPercent would give us the control needed to ensure smoother transitions during deployments.

Proposed Solution

Modify the ECS service hotswap implementation to accept minimumHealthyPercent and maximumHealthyPercent as configurable parameters, with default values of 0 and 200, respectively. These parameters should be exposed in the CDK, allowing users to override them as needed.

For example, the current implementation at:

It could be modified to accept these parameters from the user's CDK stack configuration.

Other Information

No response

Acknowledgements

  • I may be able to implement this feature request
  • This feature might incur a breaking change

CDK version used

2.117.0

Environment details (OS name and version, etc.)

Darwin Kernel Version 23.3.0

@rexrafa rexrafa added feature-request A feature should be added or improved. needs-triage This issue or PR still needs to be triaged. labels Mar 26, 2024
@github-actions github-actions bot added the @aws-cdk/aws-ecs Related to Amazon Elastic Container label Mar 26, 2024
@tim-finnigan tim-finnigan self-assigned this Mar 26, 2024
@tim-finnigan tim-finnigan added investigating This issue is being investigated and/or work is in progress to resolve the issue. and removed needs-triage This issue or PR still needs to be triaged. labels Mar 26, 2024
@tim-finnigan
Copy link

Thanks for the feature request. I saw some related discussion on this topic and @pahud's comment here: #22241 (comment)

I am not sure if this is a good default value but I believe if you run an ecs service with just single task with maximumHealthyPercent=100 and you need force the task update, you probably have to set the minimumHealthyPercent to 0 otherwise the only task might not be replaced unless maximumHealthyPercent is set to 200. However, with maximumHealthyPercent=200 the new deployment will have to bring up an additional healthy task before it can terminate the original one. If the additional one is not in healthy state min>0 max>100 might not be able to replace the only task. With min=0, the only task will be immediately tore down and replaced.

As cdk hot swap is designed for development only I believe setting min=0 makes sense here.

I think this request needs some further review and discussion, would be interested to get your thoughts on that comment.

@tim-finnigan tim-finnigan added p2 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. effort/medium Medium work item – several days of effort and removed investigating This issue is being investigated and/or work is in progress to resolve the issue. labels Mar 26, 2024
@tim-finnigan tim-finnigan removed their assignment Mar 26, 2024
@dysosmus
Copy link

@tim-finnigan , answering for my co-worker @rexrafa as he is OoO.

We mostly agree with @pahud's comment that min=0 is a good default and shouldn't be changed as the default value.
Our opinion is that allowing the user to customize that value will enable them to determine the exact trade-off between speed, safety, and cost they desire in their development process. It also helps to better account for development environments where ECS Services run more than a single task.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-ecs Related to Amazon Elastic Container effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

No branches or pull requests

3 participants