aws_ecs_service - deployment_minimum_healthy_percent for DAEMON

[ghost]

This issue was originally opened by @nirpod as hashicorp/terraform#18480. It was migrated here as a result of the provider split. The original body of the issue is below.

---

why wouldn't you support deployment_minimum_healthy_percent for DAEMON (for aws_ecs_service rss)?
AWS support this feature.

[mmacdermaid]

Any news on this?

deployment_minimum_healthy_percent

Definitely affects DAEMON deploys on ECS clusters. Having the setting at 0 will bring down your entire cluster instead of 50% which will do rolling deploys.

Edit: For now, you can get around this when force deploying your ECS cluster. Specify

--deployment-configuration='{"maximumPercent":100,"minimumHealthyPercent":50}'

for the command

aws ecs update-service

[andyalm]

We just hit this issue as well. 0 is not a safe default for minHealthyPercent, and it should be settable for a DAEMON service.

[vkruoso]

Looks like this is ignored in the code explicitly, and that "deployment_maximum_percent" is also ignored when the strategy is DAEMON.

This was introduced by aca44f4. I'm not sure if only removing those conditions would work, but would be happy to provide a PR if that looks like the answer.

[mschurenko]

+1 for fixing this. Just ran into this issue. I would be fine with providing a PR as well if it will fix this.

[bflad]

Support has been merged and will release with version 1.42.0 of the AWS provider, likely later today. 👍

[bflad]

This has been released in version 1.42.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[tglastra]

The value does not work correctly on consecutive apply attempts. Steps to reproduce:

• Use provider 1.42.0
• Set deployment_minimum_healthy_percent = 50
• Apply
• Set deployment_minimum_healthy_percent = 0

Expected result:

• Terraform sees a change in deployment_minimum_healthy_percent and makes a plan to reduce it to 0

Outcome:

• Terraform only sees changes when the value is anything higher than 0

Problem:

• Using Terraform for automation, we calculate deployment_minimum_healthy_percent based desired EC2 instances.
  • A calculation for one instance results in 0% deployment_minimum_healthy_percent,
  • A calculation for three instances results in 66%, forcing one instance to be replaced at a time
  • ... And so on

Right now, we need to manually set the cluster to deployment_minimum_healthy_percent = 0 when we revert to a single instance.

[mschurenko]

As a workaround can you set deployment_minimum_healthy_percent to 1 instead of 0? This should have the same affect provided you have less than 100 instances.

[tglastra]

@mschurenko Thanks for the suggestion. My understanding is that 1% health is only achievable by running at least one instance. Hence, if there is only one instance, it cannot be terminated since that would drop the healthy percentage below 1%, and thus I need additional cluster capacity (to have at least two instances running during deployment), which does not play well with a daemon cluster setup. I'd have to test to be sure though.

[mschurenko]

If that’s the case can you create a new issue specifically for this since this one has already been closed?

[GuiTeK]

@tglastra @mschurenko Do you know if this has been taken care of? I'm using AWS provider 1.42.0 and I'm experiencing the following:

• Creating an ECS service with deployment_minimum_healthy_percent = 0 actually results in having deployment_minimum_healthy_percent = 100
• Manually setting deployment_minimum_healthy_percent = 0 in the AWS Console works and Terraform doesn't set it back to 100 when planning/applying

[bflad]

Hi @GuiTeK 👋 Does #6315 describe your problem? If so, upgrading to version 1.43.0 of the Terraform AWS provider has the fix. If not, I would suggest filling out a bug report with all the relevant details, thanks. 👍

[CalvinWong-Reflektion]

we just noticed this issue as well... not sure how our other services had 0 set. but the new service was automatically set to 100 when i specifically have 0 in terraform :-\

[LeeXun]

Short solution:

  scheduling_strategy = "DAEMON"
  deployment_controller {
    type = "ECS"
  }
  deployment_maximum_percent = 100
  deployment_minimum_healthy_percent = 50 # less than and not equal to 100

I have checked that in v2.30.0, here is the code that affect these variables. But as far as I know, AWS will only allow us to set the maximum to 100% when the scheduling_strategy is "DAEMON", which means it can't be more than one task in the same EC2 instance.

That is, max should always be 100, and min should lower than 100.

Furthermore, setting min to 50 and only running one available EC2 instance will cause new task pending forever. Because old task should hold to maintain to 50% minimun requirement. The new task has no where to deploy. So if you are using the setting above(100, 50), available EC2 instances should be greater than 2 at least.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!