ECS Fargate Service: service not created because of alarm based rollback

[srshkmr]

Describe the bug

During the deployment process with the same code which was working previously is returning the following error.

4:43:04 pm | CREATE_FAILED        | AWS::ECS::Service                               | xxxx
Resource handler returned message: "Invalid request provided: CreateService error: Alarm based rollback feature is only supported with ECS deployment controller. Update t
o ECS deployment controller and try again. (Service: AmazonECS; Status Code: 400; Error Code: InvalidParameterException; Request ID: xxx;
Proxy: null)" (RequestToken: xxx, HandlerErrorCode: InvalidRequest)

The following is my ECS deployment group

const ecsDeploymentGroup = new codeDeploy.EcsDeploymentGroup(this, 'ecsDeploymentGroup', {
      application: ecsApplication,
      deploymentGroupName: props.ECSTaskFamilyName,
      deploymentConfig,
      alarms: ecsServiceAlarms.alarms,
      service: ecsBlueGreenService.ecsService,
      blueGreenDeploymentConfig: {
        blueTargetGroup: ecsBlueGreenService.blueTargetGroup,
        greenTargetGroup: ecsBlueGreenService.greenTargetGroup,
        listener: ecsBlueGreenService.albProdListener,
        testListener: ecsBlueGreenService.albHttpsListener,
        terminationWaitTime: Duration.minutes(10),
      },
      autoRollback: {
        stoppedDeployment: true,
      }
    });

The application is deployed using Code deploy, So the deployment controller is set as CODE_DEPLOY

this.ecsService = new ecs.FargateService(this, 'FargateService', {
      cluster: props.cluster!,
      desiredCount: 1,
      taskDefinition,
      minHealthyPercent: 100,
      maxHealthyPercent: 200,
      enableExecuteCommand: true,
      deploymentController: {
        type: DeploymentControllerType.CODE_DEPLOY
      },
      healthCheckGracePeriod: Duration.seconds(60)
    });

Expected Behavior

ECS Cluster gets deployed successfully

Current Behavior

ECS creation failed

Reproduction Steps

Use the code in the bug description

Possible Solution

No response

Additional Information/Context

No response

CDK CLI Version

2.87.0

Framework Version

No response

Node.js Version

v18.15.0

OS

MacOS 14.0

Language

Typescript

Language Version

5.1.6

Other information

No response

[peterwoodworth]

During the deployment process with the same code which was working previously is returning the following error.

Could you share the difference in output between when this was working and when this was not? (i.e. cdk diff) - Additionally, when did this stop working, which version did you upgrade from to start encountering this error?

[peterwoodworth]

Though this just looks like the service rejecting the configuration given your deployment controller type, but I am pretty sure that auto rollback should be able to be selected for the codedeploy controller type

[jiem-ying]

Hi +1 to this issue.

Confirmed from ECS side that if using Code_Deploy type, the CreateService API would fail with Validation when "CODE_DEPLOY" and "DeploymentConfiguration.Alarm" combination detected:

{
  ...
  "DeploymentController": {
    "Type": "CODE_DEPLOY"
  },
  ...
  "DeploymentConfiguration": {
    "Alarms": {
      "AlarmNames": [],
      "Enable": "false",
      "Rollback": "false"
    },
    ...
  },
 ...
}

And as ECS is designed that the DeploymentConfiguration.Alarm can only used by default controller AKA "ECS deployment controller". https://docs.aws.amazon.com/AmazonECS/latest/developerguide/deployment-alarm-failure.html

I understand this new change is from this release #25840

Given it is ECS service API limitation, the L2 construct should have a judgement that if "DeploymentController.Type" is "CODE_DEPLOY", do not explicitly specify the DeploymentConfiguration.Alarms.

Currently the workaround is using Escape Hatches then override deletion:
CfnService.addPropertyDeletionOverride('DeploymentConfiguration.Alarms');

Let me know if you need more clarifications.

[srshkmr]

Could you share the difference in output between when this was working and when this was not? (i.e. cdk diff) - Additionally, when did this stop working, which version did you upgrade from to start encountering this error?

I upgraded from 2.81.0 to 2.87.0, That's when this issue started occuring, Weird thing is even after downgrading the CDK version to 2.81 I am getting the same error. It was working fine on Thursday last week

[tam0ri]

When deploymentAlarms property is defined explicitly, CDK checks deploymentController is ECS or not.

aws-cdk/packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts

Lines 632 to 636 in 9fca790

	if (props.deploymentAlarms
	&& deploymentController
	&& deploymentController.type !== DeploymentControllerType.ECS) {
	throw new Error('Deployment alarms requires the ECS deployment controller.');
	}

On the other hand, currently CDK never validates deploymentController property for the default alarms settings.

aws-cdk/packages/aws-cdk-lib/aws-ecs/lib/base/base-service.ts

Lines 685 to 691 in 9fca790

	} else if (this.deploymentAlarmsAvailableInRegion()) {
	this.deploymentAlarms = {
	alarmNames: [],
	enable: false,
	rollback: false,
	};
	}

I just submitted PR to fix this.

[peterwoodworth]

Thanks a bunch @tam0ri!

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.