Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When deploying ECS service with blue/green deployment, task definition and target group changes can't be ignored #1096

Open
morinrod-cardioscan opened this issue Aug 27, 2020 · 5 comments
Labels
bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec service/ecs ECS related things

Comments

@morinrod-cardioscan
Copy link

I created a stack deploying ECS service with blue/green deployment(CodeDeploy) using aws.ecs.Service().
After the creation of the service in AWS, when running "pulumi up" again without any change to the existing service, Pulumi detects there was an update to the service.(because the TG or task definition revision was changed on AWS)
Pulumi up -> the service is created.
Running Pulum up without making any changes -> an update is detected on the service.

Because of AWS limitations, the update fails : “InvalidParameterException: Unable to update network parameters on services with a CODE_DEPLOY deployment controller. Use AWS CodeDeploy to trigger a new deployment.”
I tried using the "ignore changes" and transformations properties when creating the service but it didn't help.

The issue forces me to delete all existing services and recreate them, If I need to make additional changes to the stack.

Related issues:
pulumi/pulumi-awsx#856
#1982

The service code:

const UsersService = new aws.ecs.Service("UsersService", {
    name: 'UsersService',
    cluster: ecs_cluster.id,
    schedulingStrategy: 'REPLICA',
    taskDefinition: 'Users-TD:1',
    platformVersion: 'LATEST',
    deploymentController:{
        type:'CODE_DEPLOY'
     } ,
    deploymentMaximumPercent: 200,
    deploymentMinimumHealthyPercent: 100,
    desiredCount: 2,
    launchType: 'FARGATE',

    networkConfiguration:{
        subnets: [subnet1,subnet2],
        assignPublicIp: false,
        securityGroups: [sg]
    },
    healthCheckGracePeriodSeconds: 20,
    loadBalancers:[
        {
            containerName:'UsersContainer',
            containerPort:'80',
            targetGroupArn: UsersTG1.arn

        }
    ],   
},
{
    ignoreChanges: ['desiredCount',
                'loadBalancers',
                'taskDefinition',
                ],
    transformations: [
        args => {
        if (args.type === 'aws:ecs/service:Service') {
            return {
            props: args.props,
            opts: pulumi.mergeOptions(args.opts, {
                ignoreChanges: [
                'desiredCount',
                'loadBalancers',
                'taskDefinition',
                ],
            }),
            };
        }
        return undefined;
        },
    ]
});

@morinrod-cardioscan morinrod-cardioscan changed the title Deploying ECS service with blue/green deployment doesn't ignore task definition and target group changes After deploying ECS service with blue/green deployment, task definition and target group changes can't be ignored Aug 27, 2020
@morinrod-cardioscan morinrod-cardioscan changed the title After deploying ECS service with blue/green deployment, task definition and target group changes can't be ignored When deploying ECS service with blue/green deployment, task definition and target group changes can't be ignored Aug 27, 2020
@leezen
Copy link
Contributor

leezen commented Aug 27, 2020

Does pulumi refresh help with this? That would sync the actual state to your current state and then I would expect the subsequent update to be correct once the code is also updated to reflect the state?

@morinrod-cardioscan
Copy link
Author

morinrod-cardioscan commented Aug 28, 2020

I tried running pulumi refresh, when running pulumi up again without making any changes, there are still changes detected on the service.
Changes were also detected on the listener and deployment group - since there was a deployment(blue\green), the target group has been changed.

@iwahbe iwahbe added kind/bug Some behavior is incorrect or out of spec bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. labels Jul 17, 2023
@diegosasw
Copy link

is there any update on this?

What's the recommended approach when we have, for example, an AWS Fargate with services and task definitions, and we want to deploy new version of a service, but ignore existing ones?

I thought of running pulumi up with some env variables, and then, if the env variable was not provided, ignore containerDefinitions, for example. But that's not ideal, as other resources are created. And, of course, I don't want to delete the applications that don't require new deployment.

@t0yv0
Copy link
Member

t0yv0 commented Mar 28, 2024

Possibly related: pulumi/pulumi-terraform-bridge#1756

@t0yv0 t0yv0 added the service/ecs ECS related things label Mar 28, 2024
@dan-cooke
Copy link

I've spent most of the day on this, and I have hacked a solution together. From looking at old terraform issues / CDK issues - its quite disappointing that we have to hack around AWS on this.

In an ideal world AWS ECS should just turn the task definition update into a new deployment on code deploy, I found a relevant issue on CloudFormation aws-cloudformation/cloudformation-coverage-roadmap#1529

But I don't believe pulumi or terraform utilise CloudFormation - so I'm resorting to a hacky method while I investigate further.

But here's what I'm doing

  1. Create ECS service with a hardcoded image, nginx:latest is what I'm using
const service = new awsx.ecs.FargateService(
  'templi-api',
  {
    cluster: baseInfra.requireOutput('clusterArn'),
    enableExecuteCommand: true,

    deploymentController: {
      type: 'CODE_DEPLOY',
    },
    networkConfiguration: {
      securityGroups: [apiSecurityGroup.id],
      assignPublicIp: true,
      subnets: baseInfra.requireOutput('vpcPublicSubnetIds'),
    },
    continueBeforeSteadyState: true,

    // TODO: this is hardcoded as nginx due to a bug with pulumi not being able
    // to ignore changes on the task definition
    // As our deployments are handled by code deploy, we should ignore changes on the task definition
    taskDefinition: 'nginx:latest',
    loadBalancers: [
      {
        containerPort: 80,
        containerName: 'templi-api',
        targetGroupArn: blueTargetGroup.arn,
      },
    ],
  },
  {
    ignoreChanges: ['taskDefinition'],
  }
);
  1. Use your latest task definition to trigger a pulumi/command to go through the aws deploy CLI
const deployment = latestTaskDefinitionArn.apply((arn) => {
  const appSpecJson = JSON.stringify({
    version: 0.0,
    Resources: [
      {
        TargetService: {
          Type: 'AWS::ECS::Service',
          Properties: {
            TaskDefinition: arn,
            LoadBalancerInfo: {
              ContainerName: 'templi-api',
              ContainerPort: 80,
            },
          },
        },
      },
    ],
  });

  const cliInputJson = JSON.stringify({
    applicationName: 'templi-api',
    deploymentGroupName: 'api-deployment-group',
    revision: {
      revisionType: 'AppSpecContent',
      appSpecContent: {
        content: appSpecJson,
      },
    },
  });

  return new local.Command('create-deployment', {
    create: `aws deploy create-deployment --cli-input-json '${cliInputJson}'`,
    environment: {
      CACHE_BUSTER: new Date().getTime().toString(),
    },
  });
});

export const deploymentId = deployment.stdout;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug/diff kind/bug related to Pulumi generating wrong diffs on preview or up. kind/bug Some behavior is incorrect or out of spec service/ecs ECS related things
Projects
None yet
Development

No branches or pull requests

6 participants