Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ECS, Fargate] CloudFormation support for scheduled tasks with Fargate Launch Type #392

Closed
coultn opened this issue Jun 28, 2019 · 15 comments
Labels
ECS Amazon Elastic Container Service Fargate AWS Fargate

Comments

@coultn
Copy link

coultn commented Jun 28, 2019

ECS supports scheduled tasks using CloudWatch events rules. This issue is specifically to add CloudFormation support for scheduled tasks when using the Fargate launch type.

@mildebrandt
Copy link

Isn't this supported already? https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-rule.html

For example, to run a task every 20 minutes:

apiScheduledTaskRule:
    Type: AWS::Events::Rule
    Properties:
      ScheduleExpression: 'cron(*/20 * * * ? *)'
      State: ENABLED
      Targets:
        - Arn: <your cluster ARN>
          RoleArn: <your task execution role ARN>
          Id: apiScheduledTask
          EcsParameters:
            TaskDefinitionArn: <reference to your task definition>
            LaunchType: FARGATE
            NetworkConfiguration:
              AwsVpcConfiguration:
                SecurityGroups:
                  - <your security groups>
                Subnets:
                  - <your subnets>

We do this to regularly run tasks.

Of course, replace ScheduleExpression with EventPattern for CloudWatch events. Does that do what you're expecting, or am I missing something?

@m8786
Copy link

m8786 commented Jul 27, 2019

I was just wrestling with this challenge for the first time this week. When you add a scheduled ECS task through the console and describe it through the CLI, that is the structure that you see. However, the EcsParameters Cloudformation documentation doesn't list all of those things:
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-events-rule-ecsparameters.html
It only has TaskDefinition and TaskCount. Whlie I can add LaunchType: Fargate even though it's not in the documentation, CloudFormation then says: Parameter NetworkConfiguration must be specified for target <service name>-Fargate-Task when launch type is FARGATE (Service: AmazonCloudWatchEvents; Status Code: 400; Error Code: ValidationException; Request ID: ...). And when I add an AwsVpcConfiguration, it says: Encountered unsupported property awsvpcConfiguration. (I've tried with both lowercase and capitalized awsvpcConfiguration, in both aws events put-targets from the CLI and the output from aws events list-targets-by-rule on a scheduled task created in the console, it is formatted as above). Here's what I was using, maybe I'm missing something:

    Type: AWS::Events::Rule
    Properties:
      Description: !Sub 'Trigger ${ServiceName} according to the specified schedule'
      ScheduleExpression: cron(10 * * * ? *)
      State: ENABLED
      Targets:
      - Id: !Sub '${ServiceName}-Fargate-Task'
        RoleArn: <imported role arn>
        EcsParameters:
          TaskDefinitionArn: !Ref TaskDefinition
          TaskCount: 1
          LaunchType: 'FARGATE'
          PlatformVersion: 'LATEST'
          NetworkConfiguration:
            awsvpcConfiguration:
              AssignPublicIp: DISABLED
              SecurityGroups:
                - <imported security group>
              Subnets:
                - <imported subnet 1>
                - <imported subnet 2>
        Arn: <imported cluster arn>

Edit: This is all being done in N. Virginia/us-east-1.

@mildebrandt
Copy link

Have you used the parameter name as it's listed in my example? It's AwsVpcConfiguration, capital A, V, and C.

@m8786
Copy link

m8786 commented Jul 27, 2019

Wow, you have to be kidding me. I swear I tried that combination, I was messing with this for hours but it seemed to work today! Here's the redacted template that I used for anyone's future reference:

    Type: AWS::Events::Rule
    Properties:
      Description: !Sub 'Trigger ${ServiceName} according to the specified schedule'
      ScheduleExpression: cron(10 * * * ? *)
      State: ENABLED
      Targets:
      - Id: !Sub '${ServiceName}-Fargate-Task'
        RoleArn: <Role Arn>
        EcsParameters:
          TaskDefinitionArn: !Ref TaskDefinition
          TaskCount: 1
          LaunchType: 'FARGATE'
          PlatformVersion: 'LATEST'
          NetworkConfiguration:
            AwsVpcConfiguration:
              AssignPublicIp: DISABLED
              SecurityGroups:
                - <Security Group>
              Subnets:
                - <Subnets>
        Arn: <Role ARN.

I guess AWS just needs to update their documentation. Thanks!

@deleugpn
Copy link

Careful with undocumented CloudFormation features. It could be the same thing that happened with the use of Secrets on CFN. They could be working on it and they have every right to simply revert their changes and bring it back at a later time since they haven't released it yet. It could be that today you get your template created successfully and maybe 2 days from now you try to update the template and get an error saying invalid syntax because it has changed and you don't know to what.

@georgantasp
Copy link

I'm successfully using these undocumented parameters thanks to this issue. 👍

@coultn
Copy link
Author

coultn commented Aug 30, 2019

Shipped: https://docs.aws.amazon.com/en_us/AWSCloudFormation/latest/UserGuide/ReleaseHistory.html

@coultn coultn closed this as completed Aug 30, 2019
@coultn coultn moved this from Coming Soon to Just Shipped in containers-roadmap Aug 30, 2019
@whereisaaron
Copy link

@m8786 you shouldn't kick your self for the field name AwsVpcConfiguration vs AwsvpcConfiguration because here's the thing:

  1. For AWS::Events::Rule the field name is AwsVpcConfiguration, but...
  2. For AWS::ECS::Service the field name is AwsvpcConfiguration!!!

Both captializations are used by AWS in different API's, so no wonder we users are confused!

@leacoco
Copy link

leacoco commented Dec 13, 2019

It is very confusing indeed

@brsanthu
Copy link

brsanthu commented Feb 28, 2020

To add on more confusion, the rule doc page uses awsvpcConfiguration

sigh!

@vpbobade
Copy link

vpbobade commented Apr 6, 2020

Hello,
I have a taskdefinition to launch a ECS fargate task however I do not want to create a service instead want to trigger it with Cloudwatch events rule. Could you please suggest if this is possible if not any other way we can achieve this. Sample snippet if any would also suffice.

TIA

@mibollma
Copy link

  1. For AWS::Events::Rule the field name is AwsVpcConfiguration, but...
  2. For AWS::ECS::Service the field name is AwsvpcConfiguration!!!

Both captializations are used by AWS in different API's, so no wonder we users are confused!

Just ran into this issue and can confirm the capitalization of the same type is still different across those two services

@dezren39
Copy link

I have ran into, forgotten about, and ran into this issue again multiple times. Wishing for consistency or case insensitivity.

@tfitz237
Copy link

The case inconsistency is still a thing in 2022!

@scottgigante
Copy link

Hello from the future where this is still a thing in 2023. Thank you all for making me feel better about my last hour of work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ECS Amazon Elastic Container Service Fargate AWS Fargate
Projects
Development

No branches or pull requests