Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions-tasks): task construct to call RunJob on ECS #8451

Merged
merged 33 commits into from Jun 30, 2020

Conversation

shivlaks
Copy link
Contributor

@shivlaks shivlaks commented Jun 9, 2020

Replacement for the current implementation of ECS service integration.
Merges state and service integration properties and represents them as a
construct.

The notable differences from the current implementation are:

  • single implementation classrunTask with an added launchTarget required
    property where the properties specific to the launch type are specified
  • linter disable of ref-via-interface as we need to use TaskDefinition
    because properties that are not known for imported task definitions are required
  • older implementations have been marked deprecated with direction to the
    replacement.

Left all of the unit tests (except fargate platform version which was not
previously honored) and integ test expectations verbatim to ensure that
we have not lost fidelity.

BREAKING CHANGE:

  • containerName is not supported as an override anymore and has been replaced by containerDefinition

Closes #8610


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@shivlaks shivlaks requested review from rix0rrr and nija-at June 9, 2020 16:36
@shivlaks shivlaks self-assigned this Jun 9, 2020
@shivlaks shivlaks requested a review from a team June 9, 2020 16:36
@ccfife ccfife mentioned this pull request Jun 9, 2020
24 tasks
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Jun 9, 2020
@shivlaks shivlaks requested a review from nija-at June 18, 2020 07:16
Copy link
Contributor

@MrArnoldPalmer MrArnoldPalmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of small nits but the APIs feel right to me. As far as bind vs base class I would lean towards bind as well.

@MrArnoldPalmer
Copy link
Contributor

Changes lgtm. Will leave approval to @nija-at so it doesn’t auto merge before he gets a second look.

Copy link
Contributor

@nija-at nija-at left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take a look at the comment around bind() signature and couple of comments around security groups. These two need to be addressed, a few around code simplification, the rest are just slight adjustments.

Comment on lines +242 to +250
for (const override of this.props.containerOverrides ?? []) {
const name = override.containerDefinition.containerName;
if (!cdk.Token.isUnresolved(name)) {
const cont = this.props.taskDefinition.node.tryFindChild(name);
if (!cont) {
throw new Error(`Overrides mention container with name '${name}', but no such container in task definition`);
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this can be dropped in favour of -

taskDefinition.containers.includes(override.containerDefinition)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought so too, but containers is a protected property and not public

@shivlaks shivlaks requested a review from nija-at June 29, 2020 07:05
@mb-dev
Copy link

mb-dev commented Jun 29, 2020

Wait, launchType is now optional. and if omitted the default capacity provider is used:
https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_RunTask.html#ECS-RunTask-request-launchType

capacity providers are coming to cloudformation soon. should this change be compatible and not assume launch type is EC2 or Fargate?

@shivlaks
Copy link
Contributor Author

@mb-dev - going from required to optional is not a breaking change. My initial instinct is that we can introduce that change when it's more fleshed out or in a follow-on PR.

Is there a use case this would preclude today? In that case, I'd push for implementing it (probably in a follow-up PR as well)

@shivlaks
Copy link
Contributor Author

@mb-dev - just saw that you'd already created #7967 a while back. I'll create a PR to introduce that change. I'd like to keep the scope of this PR contained so we can iterate and keep things moving.

I don't believe capacityProviderStrategy is currently modeled in runTask and we'd need to do that ahead of making launchType optional

@shivlaks shivlaks removed the pr/do-not-merge This PR should not be merged at this time. label Jun 30, 2020
@mb-dev
Copy link

mb-dev commented Jun 30, 2020

@shivlaks what alarmed me is that the PR makes the definition depend more on launch type than before. launchTarget sounds like launchType, and it's tied to either EC2/Fargate. I am not sure if it will be clear how to express a undefined launchType while still specifying parameters to be used at creation.

Currently no launchType might look like:

const runTask = new tasks.EcsRunTask(stack, 'Run', {
    integrationPattern: sfn.IntegrationPattern.RUN_JOB,
    cluster,
This conversation was marked as resolved by shivlaks
    taskDefinition,
    launchTarget: new tasks.EcsEc2LaunchTarget({
      launchType: undefined,
    }),
  });

If launchTarget would be named something like clusterConfiguration it might be easier to parse.

Then again, I realize I am late in commenting - this PR is pretty far along. If you have an idea for a clear syntax that can be introduced in a follow up PR, that will be great. Thanks!

@mergify
Copy link
Contributor

mergify bot commented Jun 30, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@shivlaks
Copy link
Contributor Author

If launchTarget would be named something like clusterConfiguration it might be easier to parse.

Then again, I realize I am late in commenting - this PR is pretty far along. If you have an idea for a clear syntax that can be introduced in a follow up PR, that will be great. Thanks!

That's a fair point, let's continue this conversation in #7967 and get ahead of the potential usability issue / user confusion

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildProject6AEA49D1-qxepHUsryhcu
  • Commit ID: f37e936
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify
Copy link
Contributor

mergify bot commented Jun 30, 2020

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit 13deb26 into master Jun 30, 2020
@mergify mergify bot deleted the shivlaks/sfn-ecs-tasks branch June 30, 2020 07:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FargatePlatformVersion not getting respected in StepFunctions when using CDK
5 participants