feat(ecs): add support for elastic inference accelerators in ECS task defintions by upparekh · Pull Request #13950 · aws/aws-cdk

upparekh · 2021-04-01T23:25:52Z

This PR would enable users to attach inference accelerators to their ECS tasks (currently not supported by Fargate). It adds an optional inferenceAccelerators property to the EC2 Task Definition. It also adds resourceRequirement property to the Container definition to enable containers to refer to the inference accelerators provided in the task definition.

Closes #12460

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

… definition

gitpod-io · 2021-04-01T23:25:54Z

…elerators

packages/@aws-cdk/aws-ecs/README.md

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

packages/@aws-cdk/aws-ecs/README.md

packages/@aws-cdk/aws-ecs/lib/base/task-definition.ts

SoManyHs · 2021-04-02T04:47:54Z

packages/@aws-cdk/aws-ecs/lib/base/task-definition.ts

+  readonly deviceName?: string;
+
+  /**
+   * The Elastic Inference accelerator type to use.


It might be helpful to include what values are valid here in the docstring (i.e., eia2.medium, eia2.large, and eia2.xlarge, according to the docs).
Normally, having the different valid values as an enum might be nice so that an IDE can pick up the valid values, but the downside is that we'd have to manually add any new types that get added in the future at the risk of always being behind what's available, so I think a string is fine for now.

SoManyHs · 2021-04-02T19:36:57Z

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

+   * The type and amount of a resource to assign to a container.
+   * @default - No resources assigned.
+   */
+  readonly resourceRequirements?: ResourceRequirement[];


I'm wondering if we should have the new field be something that is more specific to EIA, since resourceRequirements in CFN technically encompasses both GPU and EIA. GPU currently is specified through its own field, and it may be confusing for this field to exist when we have GPU specified elsewhere.

I agree, it might get confusing this way. I will make the required changes.

I renamed the resourceRequirements prop to inferenceAcceleratorResources (as inferenceAccelerators was already a task definiton prop) and changed the type to be a list of device names (strings).

…elerators

Updates to documentation and allocation of props in tests Co-authored-by: Hsing-Hui Hsu <hsinghui@amazon.com>

iamhopaul123

Very awesome PR (even without considering it is the first PR that you have) 🎉

packages/@aws-cdk/aws-ecs/README.md

packages/@aws-cdk/aws-ecs/lib/base/task-definition.ts

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

packages/@aws-cdk/aws-ecs/test/ec2/ec2-task-definition.test.ts

Updated docs and sanity check condition Co-authored-by: Penghao He <penghaoh@amazon.com>

…nd updated tests

…elerators

iamhopaul123

Looks good just some nits. Just a reminder but make sure every error you throw is covered in the unit test!

packages/@aws-cdk/aws-ecs/test/ec2/ec2-task-definition.test.ts

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

iamhopaul123 · 2021-04-08T18:04:54Z

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

+function renderResourceRequirements(gpuCount: number = 0, inferenceAcceleratorResources: string[] = []):
+CfnTaskDefinition.ResourceRequirementProperty[] | undefined {
+  if (inferenceAcceleratorResources.length > 0 && gpuCount > 0) {
+    throw new Error('Both inference accelerator and gpu count defined in the container definition.');


Suggested change

throw new Error('Both inference accelerator and gpu count defined in the container definition.');

throw new Error('Cannot define both inference accelerator and gpu count in the container definition.');

We might need to be more specific for the reason why this condition fails https://uxdworld.com/2018/05/30/how-to-write-good-error-messages/
we also need unit test for this.

Discussed about this offline but posting here for the record, we decided to render both gpuCount and inferenceAcceleratorResources properties and let the validators in other stages handle this check.

packages/@aws-cdk/aws-ecs/test/container-definition.test.ts

Updating error messages Co-authored-by: Penghao He <penghaoh@amazon.com>

gitpod-io · 2021-04-09T16:55:56Z

…throwing error

iamhopaul123

LGTM with just two nits. Feel free to take off the do-not-merge label once those are addressed!

iamhopaul123 · 2021-04-14T17:03:33Z

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

+function renderResourceRequirements(gpuCount: number = 0, inferenceAcceleratorResources: string[] = []):
+CfnTaskDefinition.ResourceRequirementProperty[] | undefined {
+  const ret = [];
+  if (inferenceAcceleratorResources.length > 0) {


Just a nit but this check might not be required?

packages/@aws-cdk/aws-ecs/test/container-definition.test.ts

gitpod-io · 2021-04-14T18:20:15Z

Pull request has been modified.

mergify · 2021-04-14T18:52:39Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

…elerators

Pull request has been modified.

mergify · 2021-04-14T23:58:03Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

…elerators

aws-cdk-automation · 2021-04-15T09:50:29Z

AWS CodeBuild CI Report

CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
Commit ID: 790a118
Result: SUCCEEDED
Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

mergify · 2021-04-15T09:50:58Z

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

… defintions (aws#13950) This PR would enable users to attach inference accelerators to their ECS tasks (currently not supported by Fargate). It adds an optional `inferenceAccelerators` property to the EC2 Task Definition. It also adds `resourceRequirement` property to the Container definition to enable containers to refer to the inference accelerators provided in the task definition. Closes aws#12460 *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

upparekh added 2 commits April 1, 2021 16:01

feat(ecs): add support for elastic inference accelerators in ECS task…

c860a93

… definition

docs: update README

f25ad20

upparekh requested a review from SoManyHs April 1, 2021 23:26

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

620061b

…elerators

upparekh added the @aws-cdk/aws-ecs Related to Amazon Elastic Container label Apr 2, 2021

upparekh self-assigned this Apr 2, 2021

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

7c1c396

…elerators

SoManyHs reviewed Apr 2, 2021

View reviewed changes

piradeepk and others added 2 commits April 2, 2021 16:01

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

a600050

…elerators

Apply suggestions from code review

809e72e

Updates to documentation and allocation of props in tests Co-authored-by: Hsing-Hui Hsu <hsinghui@amazon.com>

upparekh requested a review from iamhopaul123 April 2, 2021 21:10

iamhopaul123 reviewed Apr 2, 2021

View reviewed changes

upparekh and others added 2 commits April 5, 2021 11:01

Apply suggestions from code review 2

2eca0bb

Updated docs and sanity check condition Co-authored-by: Penghao He <penghaoh@amazon.com>

Replaced resourceRequirements prop to inferenceAcceleratorResources a…

dd60de8

…nd updated tests

upparekh requested review from SoManyHs and iamhopaul123 April 6, 2021 17:29

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

bbcbee6

…elerators

iamhopaul123 reviewed Apr 8, 2021

View reviewed changes

Apply suggestions from code review

dd28b7a

Updating error messages Co-authored-by: Penghao He <penghaoh@amazon.com>

upparekh added 2 commits April 9, 2021 16:37

Added and updated unit tests

d640f7a

Render both gpu count and inference accelerator resources instead of …

0aaaab6

…throwing error

iamhopaul123 previously approved these changes Apr 14, 2021

View reviewed changes

iamhopaul123 added the pr/do-not-merge This PR should not be merged at this time. label Apr 14, 2021

Updated unit test

89a20bd

iamhopaul123 previously approved these changes Apr 14, 2021

View reviewed changes

iamhopaul123 removed the pr/do-not-merge This PR should not be merged at this time. label Apr 14, 2021

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

3703898

…elerators

iamhopaul123 approved these changes Apr 14, 2021

View reviewed changes

Merge branch 'master' into upparekh/support-for-elastic-inference-acc…

790a118

…elerators

mergify bot merged commit 23986d7 into aws:master Apr 15, 2021

upparekh deleted the upparekh/support-for-elastic-inference-accelerators branch April 15, 2021 22:40

	throw new Error('Both inference accelerator and gpu count defined in the container definition.');
	throw new Error('Cannot define both inference accelerator and gpu count in the container definition.');

Conversation

upparekh commented Apr 1, 2021

Uh oh!

gitpod-io bot commented Apr 1, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

SoManyHs Apr 2, 2021

Choose a reason for hiding this comment

Uh oh!

SoManyHs Apr 2, 2021

Choose a reason for hiding this comment

Uh oh!

upparekh Apr 2, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

upparekh Apr 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iamhopaul123 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iamhopaul123 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iamhopaul123 Apr 8, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

upparekh Apr 14, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gitpod-io bot commented Apr 9, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

iamhopaul123 left a comment

Choose a reason for hiding this comment

Uh oh!

iamhopaul123 Apr 14, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

gitpod-io bot commented Apr 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mergify bot commented Apr 14, 2021

Uh oh!

mergify bot commented Apr 14, 2021

Uh oh!

aws-cdk-automation commented Apr 15, 2021

AWS CodeBuild CI Report

Uh oh!

mergify bot commented Apr 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gitpod-io bot commented Apr 1, 2021 •

edited

Loading

upparekh Apr 2, 2021 •

edited

Loading

upparekh Apr 6, 2021 •

edited

Loading

iamhopaul123 Apr 8, 2021 •

edited

Loading

gitpod-io bot commented Apr 9, 2021 •

edited

Loading

gitpod-io bot commented Apr 14, 2021 •

edited

Loading