Skip to content

runner.aws_batch: Use containerOverrides.resourceRequirements instead of .vcpu and .memory#177

Merged
tsibley merged 1 commit intomasterfrom
trs/aws-batch-cpus-and-memory
May 27, 2022
Merged

runner.aws_batch: Use containerOverrides.resourceRequirements instead of .vcpu and .memory#177
tsibley merged 1 commit intomasterfrom
trs/aws-batch-cpus-and-memory

Conversation

@tsibley
Copy link
Contributor

@tsibley tsibley commented May 26, 2022

Job submissions with resourceRequirements correctly override the
defaults from job definitions that use the newer resourceRequirements
and those that use the older, deprecated vcpu and memory fields. Job
submissions using the older fields only override job definitions that
also use the older fields; otherwise they're ignored with only an easy
to miss warning in the AWS console to alert you.

Notably, using the AWS console to modify an existing job definition
which uses vcpus and memory will switch the definition to
resourceRequirements automatically in the new revision. This meant
revising your old job definition in the AWS console could break --cpus
and --memory for your nextstrain build invocations. It also means
that --cpus and --memory would never have worked if your AWS job
definitions were originally created with resourceRequirements.

Resolves #144.

Testing

See #144 for how/what I tested.

… of .vcpu and .memory

Job submissions with resourceRequirements correctly override the
defaults from job definitions that use the newer resourceRequirements
and those that use the older, deprecated vcpu and memory fields.  Job
submissions using the older fields only override job definitions that
also use the older fields; otherwise they're ignored with only an easy
to miss warning in the AWS console to alert you.

Notably, using the AWS console to modify an existing job definition
which uses vcpus and memory will switch the definition to
resourceRequirements automatically in the new revision.  This meant
revising your old job definition in the AWS console could break --cpus
and --memory for your `nextstrain build` invocations.  It also means
that --cpus and --memory would never have worked if your AWS job
definitions were originally created with resourceRequirements.

Resolves <#144>.
@tsibley tsibley requested a review from a team May 26, 2022 22:47
Copy link
Member

@victorlin victorlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't test personally, but your testing looks good.

@tsibley tsibley merged commit 811f53a into master May 27, 2022
@tsibley tsibley deleted the trs/aws-batch-cpus-and-memory branch May 27, 2022 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

No open projects

Development

Successfully merging this pull request may close these issues.

[build] --cpus and --memory don't work with newer AWS Batch job definitions

2 participants