Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[aws batch] Runner should automatically provide snakemake with the appropriate --cores N option #65

Closed
tsibley opened this issue Mar 27, 2020 · 0 comments · Fixed by #81
Closed

[aws batch] Runner should automatically provide snakemake with the appropriate --cores N option #65

tsibley opened this issue Mar 27, 2020 · 0 comments · Fixed by #81
Assignees
@tsibley
Labels
enhancement New feature or request

Comments

@tsibley
Copy link
Member

tsibley commented Mar 27, 2020

If the AWS Batch runner is executing snakemake, it should provide a default --cores N / --jobs N option where N defaults to the number of CPUs in the Batch job definition (which may be overridden by the environment or CLI options).

This will let the Nextstrain build use all the available resources by default, instead of defaulting to a single core.

Any user-provided --cores, --jobs, or -j option shouldn't be overridden.
@tsibley tsibley added the enhancement New feature or request label Mar 27, 2020
@tsibley tsibley changed the title AWS Batch runner should automatically provide snakemake with the appropriate --cores N option [aws batch] Runner should automatically provide snakemake with the appropriate --cores N option May 1, 2020
@tsibley tsibley mentioned this issue May 2, 2020
Closed
@tsibley tsibley self-assigned this May 20, 2020
tsibley added a commit that referenced this issue May 21, 2020
@tsibley
build: Add --cpus and --memory options
9635b68 
Limits containerized (Docker, AWS Batch) builds to the given amounts and
when applicable, informs Snakemake's resource scheduler and/or the AWS
Batch instance size selection.

Resolves #65 and #77.
@tsibley tsibley mentioned this issue May 21, 2020
Merged
tsibley added a commit that referenced this issue May 22, 2020
@tsibley
build: Add --cpus and --memory options
4f09456 
Limits containerized (Docker, AWS Batch) builds to the given amounts and
when applicable, informs Snakemake's resource scheduler and/or the AWS
Batch instance size selection.

Resolves #65 and #77.
tsibley added a commit that referenced this issue May 22, 2020
@tsibley
build: Add --cpus and --memory options
881732a 
Limits containerized (Docker, AWS Batch) builds to the given amounts and
when applicable, informs Snakemake's resource scheduler and/or the AWS
Batch instance size selection.

Resolves #65 and #77.
@tsibley tsibley closed this as completed in acb2f3c Jun 1, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tsibley tsibley

Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[build] Add --cpus and --memory options nextstrain/cli
1 participant
@tsibley