New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create the $GALAXY_MEMORY_MB tool job environment variable #4958

Merged
merged 2 commits into from Nov 13, 2017

Conversation

Projects
None yet
2 participants
@natefoo
Member

natefoo commented Nov 8, 2017

And attempt to set it automatically under Slurm. See my malloc testing tool for a usage example. Fixes #4741.

@@ -0,0 +1,4 @@
if [ -n "$SLURM_JOB_ID" ]; then
GALAXY_MEMORY_MB=`scontrol -do show job "$SLURM_JOB_ID" | sed 's/.*\( \|^\)Mem=\([0-9][0-9]*\)\( \|$\).*/\2/p;d'`

This comment has been minimized.

@jmchilton

jmchilton Nov 10, 2017

Member

I thought you were going to go out of your way to hide problems if this didn't work automatically?

As it stands now - this can cause jobs to fail by spitting out things to stderr right? Or is that assumption wrong?

This comment has been minimized.

@natefoo

natefoo Nov 10, 2017

Member

I was working under the false assumption that only the tool's stderr counts anymore. Perhaps that should be the case (in a separate PR)?

When setting $GALAXY_MEMORY_MB, redirect stderr to memory_statement.log
in the job directory.

Also, use -1 as the default for the final test to see if
$GALAXY_MEMORY_MB is a positive integer. This allows an admin to
differentiate between a legit case where we don't set $GALAXY_MEMORY_MB
(currently any DRM other than Slurm) and where the command ran but
returned nothing on stdout.

@jmchilton jmchilton merged commit a18835b into galaxyproject:dev Nov 13, 2017

7 checks passed

api test Build finished. 310 tests run, 4 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 162 tests run, 0 skipped, 0 failed.
Details
integration test Build finished. 57 tests run, 0 skipped, 0 failed.
Details
lgtm analysis: JavaScript No alert changes
Details
selenium test Build finished. 98 tests run, 1 skipped, 0 failed.
Details
toolshed test Build finished. 577 tests run, 0 skipped, 0 failed.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment