Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Per user total walltime limit. #3217

Merged
merged 8 commits into from Dec 3, 2016

Conversation

Projects
None yet
6 participants
@maxf130
Copy link
Contributor

commented Nov 24, 2016

Add a limit type that makes it possible to limit a user to a total amount of wall time during a specified time window. If (when a job is submitted) the total wall time for finished jobs within the last n days (specified as window in the configuration) exceeds the specified limit, the job is paused.

@mvdbeek
Copy link
Member

left a comment

This looks very good to me, thank you!

@@ -762,6 +762,13 @@
will be terminated by Galaxy.
-->
<limit type="walltime">24:00:00</limit>
<!-- total_walltime:
Total walltime that jobs may not exceed during during a

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Nov 24, 2016

Member

s/during during/during/

<!-- total_walltime:
Total walltime that jobs may not exceed during during a
set period. If total walltime of finished jobs exceeds
this value, any new jobs are paused. window is a number

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Nov 24, 2016

Member

s/ window/ Window/

This comment has been minimized.

Copy link
@maxf130

maxf130 Nov 25, 2016

Author Contributor

window is referring to an attribute here, so it should probably be lowercase. Maybe wrap in quotations marks?

@@ -6,6 +6,7 @@
import time
import logging
import threading
import datetime

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Nov 24, 2016

Member

This should go to the top

@maxf130

This comment has been minimized.

Copy link
Contributor Author

commented Nov 25, 2016

Pleasure. Is there any way of modifying the files within the pull request? Or should I commit changes to dev and start a new one?

[EDIT]: Nevermind, I figured it out. Pushing to [dev] did the trick.

@maxf130

This comment has been minimized.

Copy link
Contributor Author

commented Nov 28, 2016

I'm not entirely certain why the tests are failing. Can I reproduce them locally?

In principle I could check for the existance of limits.total_walltime before doing anything else, but fixing the test would be the better option.

@nsoranzo

This comment has been minimized.

Copy link
Member

commented Nov 28, 2016

@maxf130 You need to add total_walltime={} also to the Bunch() at the end of JobConfiguration.__parse_job_conf_legacy() method in lib/galaxy/jobs/__init__.py

@maxf130

This comment has been minimized.

Copy link
Contributor Author

commented Nov 28, 2016

@nsoranzo Thanks for the tip.

@nsoranzo

This comment has been minimized.

Copy link
Member

commented Nov 28, 2016

@galaxybot test this

@jmchilton

This comment has been minimized.

Copy link
Member

commented Dec 2, 2016

Thanks for the contribution. Looks pretty good at my first glance but I don't know much about the limits stuff. Ping @natefoo - any thoughts?

@natefoo

This comment has been minimized.

Copy link
Member

commented Dec 2, 2016

LGTM. I think the total runtime-over-period calculation might be pretty expensive for busier sites, and I wonder whether the window ought to accept formats other than just days, but we can probably address those issues if they become problems.

Thanks @maxf130!

@bgruening

This comment has been minimized.

Copy link
Member

commented Dec 3, 2016

@maxf130 great work! Thanks a lot! Do you have a twitter account? We would like to tweet your work and give you some credits!

@bgruening bgruening merged commit 982a35d into galaxyproject:dev Dec 3, 2016

4 checks passed

api test Build finished. 230 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
framework test Build finished. 126 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 580 tests run, 0 skipped, 0 failed.
Details
@maxf130

This comment has been minimized.

Copy link
Contributor Author

commented Dec 5, 2016

No problem! Yes, my handle is @maxf130.
Thanks
Max

@bgruening

This comment has been minimized.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Dec 6, 2017

Migrate documentation for setting up a production instance to Galaxy.
This is a bugfix for the future bug of missing documentation for 17.09 and before. 18.01 will require a significant rewrite of these pages for the uWSGI and configuration changes to Galaxy and the release_17.09 branch of these docs is now being hosted https://docs.galaxyproject.org/en/release_17.09/, so backporting this configuration documentation will allow us to provide permenant links to this 17.09 varaint of all this documentation (with information about paste and galaxy.ini for instance).

This content currently includes:

- the production configuration page
- the cluster configuration page
- the job configuration page
- the scaling information page
- the ftp configuration page
- the top-level nginx page (for external auth content still linking out to the hub)
- the top-level apache page (for external auth content still linking out to the hub)
- the job metrics configuration page

Includes small tweaks to sphinx documentation generation.

- Add a quick build mode that skips source generation / Python docs (export GALAXY_DOCS_SKIP_SOURCE=1 to build this way). This speeds up review of core Sphinx docs.
- Use AutoStructify for better Markdown integration.

Updates to the documentation above and beyond what is on the hub to get it to be closer to reflect the latest and greatest in 17.09 including:

- Update the job configuration page with information about XML macros.
- Mention and link to newer style resubmission condition expressions (not just an enumerated string anymore).
- Updated list of dynamic job runner configuration parameters.
- Mention the ``raw`` attribute for ``env`` directives in the job configuration page.
- Mention the ``total_walltime`` limit added in galaxyproject#3217.
- Drop mentions of older URL-based job conf configuration stuff.
- Mention newer Local and Paramiko based shell runners in CLI section.
- Mention newer slurm job styles in CLI section.
- Improved Pulsar links.
- Fix dataset security link in Apache display sites config.
- Update job metrics page to link to a new Github issue instead of old Trello issues.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Dec 6, 2017

Migrate documentation for setting up a production instance to Galaxy.
This is a bugfix for the future bug of missing documentation for 17.09 and before. 18.01 will require a significant rewrite of these pages for the uWSGI and configuration changes to Galaxy and the release_17.09 branch of these docs is now being hosted https://docs.galaxyproject.org/en/release_17.09/, so backporting this configuration documentation will allow us to provide permenant links to this 17.09 varaint of all this documentation (with information about paste and galaxy.ini for instance).

This content currently includes:

- the production configuration page
- the cluster configuration page
- the job configuration page
- the scaling information page
- the ftp configuration page
- the top-level nginx page (for external auth content still linking out to the hub)
- the top-level apache page (for external auth content still linking out to the hub)
- the job metrics configuration page

Includes small tweaks to sphinx documentation generation.

- Add a quick build mode that skips source generation / Python docs (export GALAXY_DOCS_SKIP_SOURCE=1 to build this way). This speeds up review of core Sphinx docs.
- Use AutoStructify for better Markdown integration.

Updates to the documentation above and beyond what is on the corresponding Hub pages to reflect more of the state of art as of 17.09 including:

- Update the job configuration page with information about XML macros.
- Mention and link to newer style resubmission condition expressions (not just an enumerated string anymore).
- Updated list of dynamic job runner configuration parameters.
- Mention the ``raw`` attribute for ``env`` directives in the job configuration page.
- Mention the ``total_walltime`` limit added in galaxyproject#3217.
- Drop mentions of older URL-based job conf configuration stuff.
- Mention newer Local and Paramiko based shell runners in CLI section.
- Mention newer slurm job styles in CLI section.
- Improved Pulsar links.
- Fix dataset security link in Apache display sites config.
- Update job metrics page to link to a new Github issue instead of old Trello issues.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.