0.10.1 sge #152
Merged
0.10.1 sge #152
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
SGE job arrays allow one job id to be associated with a set of processes on an SGE cluster. modified SGEEngineSet to be a subclass PBSEngineSet ipcluster will now generate a default SGE job script if --sge-script is not provided. most folks should ignore the --sge-script option unless they know they need it. if --sge-script is passed, check that the script exists and that the user has defined a "#$ -t" setting within the script. if not, add the setting for them by copying the script to a temp file and launching the job array using the modified temp file. ipengines terminate cleanly now when the ipcluster command exits. i think we still need to handle furl files when engines are assigned to different hosts via SGE. without using ssh or nfs, the only other way is to put the contents of the furl in the job script before submission but this is less secure. need to discuss this.
refactored base class BatchEngineSet to handle job arrays. PBS and SGE engine sets are much simpler now.
LSF requires that job scripts be submitted to bsub through stdin. twisted.internet.utils.getProcessOutput does not support shell redirection so the bsub wrapper takes care of doing this.
… posix compatibility
|
Many thanks for this, guys! Merged and closed with f780f2e. |
minrk
pushed a commit
to minrk/ipython
that referenced
this pull request
Jul 1, 2013
four small fixes
This pull request was closed.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
ssh now supports sending local environment variables over with the sshx.sh script. this may or may not be useful in all situations, but one thing we should try to determine while testing is whether this generates any security loopholes.