Refactor processing parameters #61
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Newer version (0.22.0) is available but not tested. Therefore kept it to what we know works, but update is recommended.
…ses, as scratch location is not JobManager-specific but specific for the concrete on-site cluster setup and should be configurable.
…ingDirectory() to avoid infinite recursion with call from BEJob.getLoggingDirectory. Added isParameterFileEnabled to BEJobManager.
* Reduced number of dependencies on Job in LSFCommand. * Removed previously introduced isParameterFileEnabled: PARAMETER_FILE should be treated like a normal variable.
Removed JobConstants only used in Roddy but not in BE.
…bInfo jobInfo) from JobManagers.
Problem: Previously CONFIG_FILE and PARAMETER_FILE were only used in PBS, so LSF did not work. The ideal solution is to leave the definition and handling of parameter files to the client code or to come up with a clear interface for using parameter files. For the former case it is currently not trivial to remove the parameters from the job (one would have to either create a temporary job object with reduced parameters, or allow explicit setting of parameters during job running. The code is currently not well prepared for this). The latter solution has some unresolved questions: How to set the name of the parameter file variable? Where to save the files (tmp, some notion of execution directory, ...)?
…riables that need to be directly given to the wrapper script (for debugging).
…ter return a constant value. Added some variables from Frank's table of batch processing systems (for workdir and alike). Removed possibility to set a custom variable name as the client code, to actually set these variable was only in one site in Roddy and not even in BE.
…thUniqueValidJobId.
…factored out common code.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.