Overhaul of Tools and Jobs for 16.04 #1688
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Feb 5, 2016
jmchilton
force-pushed
the
16.04_jobs
branch
5 times, most recently
from
a362f94
to
c4eadc6
Compare
jmchilton
force-pushed
the
16.04_jobs
branch
2 times, most recently
from
9d99e85
to
fe0c6bb
Compare
| @@ -1200,7 +1231,8 @@ def finish( self, stdout, stderr, tool_exit_code=None, remote_working_directory= | |||
| # either use the metadata from originating output dataset, or call set_meta on the copies | |||
| # it would be quicker to just copy the metadata from the originating output dataset, | |||
| # but somewhat trickier (need to recurse up the copied_from tree), for now we'll call set_meta() | |||
| if ( self.app.config.retry_metadata_internally and | |||
| retry_internally = util.asbool(self.get_destination_configuration("retry_metadata_internally", True)) | |||
| if ( retry_internally and | |||
There was a problem hiding this comment.
Will rebase with this change, thanks.
This reverts commit 602dc7f. 16.04 will use bash as the default.
Newer tool formats (CWL and YAML just enable this by default), but XML tools can add a strict="true" to the command block to force "set -e" handling on the shell part of tool execution. Run the framework tests verifying the new and older default behavior using the following commands: ``` ./run_tests.sh -framework -id strict_shell_default_off ./run_tests.sh -framework -id strict_shell ```
- set -e in tool command scripts. - Disable implicit extra file collection. All dynamic extra file collection requires a <discover_datasets tag. - Disable format="input" - require explicit metadata targets (metadata_source, format_source). - Disable interpreter= - hacky and doesn't work as expected in too many circumstances. Use $__tool_directory__. - Disable $param_file - grepping the command for the string is impercise. If useful we can add flag to match the flag used to produce inputs as a JSON file. - Disable default version of 1.0.0, the IUC has made it clear they prefer all tools have an explicit version. Fixes galaxyproject#1609.
|
👍 |
|
Looks good to me (: |
guerler
changed the title
bgruening
added a commit
that referenced
this pull request
Feb 27, 2016
Overhaul of Tools and Jobs for 16.04
|
Thanks @jmchilton and @guerler! |
|
@jmchilton @guerler Do you mind to give an example of |
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Mar 22, 2016
Thanks for @kellrott for the bug report. \#minor
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Mar 22, 2016
Thanks for @kellrott for the bug report. \#minor
dannon
added a commit
that referenced
this pull request
Mar 22, 2016
Fix docker tool execution for #1688.
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Mar 31, 2016
Merged
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Mar 31, 2016
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Apr 1, 2016
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Apr 5, 2016
Broken in 16.04 with 81b5ca5 in PR galaxyproject#1688. Minor.
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
May 19, 2016
dannon
pushed a commit
that referenced
this pull request
May 20, 2016
1 task
manabuishii
added a commit
to manabuishii/galaxy
that referenced
this pull request
Aug 5, 2016
When docker jobs is run, it try to execute script in working directory. I think this related galaxyproject#1688 and 975937e
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Aug 5, 2016
The default for this was including GALAXY_ROOT so in testing (and many deployments) the job directory would have been picked up anyway because it is in `database/` but if job directory was outside of `$GALAXY_ROOT/database/` one needed to add it to the volumes mounted by Docker. This fix mounts just that directory - so no need for instance to mount all job directories which one could have done.
jmchilton
added a commit
to jmchilton/galaxy
that referenced
this pull request
Aug 5, 2016
The default for this was including GALAXY_ROOT so in testing (and many deployments) the job directory would have been picked up anyway because it is in `database/` but if job directory was outside of `$GALAXY_ROOT/database/` one needed to add it to the volumes mounted by Docker. This fix mounts just that directory - so no need for instance to mount all job directories which one could have done.
martenson
added a commit
that referenced
this pull request
Aug 8, 2016
…y_default [16.04] Fix to mount job directory in Docker for #1688.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a fairly substantial cleanup of tool and job handling.
profile="16.04"attribute tools, the resulting tools have all new defaults that should really simplify tool development. Schema Version 2 Tools #1609#set -eon tool executions.interpreterattribute for instance.