galaxyproject / galaxy Public
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow API import/export of format 2 workflows. #6776
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
a90464c
to
bc9717b
50bdfd5
to
eb0ddac
540706a
to
3be0747
Avoid re-arranging dictionaries when JSON is being handled for instance.
mvdbeek
approved these changes
Oct 26, 2018
|
Thanks a million @mvdbeek ! |
nsoranzo
added a commit
to nsoranzo/galaxy
that referenced
this issue
Nov 30, 2018
with `make config-rebuild` . Follow-up on galaxyproject#6776 .
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
This builds on a set of Galaxy PRs #6746, #6807, #6811 and a series of gxformat2 releases.
In addition to bringing in the latest changes to gxformat2 to allow these syntax changes (described below) and updating hundreds of lines of test workflows - this PR finally enables such workflows to work directly with the Galaxy API (switched on
enable_beta_workflow_format). When enabled, workflows imported from dictionaries will now be checked and if they look like a format2 workflow they will be pre-converted to a native format before import - uploading YAML wrapped in JSON to avoid dictionary handling changes is allowed also. Likewise, there is a couple new styles of download that attempt to extract format 2 workflows from the native representation (using new functionality added to recent versions of gxformat2). These can be downloaded as JSON or as YAML string content wrapped up in JSON in order to preserve the pretty YAML formatting and dictionary ordering implemented in gxformat2.If the JSON wrapped YAML-in-strings seems odd - consider also an important future direction of this work is likely to store the original supplied YAML alongside the native representation of the workflow and then allow the workflow editor to operate a series of deltas to both in parallel. This can be done with a round trip aware YAML parsing/writer such as ruamel.yaml so that user comments and formatting as well as extraneous data in the YAML are preserved.
This PR continues to refine the workflow syntax toward a more concise and CWL compatible syntax. The following two code blocks are a before and after example demonstrating the syntax changes.
Before this series of PRs:
My recommended best practice after this PR is merged:
Prior to this pull request, all subworkflows (format-version 0.1 or 2) would be imported repeatedly - once per step that referenced them. This PR introduces a CWL-derived syntax for repeatedly referencing the same workflow and updates the Galaxy import functionality to properly resolve these references and import such workflows only once instead of once per step.
The text was updated successfully, but these errors were encountered: