New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement ``Engine`` and ``Runnable`` abstractions. #454

Merged
merged 6 commits into from May 15, 2016

Conversation

Projects
None yet
3 participants
@jmchilton
Copy link
Member

jmchilton commented Apr 15, 2016

Runnables are things such as Galaxy tools, Galaxy workflows, CWL tools, CWL workflows and Engines provide operations over Runnables. There are two Engines right now - cwltool and Galaxy and two command that uses the Engine abstraction (test and run - formerly cwl_run).

Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on aenum is added to get enable nice enum functionality in planemo to describe runnable types.

The Engine abstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements #449 and #364 - at least in some initial fashion.

Initial infrastructure for a generic test command across engine and runnable types is also present (xref #448). If planemo test encounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used by cwltest.py. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly.

This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable. planemo test_report and planemo share_test should work for any Engine and any RunnableType.

Despite not yet using Engines infrastructure, the serve command and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either .ga files directly or Format 2 Workflows through the new gxformat2 library).

In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used.

The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script scripts/test_confromance_cwl.bash that runs the common workflow language conformance tests against Planemo/Galaxy. Fixes #453.

This commit also includes a variety of improvements to run and serve tests.

  • Refactor serve test cases to allow more reuse.
  • Fix profile test to allow rerunning on the same machine more than once.
  • Docstrings.
@coveralls

This comment has been minimized.

Copy link

coveralls commented Apr 15, 2016

Coverage Status

Coverage decreased (-0.1%) to 79.37% when pulling 9b87cc0 on jmchilton:engine into 47bb477 on galaxyproject:master.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 5 times, most recently from 5606bbb to 461d2ce Apr 18, 2016

@coveralls

This comment has been minimized.

Copy link

coveralls commented Apr 22, 2016

Coverage Status

Coverage decreased (-0.09%) to 79.379% when pulling 461d2ce on jmchilton:engine into 069e7ba on galaxyproject:master.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 2 times, most recently from 5a4baee to 2433f6d Apr 25, 2016

@coveralls

This comment has been minimized.

Copy link

coveralls commented Apr 25, 2016

Coverage Status

Coverage increased (+1.6%) to 81.092% when pulling 2433f6d on jmchilton:engine into 473e12d on galaxyproject:master.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 2 times, most recently from 557ccdc to 6acb923 Apr 25, 2016

@coveralls

This comment has been minimized.

Copy link

coveralls commented Apr 25, 2016

Coverage Status

Coverage increased (+1.9%) to 81.263% when pulling 6acb923 on jmchilton:engine into 4035d37 on galaxyproject:master.

@jmchilton jmchilton force-pushed the jmchilton:engine branch from 6acb923 to 672893f Apr 26, 2016

@coveralls

This comment has been minimized.

Copy link

coveralls commented Apr 26, 2016

Coverage Status

Coverage increased (+2.0%) to 81.343% when pulling 672893f on jmchilton:engine into 4035d37 on galaxyproject:master.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 3 times, most recently from 4a7e9c9 to c24e088 Apr 27, 2016

This was referenced May 1, 2016

@jmchilton jmchilton force-pushed the jmchilton:engine branch 7 times, most recently from 83fc78f to e88db78 May 2, 2016

@jmchilton jmchilton force-pushed the jmchilton:engine branch 3 times, most recently from 19fc6a4 to 23d5208 May 9, 2016

@jmchilton jmchilton force-pushed the jmchilton:engine branch 3 times, most recently from fe92e34 to 0c7ffd0 May 10, 2016

@jmchilton jmchilton force-pushed the jmchilton:engine branch from 0c7ffd0 to ed3a896 May 12, 2016

@jmchilton jmchilton referenced this pull request May 12, 2016

Closed

Generalized Test Follow Up #476

0 of 5 tasks complete
Rework Galaxy testing to use structured data instead of XUnit data.
This switches planemo in a subtle way from to depending more on the sturctured data json than the XUnit - this will be benefical for non-Galaxy tool testing when XUnit will be a product of the reporting stuff and not an input to it. This refactoring/reworking should allow most of planemo/galaxy/test/actions to be used with the newer planemo-driven testing stuff.

Adds tests and doc strings.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 4 times, most recently from 043d14e to 526a484 May 12, 2016

jmchilton added some commits Apr 15, 2016

Implement ``Engine`` and ``Runnable`` abstractions.
``Runnable``s are things such as  Galaxy tools, Galaxy workflows, CWL tools, CWL workflows and ``Engine``s provide operations over ``Runnable``s. There are two Engines right now - cwltool and Galaxy and one command that uses the Engine abstraction (``run`` - formerly ``cwl_run``).

Exit code handling is improved to so fixed exit codes result when unknown runnables are encountered or when a runnable is matched with an incompatible engine. A dependency on ``aenum`` is added to get enable nice enum functionality in planemo to describe runnable types.

The ``Engine`` abstraction implements running Galaxy and CWL tools so far (with either Galaxy or cwltool) but this commit contain a lot of infrastructure for running Galaxy tools and workflows of either type. In particular, workflows will now be uploaded as needed and IDs tracked, they can be executed, and what remains is to collect outputs. The generalized run command implements #449 and #364 - at least in some initial fashion.

Initial infrastructure for a generic test command across engine and runnable types is also present (xref #448). If ``planemo test`` encounters a runnable type other than galaxy_tool and directory - such as a cwl tool, a workflow, etc... it will use a new planemo engine-driven testing procedure. To do this an engine "job" is executed, the results collected, and the results "checked". Currently, Galaxy workflows do not have their results collected and the only "check" available is exact non-File matches and sha1 checksums as used by ``cwltest.py``. A complete suite of checks matching what is available in Galaxy will hopefully be implemented shortly.

This new engine-driven testing produces a structured test data structure that should match what is generated by Galaxy regardless of which engine is used. The upshot of this is that existing test report infrastructure in planemo should be reusable. ``planemo test_report`` and ``planemo share_test`` should work for any ``Engine`` and any ``RunnableType``.

Despite not yet using ``Engine``s infrastructure, the ``serve`` command and related infrastructure for Galaxy has been updated to consume runnables instead of paths. An upshot of this is that serve has been updated to allow serving workflows (either ``.ga`` files directly or Format 2 Workflows through the new gxformat2 library).

In order to serve, run, and test gxformat2 workflows - planemo now depends on the gxformat2 library (https://github.com/jmchilton/gxformat2). The depenedency on bioblend has been constrained to be newer so that workflow invocation client stuff can be used.

The commit also contains general improvements for CWL support - in particular the cwl-runner implementation driven by planemo is much enhanced and now handles (in at least a minimal fashion) all arguments required to be used by the conformance test code. This commit also introduces a test script ``scripts/test_confromance_cwl.bash`` that runs the common workflow language conformance tests against Planemo/Galaxy. Fixes #453.

This commit also includes a variety of improvements to ``run`` and ``serve`` tests.

 - Refactor serve test cases to allow more reuse.
 - Fix profile test to allow rerunning on the same machine more than once.
 - Docstrings.
More Work on Slides
 - Start work on workflow SDK slides for GCC/BOSC.
 - Rebuild slides as part of ready-docs Make target.

@jmchilton jmchilton force-pushed the jmchilton:engine branch 2 times, most recently from 430ff44 to 8901104 May 13, 2016

jmchilton added some commits May 14, 2016

@jmchilton jmchilton force-pushed the jmchilton:engine branch from 8901104 to c033491 May 15, 2016

@jmchilton jmchilton changed the title [WIP] Refactor toward abstract description of an "Engine" Implement ``Engine`` and ``Runnable`` abstractions. May 15, 2016

@jmchilton jmchilton merged commit 667e404 into galaxyproject:master May 15, 2016

1 check was pending

continuous-integration/travis-ci/pr The Travis CI build is in progress
Details
@mr-c

This comment has been minimized.

Copy link
Contributor

mr-c commented May 15, 2016

Congratulations! I look forward to trying this out tomorrow :-)

jmchilton added a commit that referenced this pull request May 15, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment