Skip to content

Commit

Permalink
Update README.md for v0.5.0
Browse files Browse the repository at this point in the history
  • Loading branch information
mlin committed Oct 2, 2019
1 parent 14ffb6e commit 6ce7e45
Show file tree
Hide file tree
Showing 2 changed files with 23 additions and 16 deletions.
37 changes: 22 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# miniwdl
**[Workflow Description Language](http://openwdl.org/) toolkit for Python 3.6+**

![Project Status](https://img.shields.io/badge/status-alpha-red.svg)
![Project Status](https://img.shields.io/badge/status-beta-yellow.svg)
[![MIT license](https://img.shields.io/badge/license-MIT-brightgreen.svg)](https://github.com/chanzuckerberg/miniwdl/blob/master/LICENSE)
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
[![Build Status](https://travis-ci.org/chanzuckerberg/miniwdl.svg?branch=master)](https://travis-ci.org/chanzuckerberg/miniwdl) [![Coverage Status](https://coveralls.io/repos/github/chanzuckerberg/miniwdl/badge.svg?branch=master)](https://coveralls.io/github/chanzuckerberg/miniwdl?branch=master)
Expand All @@ -13,8 +13,8 @@
- [Installation](#installation)
- [Command-line tools](#command-line-tools)
- [miniwdl check](#miniwdl-check)
- [miniwdl cromwell](#miniwdl-cromwell)
- [miniwdl run](#miniwdl-run)
- [miniwdl cromwell](#miniwdl-cromwell)
- [WDL Python library](#wdl-python-library)
- [API documentation](#api-documentation)
- [Contributing](#contributing)
Expand Down Expand Up @@ -79,9 +79,13 @@ In addition to its suite of WDL-specific warnings, `miniwdl check` uses [ShellCh

If you haven't installed the PyPI package to get the `miniwdl` entry point, equivalently `PYTHONPATH=$PYTHONPATH:/path/to/miniwdl python3 -m WDL check ...`.

### `miniwdl cromwell`
### `miniwdl run`

miniwdl can run a parallelized workflow on the local host, provided that [Docker is installed](https://docs.docker.com/install/) and the invoking user has [permission to control it](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user). (miniwdl uses the built-in [Docker Swarm](https://docs.docker.com/engine/swarm/) mode, which it'll enable locally if it isn't already.)

This tool provides a nicer command-line interface for running a workflow locally using [Cromwell](https://github.com/broadinstitute/cromwell). Example:
* Start with `miniwdl run_self_test` for a quick viability check.

By analyzing the WDL file, the runner can receive workflow inputs via the command line, as illustrated:

```
$ cat << 'EOF' > hello.wdl
Expand All @@ -100,7 +104,7 @@ task hello {
}
}
EOF
$ miniwdl cromwell hello.wdl
$ miniwdl run hello.wdl
missing required inputs for hello: who
required inputs:
Array[String]+ who
Expand All @@ -109,7 +113,7 @@ optional inputs:
outputs:
Array[String]+ messages
Int meaning_of_life
$ miniwdl cromwell hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
$ miniwdl run hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
{
"outputs": {
"hello.messages": [
Expand All @@ -118,23 +122,22 @@ $ miniwdl cromwell hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
],
"hello.meaning_of_life": 42
},
"id": "b75f3449-344f-45ec-86b2-c004a3adc289",
"dir": "/home/user/20190203_215657_hello"
"dir": "/home/user/20190718_213847_hello"
}
```

By first analyzing the WDL code, this tool translates the freeform command-line arguments into appropriately-typed JSON inputs for Cromwell. It downloads the Cromwell JAR file automatically to a temporary location; a compatible `java` JRE must be available to run it. You can use the `-r/--jar` option if you already have a local copy of Cromwell; other Cromwell configuration options are available (see `--help`). The invoking user must have [permission to control Docker](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user).
Relative or absolute paths are accepted for File inputs. The runner can also provide shell tab-completion for the workflow's available inputs. To use this, enable [argcomplete](https://argcomplete.readthedocs.io/en/latest/) global completion by invoking `activate-global-python-argcomplete` and starting a new shell session. Then, start a command line `miniwdl run hello.wdl ` and try double-tab.

The tool supports shell tab-completion for the workflow's available input names. To use this, enable [argcomplete](https://argcomplete.readthedocs.io/en/latest/) global completion by invoking `activate-global-python-argcomplete` and starting a new shell session. Then, start a command line `miniwdl cromwell hello.wdl ` and try double-tab.
Lastly, inputs can be supplied through a Cromwell-style JSON file; see `miniwdl run --help` for this and other options.

### `miniwdl run`
The miniwdl runner is still in beta testing, and the [Releases](https://github.com/chanzuckerberg/miniwdl/releases) page documents certain existing limitations. If you encounter a WDL 1.0 interoperability problem not mentioned there, please file it via [Issues](https://github.com/chanzuckerberg/miniwdl/issues).

miniwdl's built-in capability to execute workflows on the local host is in early testing. The [Releases](https://github.com/chanzuckerberg/miniwdl/releases) page documents current salient limitations. If you encounter an interoperability problem not mentioned there, we want to hear about it via [Issues](https://github.com/chanzuckerberg/miniwdl/issues)!
### `miniwdl cromwell`

Operation is nearly identical to `miniwdl cromwell`. The invoking user must have [permission to control Docker](https://docs.docker.com/install/linux/linux-postinstall/#manage-docker-as-a-non-root-user).
This tool provides `miniwidl run`'s command-line interface for supplying the workflow's inputs, but calls out to [Cromwell](https://github.com/broadinstitute/cromwell) to actually run it instead of the built-in runtime.

```
$ miniwdl run hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
$ miniwdl cromwell hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
{
"outputs": {
"hello.messages": [
Expand All @@ -143,10 +146,14 @@ $ miniwdl run hello.wdl who=Alyssa "who=Ben Bitdiddle" x=41
],
"hello.meaning_of_life": 42
},
"dir": "/home/user/20190718_213847_hello"
"id": "b75f3449-344f-45ec-86b2-c004a3adc289",
"dir": "/home/user/20190203_215657_hello"
}
```


It downloads the Cromwell JAR file automatically to a temporary location; a compatible `java` JRE must be available. You can use the `-r/--jar` option if you already have a local copy of Cromwell; other Cromwell configuration options are available (see `miniwdl cromwell --help`).

## WDL Python library

The `WDL` package provides programmatic access to the WDL parser and AST. The following example prints all declarations in a workflow, descending into `scatter` and `if` stanzas as needed.
Expand Down
2 changes: 1 addition & 1 deletion WDL/CLI.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,7 +275,7 @@ async def read_source(uri, path, importer_uri):
def fill_run_subparser(subparsers):
run_parser = subparsers.add_parser(
"run",
help="Run workflow/task locally with built-in runtime [alpha test]",
help="Run workflow/task locally with built-in runtime [beta test]",
epilog="NO_COLOR= environment variable disables terminal colors in log messages.",
)
run_parser.add_argument("uri", metavar="URI", type=str, help="WDL document filename/URI")
Expand Down

0 comments on commit 6ce7e45

Please sign in to comment.