Permalink
Browse files

Command line parsing redo.

  • Loading branch information...
1 parent 13691aa commit 308f087287f57585de89c7c41f371db031fb2d7f @mcovarr mcovarr committed with mcovarr Jun 16, 2017
View
@@ -6,6 +6,9 @@
* Request timeouts for HTTP requests on the REST API now return a 503 status code instead of 500. The response for a request timeout is no longer in JSON format.
+* Command line usage has been extensively revised for Cromwell 29. Please see the
+[README](https://github.com/broadinstitute/cromwell#command-line-usage) for details.
+
## 28
### Bug Fixes
View
202 README.md
@@ -17,7 +17,7 @@ A [Workflow Management System](https://en.wikipedia.org/wiki/Workflow_management
* [Building](#building)
* [Installing](#installing)
* [Upgrading from 0.19 to 0.21](#upgrading-from-019-to-021)
-* [**NEW** Command Line Usage](http://gatkforums.broadinstitute.org/wdl/discussion/8782/command-line-cromwell) (on the WDL/Cromwell Website)
+* [Command Line Usage](#command-line-usage)
* [Getting Started with WDL](#getting-started-with-wdl)
* [WDL Support](#wdl-support)
* [Configuring Cromwell](#configuring-cromwell)
@@ -138,6 +138,206 @@ OS X users can install Cromwell with Homebrew: `brew install cromwell`.
See the [migration document](MIGRATION.md) for more details.
+# Command Line Usage
+
+For built-in documentation of Cromwell command line usage, run the Cromwell JAR file with no arguments:
+
+```
+$ java -jar cromwell-<versionNumber>.jar
+```
+
+For example, `$ java -jar cromwell-29.jar`. You will get a usage message like the following:
+
+```
+cromwell 29
+Usage: java -jar /path/to/cromwell.jar [server|run] [options] <args>...
+
+ --help Cromwell - Workflow Execution Engine
+ --version
+Command: server
+Starts a web server on port 8000. See the web server documentation for more details about the API endpoints.
+Command: run [options] workflow-source
+Run the workflow and print out the outputs in JSON format.
+ workflow-source Workflow source file.
+ -i, --inputs <value> Workflow inputs file.
+ -o, --options <value> Workflow options file.
+ -t, --type <value> Workflow type.
+ -v, --type-version <value>
+ Workflow type version.
+ -l, --labels <value> Workflow labels file.
+ -p, --imports <value> A directory or zipfile to search for workflow imports.
+ -m, --metadata-output <value>
+ An optional directory path to output metadata.
+```
+
+## --version
+
+The `--version` option prints the version of Cromwell and exits.
+
+## --help
+
+The `--help` option prints the full help text above and exits.
+
+## server
+
+The `server` command runs Cromwell as a web server. No arguments are accepted.
+See the documentation for Cromwell's REST endpoints [here](#rest-api).
+
+## run
+
+The `run` command executes a single workflow in Cromwell.
+
+### workflow-source
+The `run` command requires a single argument for the workflow source file.
+
+### --inputs
+An optional file of workflow inputs. Although optional, it is a best practice to use an inputs file to satisfy workflow
+requirements rather than hardcoding inputs directly into a workflow source file.
+
+### --options
+An optional file of workflow options. Some options are global (supported by all backends), while others are backend-specific.
+See the [workflow options](#workflow-options) documentation for more details.
+
+### --type
+An optional parameter to specify the language for the workflow source. Any value specified for this parameter is currently
+ignored and internally the value `WDL` is used.
+
+### --type-version
+An optional parameter to specify the version of the language for the workflow source. Currently any specified value is ignored.
+
+### --labels
+An optional parameter to specify a file of JSON key-value label pairs to associate with the workflow.
+
+### --imports
+You have the option of importing WDL workflows or tasks to use within your workflow, known as sub-workflows.
+If you use sub-workflows within your primary workflow then you must include a zip file with the WDL import files.
+
+For example, say you have a directory of WDL files:
+
+```
+wdl_library
+└──cgrep.wdl
+└──ps.wdl
+└──wc.wdl
+```
+
+If you zip that directory into `wdl_library.zip`, then you can reference and use these WDLs within your primary WDL.
+
+This could be your primary WDL:
+
+```
+import "ps.wdl" as ps
+import "cgrep.wdl"
+import "wc.wdl" as wordCount
+
+workflow my_wf {
+
+call ps.ps as getStatus
+call cgrep.cgrep { input: str = getStatus.x }
+call wordCount { input: str = ... }
+
+}
+```
+
+Then to run this WDL without any inputs, workflow options, or metadata files, you would enter:
+
+`$ java -jar cromwell-<versionNumber>.jar run my_wf.wdl --imports /path/to/wdl_library.zip`
+
+### --metadata-output
+
+You can include a path where Cromwell will write the workflow metadata JSON, such as start/end timestamps, status, inputs, and outputs. By default, Cromwell does not write workflow metadata.
+
+This example includes a metadata path called `/path/to/my_wf.metadata`:
+
+```
+$ java -jar cromwell-<versionNumber>.jar run my_wf.wdl --metadata-output /path/to/my_wf.metadata
+```
+
+Again, Cromwell is very verbose. Here is the metadata output in my_wf.metadata:
+
+```
+{
+ "workflowName": "my_wf",
+ "submittedFiles": {
+ "inputs": "{\"my_wf.hello.addressee\":\"m'Lord\"}",
+ "workflow": "\ntask hello {\n String addressee\n command {\n echo \"Hello ${addressee}!\"\n }\n output {\n String salutation = read_string(stdout())\n }\n runtime {\n
+\n }\n}\n\nworkflow my_wf {\n call hello\n output {\n hello.salutation\n }\n}\n",
+ "options": "{\n\n}"
+ },
+ "calls": {
+ "my_wf.hello": [
+ {
+ "executionStatus": "Done",
+ "stdout": "/Users/jdoe/Documents/cromwell-executions/my_wf/cd0fe94a-984e-4a19-ab4c-8f7f07038068/call-hello/execution/stdout",
+ "backendStatus": "Done",
+ "shardIndex": -1,
+ "outputs": {
+ "salutation": "Hello m'Lord!"
+ },
+ "runtimeAttributes": {
+ "continueOnReturnCode": "0",
+ "failOnStderr": "false"
+ },
+ "callCaching": {
+ "allowResultReuse": false,
+ "effectiveCallCachingMode": "CallCachingOff"
+ },
+ "inputs": {
+ "addressee": "m'Lord"
+ },
+ "returnCode": 0,
+ "jobId": "28955",
+ "backend": "Local",
+ "end": "2017-04-19T10:53:25.045-04:00",
+ "stderr": "/Users/jdoe/Documents/cromwell-executions/my_wf/cd0fe94a-984e-4a19-ab4c-8f7f07038068/call-hello/execution/stderr",
+ "callRoot": "/Users/jdoe/Documents/cromwell-executions/my_wf/cd0fe94a-984e-4a19-ab4c-8f7f07038068/call-hello",
+ "attempt": 1,
+ "executionEvents": [
+ {
+ "startTime": "2017-04-19T10:53:23.570-04:00",
+ "description": "PreparingJob",
+ "endTime": "2017-04-19T10:53:23.573-04:00"
+ },
+ {
+ "startTime": "2017-04-19T10:53:23.569-04:00",
+ "description": "Pending",
+ "endTime": "2017-04-19T10:53:23.570-04:00"
+ },
+ {
+ "startTime": "2017-04-19T10:53:25.040-04:00",
+ "description": "UpdatingJobStore",
+ "endTime": "2017-04-19T10:53:25.045-04:00"
+ },
+ {
+ "startTime": "2017-04-19T10:53:23.570-04:00",
+ "description": "RequestingExecutionToken",
+ "endTime": "2017-04-19T10:53:23.570-04:00"
+ },
+ {
+ "startTime": "2017-04-19T10:53:23.573-04:00",
+ "description": "RunningJob",
+ "endTime": "2017-04-19T10:53:25.040-04:00"
+ }
+ ],
+ "start": "2017-04-19T10:53:23.569-04:00"
+ }
+ ]
+ },
+ "outputs": {
+ "my_wf.hello.salutation": "Hello m'Lord!"
+ },
+ "workflowRoot": "/Users/jdoe/Documents/cromwell-executions/my_wf/cd0fe94a-984e-4a19-ab4c-8f7f07038068",
+ "id": "cd0fe94a-984e-4a19-ab4c-8f7f07038068",
+ "inputs": {
+ "my_wf.hello.addressee": "m'Lord"
+ },
+ "submission": "2017-04-19T10:53:19.565-04:00",
+ "status": "Succeeded",
+ "end": "2017-04-19T10:53:25.063-04:00",
+ "start": "2017-04-19T10:53:23.535-04:00"
+}
+```
+
# Getting Started with WDL
For many examples on how to use WDL see [the WDL site](https://github.com/broadinstitute/wdl#getting-started-with-wdl)
@@ -127,7 +127,8 @@ object Dependencies {
"com.google.guava" % "guava" % "22.0",
"com.google.auth" % "google-auth-library-oauth2-http" % "0.7.0",
"com.typesafe.akka" %% "akka-stream-testkit" % akkaV,
- "com.chuusai" %% "shapeless" % "2.3.2"
+ "com.chuusai" %% "shapeless" % "2.3.2",
+ "com.github.scopt" %% "scopt" % "3.6.0"
) ++ baseDependencies ++ googleApiClientDependencies ++
// TODO: We're not using the "F" in slf4j. Core only supports logback, specifically the WorkflowLogger.
slf4jBindingDependencies
@@ -110,7 +110,7 @@ fi
JAR_GCS_PATH=gs://cloud-cromwell-dev/travis-centaur/${CROMWELL_JAR}
gsutil cp target/scala-2.12/cromwell-*.jar "${JAR_GCS_PATH}"
-java -Dconfig.file=./jes.conf -jar target/scala-2.12/cromwell-*.jar run src/bin/travis/resources/centaur.wdl src/bin/travis/resources/centaur.inputs | tee log.txt
+java -Dconfig.file=./jes.conf -jar target/scala-2.12/cromwell-*.jar run src/bin/travis/resources/centaur.wdl --inputs src/bin/travis/resources/centaur.inputs | tee log.txt
EXIT_CODE="${PIPESTATUS[0]}"
# The perl code below is to remove our lovely color highlighting
@@ -0,0 +1,107 @@
+package cromwell
+
+import com.typesafe.config.ConfigFactory
+import cromwell.core.path.{DefaultPathBuilder, Path}
+import scopt.OptionParser
+
+object CommandLineParser extends App {
+
+ sealed trait Command
+ case object Run extends Command
+ case object Server extends Command
+
+ case class CommandLineArguments(command: Option[Command] = None,
+ workflowSource: Option[Path] = None,
+ workflowInputs: Option[Path] = None,
+ workflowOptions: Option[Path] = None,
+ workflowType: Option[String] = Option("WDL"),
+ workflowTypeVersion: Option[String] = Option("v2.0-draft"),
+ workflowLabels: Option[Path] = None,
+ imports: Option[Path] = None,
+ metadataOutput: Option[Path] = None
+ )
+
+ lazy val cromwellVersion = ConfigFactory.load("cromwell-version.conf").getConfig("version").getString("cromwell")
+
+ case class ParserAndCommand(parser: OptionParser[CommandLineArguments], command: Option[Command])
+
+ // cromwell 29
+ // Usage: java -jar /path/to/cromwell.jar [server|run] [options] <args>...
+ //
+ // --help Cromwell - Workflow Execution Engine
+ // --version
+ // Command: server
+ // Starts a web server on port 8000. See the web server documentation for more details about the API endpoints.
+ // Command: run [options] workflow-source
+ // Run the workflow and print out the outputs in JSON format.
+ // workflow-source Workflow source file.
+ // -i, --inputs <value> Workflow inputs file.
+ // -o, --options <value> Workflow options file.
+ // -t, --type <value> Workflow type.
+ // -v, --type-version <value>
+ // Workflow type version.
+ // -l, --labels <value> Workflow labels file.
+ // -p, --imports <value> A directory or zipfile to search for workflow imports.
+ // -m, --metadata-output <value>
+ // An optional directory path to output metadata.
+
+ def buildParser(): scopt.OptionParser[CommandLineArguments] = {
+ new scopt.OptionParser[CommandLineArguments]("java -jar /path/to/cromwell.jar") {
+ head("cromwell", cromwellVersion)
+
+ help("help").text("Cromwell - Workflow Execution Engine")
+
+ version("version")
+
+ cmd("server").action((_, c) => c.copy(command = Option(Server))).text(
+ "Starts a web server on port 8000. See the web server documentation for more details about the API endpoints.")
+
+ cmd("run").
+ action((_, c) => c.copy(command = Option(Run))).
+ text("Run the workflow and print out the outputs in JSON format.").
+ children(
+ arg[String]("workflow-source").text("Workflow source file.").required().
+ action((s, c) => c.copy(workflowSource = Option(DefaultPathBuilder.get(s)))),
+ opt[String]('i', "inputs").text("Workflow inputs file.").
+ action((s, c) =>
+ c.copy(workflowInputs = Option(DefaultPathBuilder.get(s)))),
+ opt[String]('o', "options").text("Workflow options file.").
+ action((s, c) =>
+ c.copy(workflowOptions = Option(DefaultPathBuilder.get(s)))),
+ opt[String]('t', "type").text("Workflow type.").
+ action((s, c) =>
+ c.copy(workflowType = Option(s))),
+ opt[String]('v', "type-version").text("Workflow type version.").
+ action((s, c) =>
+ c.copy(workflowTypeVersion = Option(s))),
+ opt[String]('l', "labels").text("Workflow labels file.").
+ action((s, c) =>
+ c.copy(workflowLabels = Option(DefaultPathBuilder.get(s)))),
+ opt[String]('p', "imports").text(
+ "A directory or zipfile to search for workflow imports.").
+ action((s, c) =>
+ c.copy(imports = Option(DefaultPathBuilder.get(s)))),
+ opt[String]('m', "metadata-output").text(
+ "An optional directory path to output metadata.").
+ action((s, c) =>
+ c.copy(metadataOutput = Option(DefaultPathBuilder.get(s))))
+ )
+ }
+ }
+
+ def runCromwell(args: CommandLineArguments): Unit = {
+ args.command match {
+ case Some(Run) => CromwellEntryPoint.runSingle(args)
+ case Some(Server) => CromwellEntryPoint.runServer()
+ case None => parser.showUsage()
+ }
+ }
+
+ val parser = buildParser()
+
+ val parsedArgs = parser.parse(args, CommandLineArguments())
+ parsedArgs match {
+ case Some(pa) => runCromwell(pa)
+ case None => parser.showUsage()
+ }
+}
Oops, something went wrong.

0 comments on commit 308f087

Please sign in to comment.