Skip to content

Commit

Permalink
Fix reference links, use CWL syntax, use bash/yaml syntax too for pyg…
Browse files Browse the repository at this point in the history
…ments (syntax highlight)
  • Loading branch information
kinow committed Jul 6, 2022
1 parent 5b1af90 commit b329bb0
Show file tree
Hide file tree
Showing 37 changed files with 271 additions and 414 deletions.
5 changes: 1 addition & 4 deletions 01-introduction/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,14 +21,11 @@ software, tools and workflows described using CWL are portable across a variety
of platforms that support the CWL standard.

CWL has roots in "make" and many similar tools that determine order of
execution based on dependencies between tasks. However unlike "make", CWL
execution based on dependencies between tasks. However, unlike "make", CWL
tasks are isolated and you must be explicit about your inputs and outputs. The
benefit of explicitness and isolation are flexibility, portability, and
scalability: tools and workflows described with CWL can transparently leverage
technologies such as Docker and be used with CWL implementations from different
vendors. CWL is well suited for describing large-scale workflows in cluster,
cloud and high performance computing environments where tasks are scheduled in
parallel across many nodes.

```{include} ../_includes/links.md
```
13 changes: 6 additions & 7 deletions 02-1st-example/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ specified to produce text in YAML or JSON. Whatever text editor you use, the ind
*1st-tool.cwl*

```{literalinclude} /_includes/cwl/02-1st-example/1st-tool.cwl
:language: yaml
:language: cwl
```

Next, create a file called `echo-job.yml`, containing the following boxed text, which will describe the input of a run:
Expand Down Expand Up @@ -59,15 +59,15 @@ CWL. The general form is `cwl-runner [tool-or-workflow-description] [input-job-s

What's going on here? Let's break down the contents of `1st-tool.cwl`:

```yaml
```cwl
cwlVersion: v1.0
class: CommandLineTool
```

The `cwlVersion` field indicates the version of the CWL spec used by the document. The `class` field indicates this document
describes a command line tool.

```yaml
```cwl
baseCommand: echo
```

Expand All @@ -93,13 +93,12 @@ on the command line.
In this example,
the `position` field indicates where it should appear on the command line.

```yaml
```cwl
outputs: []
```

This tool has no formal output, so the `outputs` section is an empty list.

[echo]: http://www.linfo.org/echo.html

```{include} ../_includes/links.md
```
[json]: http://json.org/
[yaml]: http://yaml.org/
42 changes: 20 additions & 22 deletions 03-input/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ First, create a file called inp.cwl, containing the following:
*inp.cwl*

```{literalinclude} /_includes/cwl/03-input/inp.cwl
:language: yaml
:language: cwl
```

Create a file called inp-job.yml:
Expand All @@ -48,7 +48,7 @@ object with the fields `class: File` and `path`.

Next, create a whale.txt using [touch] by typing `touch whale.txt` on the command line and then invoke `cwl-runner` with the tool wrapper and the input object on the command line, using the command `cwl-runner inp.cwl inp-job.yml`. The following boxed text describes these two commands and the expected output from the command line:

~~~
```bash
$ touch whale.txt
$ cwl-runner inp.cwl inp-job.yml
[job inp.cwl] /tmp/tmpzrSnfX$ echo \
Expand All @@ -61,70 +61,70 @@ $ cwl-runner inp.cwl inp-job.yml
[job inp.cwl] completed success
{}
Final process status is success
~~~

```{note}
> <p class="rubric">Where did those `/tmp` paths come from?</p>
>
> The CWL reference runner (cwltool) and other runners create temporary
> directories with symbolic ("soft") links to your input files to ensure that
> the tools aren't accidentally accessing files that were not explicitly
> specified
````

```{tip}
## Where did those `/tmp` paths come from?
The CWL reference runner (cwltool) and other runners create temporary
directories with symbolic ("soft") links to your input files to ensure that
the tools aren't accidentally accessing files that were not explicitly
specified
```
The field `inputBinding` is optional and indicates whether and how the
input parameter should be appear on the tool's command line. If
input parameter should appear on the tool's command line. If
`inputBinding` is missing, the parameter does not appear on the command
line. Let's look at each example in detail.
~~~
```cwl
example_flag:
type: boolean
inputBinding:
position: 1
prefix: -f
~~~
```
Boolean types are treated as a flag. If the input parameter
"example_flag" is "true", then `prefix` will be added to the
command line. If false, no flag is added.
~~~
```cwl
example_string:
type: string
inputBinding:
position: 3
prefix: --example-string
~~~
```
String types appear on the command line as literal values. The `prefix`
is optional, if provided, it appears as a separate argument on the
command line before the parameter . In the example above, this is
rendered as `--example-string hello`.
~~~
```cwl
example_int:
type: int
inputBinding:
position: 2
prefix: -i
separate: false
~~~
```
Integer (and floating point) types appear on the command line with
decimal text representation. When the option `separate` is false (the
default value is true), the prefix and value are combined into a single
argument. In the example above, this is rendered as `-i42`.
~~~
```cwl
example_file:
type: File?
inputBinding:
prefix: --file=
separate: false
position: 4
~~~
```
File types appear on the command line as the path to the file. When the
parameter type ends with a question mark `?` it indicates that the
Expand All @@ -147,5 +147,3 @@ is optional. The default position is 0.
The `baseCommand` field will always appear in the final command line before the parameters.
[touch]: http://www.linfo.org/touch.html
```{include} ../_includes/links.md
```
27 changes: 12 additions & 15 deletions 04-output/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ objectives:
- "Learn how to describe and handle outputs from a tool."
keypoints:
- "Outputs are described in the `outputs` section of a CWL description."
- "The field `outputBinding` describes how to to set the value of each
- "The field `outputBinding` describes how to set the value of each
output parameter."
- "Wildcards are allowed in the `glob` field."
---
Expand All @@ -28,18 +28,18 @@ themselves, or come from examining the content of those files.

The following example demonstrates how to return a file that has been extracted from a tar file.

```{note}
> <p class="rubric">Passing mandatory arguments to the `baseCommand`</p>
>
> In previous examples, the `baseCommand` was just a string, with any arguments passed as CWL inputs.
> Instead of a single string we can use an _array of strings_. The first element is the command to run, and
> any subsequent elements are mandatory command line arguments
```{tip}
Passing mandatory arguments to the `baseCommand`
In previous examples, the `baseCommand` was just a string, with any arguments passed as CWL inputs.
Instead of a single string we can use an _array of strings_. The first element is the command to run, and
any subsequent elements are mandatory command line arguments
```

*tar.cwl*

```{literalinclude} /_includes/cwl/04-output/tar.cwl
:language: yaml
:language: cwl
```

*tar-job.yml*
Expand All @@ -51,7 +51,7 @@ The following example demonstrates how to return a file that has been extracted
Next, create a tar file for the example and invoke `cwl-runner` with the tool
wrapper and the input object on the command line:

~~~
```bash
$ touch hello.txt && tar -cvf hello.tar hello.txt
$ cwl-runner tar.cwl tar-job.yml
[job tar.cwl] /tmp/tmpqOeawQ$ tar \
Expand All @@ -69,21 +69,18 @@ $ cwl-runner tar.cwl tar-job.yml
}
}
Final process status is success
~~~
```

The field `outputBinding` describes how to set the value of each
output parameter.

~~~
```cwl
outputs:
example_out:
type: File
outputBinding:
glob: hello.txt
~~~
```

The `glob` field consists of the name of a file in the output directory.
If you don't know name of the file in advance, you can use a wildcard pattern like `glob: '*.txt'`.

```{include} ../_includes/links.md
```
7 changes: 2 additions & 5 deletions 05-stdout/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ stdout` on the corresponding output parameter.
*stdout.cwl*

```{literalinclude} /_includes/cwl/05-stdout/stdout.cwl
:language: yaml
:language: cwl
```

*echo-job.yml*
Expand All @@ -31,7 +31,7 @@ stdout` on the corresponding output parameter.
Now invoke `cwl-runner` providing the tool wrapper and the input object
on the command line:

~~~
```bash
$ cwl-runner stdout.cwl echo-job.yml
[job stdout.cwl] /tmp/tmpE0gTz7$ echo \
'Hello world!' > /tmp/tmpE0gTz7/output.txt
Expand All @@ -47,7 +47,4 @@ $ cwl-runner stdout.cwl echo-job.yml
}
}
Final process status is success
~~~
```{include} ../_includes/links.md
```
Loading

0 comments on commit b329bb0

Please sign in to comment.