Skip to content

Commit

Permalink
WIP Migrate sections to use runcmd where possible
Browse files Browse the repository at this point in the history
  • Loading branch information
kinow committed Sep 7, 2022
1 parent 587c2ae commit 9674a51
Show file tree
Hide file tree
Showing 10 changed files with 40 additions and 122 deletions.
4 changes: 2 additions & 2 deletions src/_includes/cwl/conformance-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,8 @@

# Section 13
- doc: Test for section 13
job: 13-expressions/empty.yml
tool: 13-expressions/expression.cwl
job: expressions/empty.yml
tool: expressions/expression.cwl
output:
example_out:
class: File
Expand Down
File renamed without changes.
1 change: 1 addition & 0 deletions src/_includes/cwl/expressions/output.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
-A 2 -B baz -C 10 9 8 7 6 5 4 3 2 1
3 changes: 3 additions & 0 deletions src/_includes/cwl/hello_world-job.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"message": "こんにちは世界"
}
File renamed without changes.
File renamed without changes.
71 changes: 22 additions & 49 deletions src/introduction/prerequisites.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ for other ways to install `cwltool` with `apt` and `conda`.

Let's use a simple workflow `true.cwl` with `cwltool`.

```{literalinclude} /_includes/cwl/simplest_cwl.cwl
```{literalinclude} /_includes/cwl/true.cwl
:language: cwl
:caption: "`true.cwl`"
:name: true.cwl
Expand All @@ -75,29 +75,16 @@ to pass `--validate` to the `cwltool` command:
% TODO: maybe figure out a way to avoid /home/kinow/ etc. in the documentation
% to avoid multiple user-names/directory-locations varying in the docs.

```{code-block} console
```{runcmd} cwltool --validate true.cwl
:name: validating-truecwl-with-cwltool
:caption: Validating `true.cwl` with `cwltool`.
$ cwltool --validate true.cwl
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220406080846
INFO Resolved 'true.cwl' to 'file:///tmp/true.cwl'
true.cwl is valid CWL.
```

You can run the CWL workflow now that you know it is valid:

```{code-block} console
```{runcmd} cwltool true.cwl
:name: running-true.cwl-with-cwltool
:caption: Running `true.cwl` with `cwltool`.
$ cwltool true.cwl
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220406080846
INFO Resolved 'true.cwl' to 'file:///tmp/true.cwl'
INFO [job true.cwl] /tmp/f8xlh1pl$ true
INFO [job true.cwl] completed success
{}
INFO Final process status is success
```

### cwl-runner Python module
Expand Down Expand Up @@ -126,56 +113,42 @@ Now you can validate and run your workflow with `cwl-runner` executable,
which will invoke `cwltool`. You should have the same results and output
as in the previous section.

```{code-block} console
```{runcmd} cwl-runner --validate true.cwl
:name: validating-true.cwl-with-cwl-runner
:caption: Validating `true.cwl` with `cwl-runner`.
$ cwl-runner --validate true.cwl
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwl-runner 3.1.20220406080846
INFO Resolved 'true.cwl' to 'file:///tmp/true.cwl'
true.cwl is valid CWL.
```

```{code-block} console
```{runcmd} cwl-runner true.cwl
:name: running-true.cwl-with-cwl-runner
:caption: Running `true.cwl` with `cwl-runner`.
$ cwl-runner true.cwl
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwl-runner 3.1.20220406080846
INFO Resolved 'true.cwl' to 'file:///tmp/true.cwl'
INFO [job true.cwl] /tmp/7a2gf1nh$ true
INFO [job true.cwl] completed success
{}
INFO Final process status is success
```

Another way to execute `cwl-runner` is invoking the file directly. For that,
the first thing you need is to modify the `true.cwl` workflow and include
a special first line, a *shebang*:
the first thing you need to copy `true.cwl` workflow into a new file
`true_shebang.cwl` and include a special first line, a *shebang*:

```{literalinclude} /_includes/cwl/simplest_cwl_shebang.cwl
```{literalinclude} /_includes/cwl/true_shebang.cwl
:language: cwl
:name: cwltool-with-a-shebang
:caption: "`cwltool` with a shebang"
:name: "true_shebang.cwl"
:caption: "`true_shebang.cwl`"
```

Now, after you make the file `true.cwl` executable with `chmod u+x`,
you can execute it directly in the command-line and the program
specified in the shebang (`cwl-runner`) will be used to execute the
rest of the file.
Now you can make the file `true_shebang.cwl` executable with `chmod u+x`.

```{code-block} console
:name: making-true.cwl-executable-and-running-it
:caption: Making `true.cwl` executable and running it.
:name: making-true.cwl-executable
:caption: Making `true.cwl` executable.
$ chmod u+x true.cwl
$ ./true.cwl
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwl-runner 3.1.20220406080846
INFO Resolved './true.cwl' to 'file:///tmp/true.cwl'
INFO [job true.cwl] /tmp/jz7ups99$ true
INFO [job true.cwl] completed success
{}
INFO Final process status is success
```

And finally you can execute it directly in the command-line and the program
specified in the shebang (`cwl-runner`) will be used to execute the
rest of the file.

```{runcmd} ./true_shebang.cwl
:name: running-true_shebang.cwl-with-a-shebang
:caption: Running `true_shebang.cwl` with a shebang.
```

```{note}
Expand Down
42 changes: 5 additions & 37 deletions src/introduction/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,66 +55,34 @@ The usage of the `cwltool` command-line executable is basically
`cwltool [OPTIONS] <CWL_DOCUMENT> [INPUTS_OBJECT]`. You can run the
`hello_world.cwl` workflow without specifying any option:

```{code-block} console
```{runcmd} cwltool hello_world.cwl
:name: running-hello_world.cwl-with-cwltool
:caption: Running `hello_world.cwl` with `cwltool`.
$ cwltool hello_world.cwl
INFO /tmp/venv/bin/cwltool 3.1.20220628170238
INFO Resolved 'hello_world.cwl' to 'file:///tmp/hello_world.cwl'
INFO [job hello_world.cwl] /tmp/yn0e8xu6$ echo \
'Hello World'
Hello World
INFO [job hello_world.cwl] completed success
{}
INFO Final process status is success
```

Or you can override the default value of the input parameter `message`, similar
to how you would change the argument of the `echo` base command:

```{code-block} console
```{runcmd} cwltool hello_world.cwl --message="Hola mundo"
:name: running-hello_world.cwl-with-cwltool-passing-an-input-parameter
:caption: Running `hello_world.cwl` with `cwltool` passing an input parameter.
$ cwltool hello_world.cwl --message="Hola mundo"
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220406080846
INFO Resolved '/tmp/hello_world.cwl' to 'file:///tmp/hello_world.cwl'
INFO [job hello_world.cwl] /tmp/ua5vt9hl$ echo \
'Hola mundo'
Hola mundo
INFO [job hello_world.cwl] completed success
{}
INFO Final process status is success
```

Another way of passing values to your workflow input parameters is via an
*Inputs Object*. This is a file containing the input fields with the
corresponding values. This file can be written in JSON or YAML. For example:

```{code-block} json
```{literalinclude} /_includes/cwl/hello_world-job.json
:language: json
:name: hello_world-job.json
:caption: "`hello_world-job.json`"
{
"message": "こんにちは世界"
}
```
<p class="text-center text-muted mt-n2">hello_world-job.json</p>

You can use this Inputs Object file now to execute the “Hello World” workflow:

```{code-block} console
```{runcmd} cwltool hello_world.cwl hello_world-job.json
:name: passing-an-inputs-object-file-to-cwltool
:caption: Passing an Inputs Object file to `cwltool`.
$ cwltool hello_world.cwl hello_world-job.json
INFO /home/kinow/Development/python/workspace/user_guide/venv/bin/cwltool 3.1.20220406080846
INFO Resolved '/tmp/hello_world.cwl' to 'file:///tmp/hello_world.cwl'
INFO [job hello_world.cwl] /tmp/c5uchknw$ echo \
こんにちは世界
こんにちは世界
INFO [job hello_world.cwl] completed success
{}
INFO Final process status is success
```

```{note}
Expand Down
41 changes: 7 additions & 34 deletions src/topics/expressions.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ When manipulating file names, extensions, paths etc, consider whether one of the
etc, could be used instead.
See the [list of best practices](best-practices.md).
```{literalinclude} /_includes/cwl/13-expressions/expression.cwl
```{literalinclude} /_includes/cwl/expressions/expression.cwl
:language: cwl
:caption: "`expression.cwl`"
:name: expression.cwl
Expand All @@ -21,7 +21,7 @@ See the [list of best practices](best-practices.md).
As this tool does not require any `inputs` we can run it with an (almost) empty
job file:

```{literalinclude} /_includes/cwl/13-expressions/empty.yml
```{literalinclude} /_includes/cwl/expressions/empty.yml
:language: yaml
:caption: "`empty.yml`"
:name: empty.cwl
Expand All @@ -33,41 +33,14 @@ represented simply by a set of empty brackets.

We can then run `expression.cwl`:

```{code-block} console
```{runcmd} cwl-runner expression.cwl empty.yml
:name: running-expression.cwl
:caption: Running `expression.cwl`
:working-directory: src/_includes/cwl/expressions/
```

$ cwl-runner expression.cwl empty.yml
[job expression.cwl] /home/example$ echo \
-A \
2 \
-B \
baz \
-C \
10 \
9 \
8 \
7 \
6 \
5 \
4 \
3 \
2 \
1 > /home/example/output.txt
[job expression.cwl] completed success
{
"example_out": {
"location": "file:///home/example/output.txt",
"basename": "output.txt",
"class": "File",
"checksum": "sha1$a739a6ff72d660d32111265e508ed2fc91f01a7c",
"size": 36,
"path": "/home/example/output.txt"
}
}
Final process status is success
$ cat output.txt
-A 2 -B baz -C 10 9 8 7 6 5 4 3 2 1
```{runcmd} cat output.txt
:working-directory: src/_includes/cwl/expressions/
```

Note that requirements can be provided with the map syntax, as in the example above:
Expand Down

0 comments on commit 9674a51

Please sign in to comment.