This section present some commonly encountered use-cases and basic solutions regarding ADES
/EMS
operation or more specifically related to CWL
specification.
In most situations, the CommandLineTool
process will need to run a Docker
image. Doing so is as simple as adding the DockerRequirement
(reference_) as follows to the Application Package
definition:
{
"cwlVersion": "v1.0",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
}
},
"inputs": ["<...>"],
"outputs": ["<...>"],
}
Note
The Docker
image reference must be publicly accessible to allow CWL
to pull it. Alternatively, a private Docker
reference can be used if the image is locally available. The process will fail to execute if it cannot resolve the reference.
The Application Package
can be provided during process deployment. Please refer to below references for more details.
- Supported
Application Package
definitions inproc_wps_rest
deployment. Deploy <proc_op_deploy>
request.
Some processes expect their inputs to be writable (e.g.: ZIP files). When running an Application Package
based on a Docker
image, Weaver mounts the input files as volumes in read-only mode for security reasons. This causes these processes to immediately fail as the running user cannot override nor write temporary files in the same directory (where the volume was mounted to), as it is marked with read permissions.
To resolve this issue, the application developer should add the InitialWorkDirRequirement
(reference_, example_) to his CWL package definition. This tells CWL
to stage the files into the Docker
container into the running directory where the user will be allowed to generate outputs, and therefore, also allow edition of the inputs or generation of temporary files as when unpacking a compressed file.
As example, the CWL
definition could be similar to the following:
{
"cwlVersion": "v1.0",
"class": "CommandLineTool",
"requirements": {
"DockerRequirement": {
"dockerPull": "<docker-url>"
},
"InitialWorkDirRequirement": {
"listing": [{
"entry": "$(inputs.input_file)",
"writable": true
}
]
}
},
"arguments": ["$(runtime.outdir)"],
"inputs": {
"input_file": {
"type": "File"
}
}
Note that $(inputs.input_file)
within InitialWorkDirRequirement
tells which input to resolve for staging using the "writable": True
parameter. All files listed there will be mounted with write permissions into working runtime directory of the executed Docker
container.
CWL Workflow
Output File Format
There exist multiple sources, but official ones provided below have a create amount of examples and are being continuously improved by the developers (including being updated according to changes).
- _
- _
- _
- See also below.
Many examples are reported in detail in the _ chapter.
Yes!
Take a look at the cli
chapter for available utilities to interact with Weaver using shell or Python scripting.