-
Notifications
You must be signed in to change notification settings - Fork 55
Miscellaneous
- I cannot find my issue in this document and want more information from Cromwell, MySQL, or TES Docker container logs.
- I am running a large number of workflows and MySQL storage disk is full
- How can I run CWL files on Cromwell on Azure?
The host VM is running multiple Docker containers that enable Cromwell on Azure - mysql, broadinstitute/cromwell, cromwellonazure/tes, cromwellonazure/triggerservice. On rare occasions, you may want to debug and diagnose issues with the Docker containers. After logging in to the host VM, run:
sudo docker ps
This command will list the names of all the Docker containers currently running. To get logs for a particular container, run:
sudo docker logs 'containerName'
To ensure that no data is corrupted for MySQL backed storage for Cromwell, Cromwell on Azure mounts MySQL files on to an Azure Managed Data Disk of size 32G. In case there is a need to increase the size of this data disk, follow instructions here.
Running workflows written in the Common Workflow Language(CWL) format is possible with a few modifications to your workflow submission.
For CWL workflows, all CWL resource keywords are supported, plus preemptible (not in CWL spec). preemptible defaults to true (set in Cromwell configuration file), so use preemptible only if setting it to false (run on dedicated machine). TES keywords are also supported in CWL workflows, but we advise users to use the CWL ones.
CWL keywords: (CWL workflows only)
coresMin: number
ramMin: size in MB
tmpdirMin: size in MB - Cromwell on Azure version 2.0 and above only
outdirMin: size in MB - Cromwell on Azure version 2.0 and above only
(the final disk size is the sum of tmpDir and outDir values)
TES keywords: (both CWL and WDL workflows)
preemptible: true|false
Cromwell on Azure version 1.x known issue for CWL files: Cannot request specific HDD size Unfortunately, this is actually a bug in how Cromwell currently parses the CWL files and thus must be addressed in the Cromwell source code directly.
The current workaround for this is to increase the number of vCPUs or memory requested for a task, which will indirectly increase the amount of working disk space available. However, because this may cause inconsistent performance, we advise that if you are running a task that might consume a large amount of local scratch space, consider converting your workflow to the WDL format instead.
To search, expand the Pages section above.