- Workflow options may now contain a choice of backend specific for that workflow:
{
"backend": "JES"
}
- To support the above change, the configuration file has changed how backends are specified. You must replace
backend.backendwithbackend.defaultBackend.
In addition the optionbackend.backendsAllowedmust be specified (and should include the default), for example:
backend {
defaultBackend = "local"
backendsAllowed = [ "local", "JES", "SGE" ]
...
- New runtime option for JES:
bootDiskSizeGb. Allows specification of a boot disk size (as an Integer number of GB) that can be increased to boot a larger docker image. - Workflow options now allows you to specify a
workflowFailureModeto control workflow behavior after a call has failed, for example:{ "workflowFailureMode": "..." }. The options are:ContinueWhilePossible- continues to start and process calls in the workflow, as long as they did not depend on the failing callNoNewCalls- no new calls are started but existing calls are allowed to finish- The default is
NoNewCallsbut this can be changed using theworkflow-options.workflow-failure-modeconfiguration option.
- Bug fix: Tasks that changed directory would fail on JES because their return code file was written to the new directory instead of an absolute path
- Bug fix: Using
write_*functions in a Task's command (e.g../my_script --file=${write_file(my_array)}) will now work with JES - Changing format of the 'disks' runtime attribute slightly to allow for mounting disks at specific mountpoints
task disk_test {
command { ... }
runtime {
disks: "local-disk 20 SSD, /mnt/mnt1 200 HDD"
}
}
- Metadata now contains a list of failures for calls and workflows. This is an optional element of both
callandworkflowand is shaped thus:
"failures": [
{
"failure": "The failure message",
"timestamp": "2016-02-25T10:49:02.066-05:00"
}
]
- Added workflow options to copy the call logs and/or the workflow logs to a
call_logs_diror aworkflow_log_dir,
respectively. - The system properties
LOG_MODEandLOG_LEVELused by Logback may now be specified as environment variables. - Implemented
write_tsvfunction - Support
subfunction from the WDL Standard Library. See https://github.com/broadinstitute/wdl/blob/subFunction/SPEC.md#string-substring-string-string
Downloads
- The deprecated parse, validate, inputs and highlight functionality from the command line tool has been removed in favor of wdltool (https://github.com/broadinstitute/wdltool)
- Workflow options can now include a
defaultRuntimeOptionssection, so that the same runtime attribute is not needed in every single WDL task. E.g.:
{
"defaultRuntimeOptions": {
"docker": "ubuntu:latest"
}
}
- Changed the JES runtime attributes
defaultDisksanddefaultZonesto be simplydisksandzonesrespectively. - Liquibase scripts now run automatically. Non-persistent, in-memory databases are not affected. However Cromwell will
not start if it detects evidence of manually run liquibase migrations in a persistent database. Instead, before Cromwell
will start cleanly, the database should backed up, and then this SQL should be manually executed:
update DATABASECHANGELOG
set MD5SUM = null,
FILENAME = substr(FILENAME, instr(FILENAME, "src/main/migrations/") + length("src/main/migrations/"))
where FILENAME like '%src/main/migrations/%'- Added Preemptible VMs support for JES. This has impacts on the API Endpoint responses as a Call/Shard can now be attempted multiple times. Each attempt will have its own entry.
- Added custom thread pool to workaround Slick deadlocks. The thread pool
size defaults to the Slick configuration valuedb.numThreads, but may be increased up to Slick's
db.maxConnections, via a new propertyactionThreadPoolSize. - Added support for size WDL standard library function.
- Allow for runtime attribute values to be interpreted as full expressions. For example:
task example {
String ubuntu_tag
command { ... }
runtime {
docker: "ubuntu:" + ubuntu_tag
}
}
- Add runtime attributes in Call metadata :
{
"workflowName": "hello",
"calls": {
"hello.hello": [
{
...,
"runtimeAttributes": {
"preemptible": "0",
"failOnStderr": "false",
"disks": "local-disk 10 SSD",
"continueOnReturnCode": "0",
"docker": "ubuntu:latest",
"cpu": "1",
"zones": "us-central1-a",
"memory": "2GB"
},
...
}
]
}
}
- Added "preemptible" field in Call metadata. This only appears if the backend is JES.
{
"workflowName": "hello",
"calls": {
"hello.hello": [
{
...,
"preemptible": "true"
...
}
]
}
}
Downloads
- Validate RuntimeAttributes before starting a call
- Prevent infinite retries
- Persist disallowed return codes in DB
- Add deprecation notices to WDL specific command line functionality
- Make Call backoff polling backend specific
- Aborts jobs on shutdown.
- Numerous bug fixes and stability improvements
Downloads
- Bug fix: call caching will not calculate MD5s if turned off
- New API endpoint for generating HTML timing diagrams of a workflow
- Move the Scala bindings for WDL into a new project wdl4s
Downloads
- Call caching allows Cromwell to use results from prior invocations if it detected that it's run a
callin the past - Various bug fixes, stability improvements
- Improved REST API
Downloads
- Improved logging
- Improved concurrency handing for high workflow loads
- Bug fixes
- Improved stability
Downloads
-
Stability and bug fixes
-
implemented
read_float(),read_boolean(),read_object(), andwrite_object()WDL functions -
Allow WDL functions at the workflow level:
workflow wf { Array[Int] my_ints = read_lines("some_file") }
Downloads
Merge branch 'develop'
Downloads
PreviousNext