0.19

@Horneth Horneth released this Apr 1, 2016 · 1034 commits to develop since this release

  • Workflow options may now contain a choice of backend specific for that workflow:
{
  "backend": "JES"
}
  • To support the above change, the configuration file has changed how backends are specified. You must replace backend.backend with backend.defaultBackend.
    In addition the option backend.backendsAllowed must be specified (and should include the default), for example:
backend {
  defaultBackend = "local"
  backendsAllowed = [ "local", "JES", "SGE" ]
  ...
  • New runtime option for JES: bootDiskSizeGb. Allows specification of a boot disk size (as an Integer number of GB) that can be increased to boot a larger docker image.
  • Workflow options now allows you to specify a workflowFailureMode to control workflow behavior after a call has failed, for example:{ "workflowFailureMode": "..." }. The options are:
    • ContinueWhilePossible - continues to start and process calls in the workflow, as long as they did not depend on the failing call
    • NoNewCalls - no new calls are started but existing calls are allowed to finish
    • The default is NoNewCalls but this can be changed using the workflow-options.workflow-failure-mode configuration option.
  • Bug fix: Tasks that changed directory would fail on JES because their return code file was written to the new directory instead of an absolute path
  • Bug fix: Using write_* functions in a Task's command (e.g. ./my_script --file=${write_file(my_array)}) will now work with JES
  • Changing format of the 'disks' runtime attribute slightly to allow for mounting disks at specific mountpoints
task disk_test {
  command { ... }
  runtime {
    disks: "local-disk 20 SSD, /mnt/mnt1 200 HDD"
  }
}
  • Metadata now contains a list of failures for calls and workflows. This is an optional element of both call and workflow and is shaped thus:
"failures": [
  {
    "failure": "The failure message",
    "timestamp": "2016-02-25T10:49:02.066-05:00"
  }
]

Downloads

0.18

@scottfrazer scottfrazer released this Mar 3, 2016 · 1164 commits to develop since this release

  • The deprecated parse, validate, inputs and highlight functionality from the command line tool has been removed in favor of wdltool (https://github.com/broadinstitute/wdltool)
  • Workflow options can now include a defaultRuntimeOptions section, so that the same runtime attribute is not needed in every single WDL task. E.g.:
{
  "defaultRuntimeOptions": {
    "docker": "ubuntu:latest"
  }
}
  • Changed the JES runtime attributes defaultDisks and defaultZones to be simply disks and zones respectively.
  • Liquibase scripts now run automatically. Non-persistent, in-memory databases are not affected. However Cromwell will
    not start if it detects evidence of manually run liquibase migrations in a persistent database. Instead, before Cromwell
    will start cleanly, the database should backed up, and then this SQL should be manually executed:
update DATABASECHANGELOG
  set MD5SUM = null,
    FILENAME = substr(FILENAME, instr(FILENAME, "src/main/migrations/") + length("src/main/migrations/"))
  where FILENAME like '%src/main/migrations/%'
  • Added Preemptible VMs support for JES. This has impacts on the API Endpoint responses as a Call/Shard can now be attempted multiple times. Each attempt will have its own entry.
  • Added custom thread pool to workaround Slick deadlocks. The thread pool
    size defaults to the Slick configuration value db.numThreads, but may be increased up to Slick's
    db.maxConnections, via a new property actionThreadPoolSize.
  • Added support for size WDL standard library function.
  • Allow for runtime attribute values to be interpreted as full expressions. For example:
task example {
  String ubuntu_tag
  command { ... }
  runtime {
    docker: "ubuntu:" + ubuntu_tag
  }
}
  • Add runtime attributes in Call metadata :
{
  "workflowName": "hello",
  "calls": {
    "hello.hello": [
      {
        ...,
        "runtimeAttributes": {
                  "preemptible": "0",
                  "failOnStderr": "false",
                  "disks": "local-disk 10 SSD",
                  "continueOnReturnCode": "0",
                  "docker": "ubuntu:latest",
                  "cpu": "1",
                  "zones": "us-central1-a",
                  "memory": "2GB"
                },
        ...
      }
    ]
  }
}
  • Added "preemptible" field in Call metadata. This only appears if the backend is JES.
{
  "workflowName": "hello",
  "calls": {
    "hello.hello": [
      {
        ...,
        "preemptible": "true"
        ...
      }
    ]
  }
}

Downloads

0.17

@geoffjentry geoffjentry released this Feb 2, 2016 · 1261 commits to develop since this release

  • Validate RuntimeAttributes before starting a call
  • Prevent infinite retries
  • Persist disallowed return codes in DB
  • Add deprecation notices to WDL specific command line functionality
  • Make Call backoff polling backend specific
  • Aborts jobs on shutdown.
  • Numerous bug fixes and stability improvements

Downloads

0.16

@scottfrazer scottfrazer released this Jan 11, 2016 · 1316 commits to develop since this release

  • Bug fix: call caching will not calculate MD5s if turned off
  • New API endpoint for generating HTML timing diagrams of a workflow
  • Move the Scala bindings for WDL into a new project wdl4s

Downloads

0.15

@scottfrazer scottfrazer released this Dec 18, 2015 · 1377 commits to develop since this release

  • Call caching allows Cromwell to use results from prior invocations if it detected that it's run a call in the past
  • Various bug fixes, stability improvements
  • Improved REST API

Downloads

0.14

@scottfrazer scottfrazer released this Nov 3, 2015 · 1470 commits to develop since this release

  • Improved logging
  • Improved concurrency handing for high workflow loads
  • Bug fixes
  • Improved stability

Downloads

0.13

@scottfrazer scottfrazer released this Oct 16, 2015 · 1483 commits to master since this release

  • Stability and bug fixes

  • implemented read_float(), read_boolean(), read_object(), and write_object() WDL functions

  • Allow WDL functions at the workflow level:

    workflow wf {
      Array[Int] my_ints = read_lines("some_file")
    }
    

Downloads

0.12

@scottfrazer scottfrazer released this Oct 5, 2015 · 1524 commits to master since this release

Merge branch 'develop'

Downloads