dx_instance_type runtime keywork not recongizned in compiling WDL v1.1 and development version

[dolevrahat]

Hello

I am attempting to compile the WDL workflow attached below (which is a modification of https://github.com/dnanexus/dxCompiler/blob/90d136ee8736dd5af5f20edd2d745896a0decb4e/test/cromwell/sub_workflow_interactions_scatter/sub_workflow_interactions_import.wdl)

When I declare the workflow to be WDL version 1.0 in the wdl file header, the workflow compiles successfully.
However, when I attempt to compile it as WDL version 1.1 or the development version, compilation fails if I specify dx_instance_type in any of the task runtime sections:
Using version 1.1 I get the following error:

[error] Error translating sub_workflow_interactions_import.wdl to IR
java.lang.StackOverflowError
	at scala.runtime.Statics.anyHash(Statics.java:127)
	at scala.collection.immutable.HashMap.contains(HashMap.scala:124)
	at wdlTools.eval.Runtime.contains(Runtime.scala:32)
	at wdlTools.eval.Runtime.contains(Runtime.scala:35)

Using development I get:

[error] Error creating translator for sub_workflow_interactions_import.wdl
wdlTools.syntax.SyntaxException: error parsing document /home/hadassah/Documents/Dolev/pipeline_automation/sandbox/sub_workflow_interactions_import.wdl
	at wdlTools.syntax.v2.ParseAll.parseDocument(ParseAll.scala:402)
	at dx.core.languages.wdl.WdlUtils$.parseSource(WdlUtils.scala:82)
	at dx.core.languages.wdl.WdlUtils$.parseAndCheckSource(WdlUtils.scala:97)
	at dx.core.languages.wdl.VersionSupport$.fromSource(VersionSupport.scala:128)
	at dx.core.languages.wdl.VersionSupport$.fromSourceFile(VersionSupport.scala:141)
	at dx.translator.wdl.WdlTranslatorFactory.liftedTree1$1(WdlTranslator.scala:223)
	at dx.translator.wdl.WdlTranslatorFactory.create(WdlTranslator.scala:222)
	at dx.translator.TranslatorFactory$$anonfun$createTranslator$6.applyOrElse(Translator.scala:97)
	at dx.translator.TranslatorFactory$$anonfun$createTranslator$6.applyOrElse(Translator.scala:97)
	at scala.collection.IterableOnceOps.collectFirst(IterableOnce.scala:1086)
	at scala.collection.IterableOnceOps.collectFirst$(IterableOnce.scala:1078)
	at scala.collection.AbstractIterable.collectFirst(Iterable.scala:920)
	at dx.translator.TranslatorFactory$.createTranslator(Translator.scala:97)
	at dxCompiler.Main$.compile(Main.scala:402)
	at dxCompiler.Main$.dispatchCommand(Main.scala:778)
	at dxCompiler.Main$.main(Main.scala:884)
	at dxCompiler.MainApp$.delayedEndpoint$dxCompiler$MainApp$1(Main.scala:889)
	at dxCompiler.MainApp$delayedInit$body.apply(Main.scala:888)
	at scala.Function0.apply$mcV$sp(Function0.scala:39)
	at scala.Function0.apply$mcV$sp$(Function0.scala:39)
	at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:17)
	at scala.App.$anonfun$main$1(App.scala:73)
	at scala.App.$anonfun$main$1$adapted(App.scala:73)
	at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:553)
	at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:551)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:920)
	at scala.App.main(App.scala:73)
	at scala.App.main$(App.scala:71)
	at dxCompiler.MainApp$.main(Main.scala:888)
	at dxCompiler.MainApp.main(Main.scala)
Caused by: wdlTools.syntax.SyntaxException: invalid runtime keyword dx_instance_type at 11:8-11:43 in /home/hadassah/Documents/Dolev/pipeline_automation/sandbox/sub_workflow_interactions_import.wdl
	at wdlTools.syntax.v2.ParseTop.visitTask_runtime_kv(ParseTop.scala:859)
	at wdlTools.syntax.v2.ParseTop.$anonfun$visitTask_runtime$1(ParseTop.scala:872)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:575)
	at scala.collection.mutable.Growable.addAll(Growable.scala:62)
	at scala.collection.mutable.Growable.addAll$(Growable.scala:59)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:147)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:40)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:265)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:253)
	at scala.collection.IterableOps.map(Iterable.scala:671)
	at scala.collection.IterableOps.map$(Iterable.scala:671)
	at scala.collection.AbstractIterable.map(Iterable.scala:920)
	at wdlTools.syntax.v2.ParseTop.visitTask_runtime(ParseTop.scala:872)
	at wdlTools.syntax.v2.ParseTop.visitTask_runtime(ParseTop.scala:14)
	at org.openwdl.wdl.parser.v2.WdlV2Parser$Task_runtimeContext.accept(WdlV2Parser.java:4276)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at org.openwdl.wdl.parser.v2.WdlV2ParserBaseVisitor.visitTask_element(WdlV2ParserBaseVisitor.java:566)
	at wdlTools.syntax.v2.ParseTop.$anonfun$visitTask$1(ParseTop.scala:989)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:575)
	at scala.collection.mutable.Growable.addAll(Growable.scala:62)
	at scala.collection.mutable.Growable.addAll$(Growable.scala:59)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:147)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:40)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:265)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:253)
	at scala.collection.IterableOps.map(Iterable.scala:671)
	at scala.collection.IterableOps.map$(Iterable.scala:671)
	at scala.collection.AbstractIterable.map(Iterable.scala:920)
	at wdlTools.syntax.v2.ParseTop.visitTask(ParseTop.scala:989)
	at wdlTools.syntax.v2.ParseTop.visitTask(ParseTop.scala:14)
	at org.openwdl.wdl.parser.v2.WdlV2Parser$TaskContext.accept(WdlV2Parser.java:4949)
	at org.antlr.v4.runtime.tree.AbstractParseTreeVisitor.visitChildren(AbstractParseTreeVisitor.java:46)
	at wdlTools.syntax.v2.ParseTop.visitDocument_element(ParseTop.scala:1272)
	at wdlTools.syntax.v2.ParseTop.$anonfun$visitDocument$1(ParseTop.scala:1298)
	at scala.collection.Iterator$$anon$9.next(Iterator.scala:575)
	at scala.collection.mutable.Growable.addAll(Growable.scala:62)
	at scala.collection.mutable.Growable.addAll$(Growable.scala:59)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:147)
	at scala.collection.mutable.ArrayBuffer.addAll(ArrayBuffer.scala:40)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:265)
	at scala.collection.mutable.ArrayBuffer$.from(ArrayBuffer.scala:253)
	at scala.collection.IterableOps.map(Iterable.scala:671)
	at scala.collection.IterableOps.map$(Iterable.scala:671)
	at scala.collection.AbstractIterable.map(Iterable.scala:920)
	at wdlTools.syntax.v2.ParseTop.visitDocument(ParseTop.scala:1298)
	at wdlTools.syntax.v2.ParseTop.$anonfun$parseDocument$1(ParseTop.scala:1322)
	at wdlTools.syntax.Antlr4Util$Grammar.visitDocument(Antlr4Util.scala:255)
	at wdlTools.syntax.v2.ParseTop.parseDocument(ParseTop.scala:1322)
	at wdlTools.syntax.v2.ParseAll.parseDocument(ParseAll.scala:399)
	... 29 more

In both cases, compilation is successful if I omit the dx_instance_type setting.
I have attempted this both with dxCompiler version 2.5.0 and version 2.4.8, both leading to the same result (stack traces in error message above are from v2.5.0).

Any advice on this will be highly appreciated.

Workflow

version 1.1
task countTo {
    input{
    Int value
    }
    command {
        seq 0 1 ${value}
    }
    runtime {
        dx_instance_type: "mem1_ssd1_v2_x2"
      }
    output {
        File range = stdout()
    }
}

task filterEvens {
    input{
    File numbers
    }
    command {
        grep '[02468]$' ${numbers} > evens
        grep '[13579]$' ${numbers} > odds
    }
    runtime {
        dx_instance_type: "mem1_ssd1_v2_x2"
      }


    output {
     Map[String,File] counter_output = {"odds":"odds","evens":"evens"}
    }
}

workflow countEvens {
    input{
    Int max
    String sample
    }
    call countTo { input: value = max }
    call filterEvens { input: numbers = countTo.range }
    output {
        Pair[String,Map[String,File]] counter_wf_output = (sample,filterEvens.counter_output)
    }
}

[jdidion]

Thanks, I can reproduce this bug. It will be fixed in the next release.

[sclan]

For the development version, "dx_instance_type" needs to be in the "hints" block instead of the "runtime" block. And the docker entry in the runtime block is renamed to "container" according to the specifications:
https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md

[commandlinegirl]

This has been fixed. I tested it with the latest version of dxCompiler (2.10) and it doesn't produce any errors. Thank you for reporting!