Empty tasks

[nh13]

Just like we have EmptyDependable, I would like an EmptyTask, which does nothing (doesn't need to be run). I can then a task that conditionally runs, without having to implement a silly SimpleInJvmTask that does nothing.

[tfenne]

FYI You can already do this today. The dependency operator ==> accepts Option[Dependable] and does the right thing if you give it a None. So you can do things like:

val pre = if (doPre) Some(Pre(...)) else None
...
root => pre => thing ==> post

[nh13]

Agreed, so do pipes, but these do not always solve my use case. When scatter gathering, I basically want a conditional gather. For example, I may scatter across libraries, analyze each independently, then group the libraries by sample, and only run a task for sample that had more than one library:

val sampleScatter = libraryScatter
  .map { library => analyzeLibrary(library) }
  .groupBy { library => (library.sampleName, library.libraryId }
  .map { case (_, libraries) => analyzeSample(libraries) } // returns a task per sample

sampleScatter.gather { samples => // a Seq of tasks, one per sample
   if (samples.exists(_.libraries.size > 1) /** do something */ else NoOpTask
}  

Another example would be where this is allowed:

class ConditionalTask(condition: Boolean, task: Task) extends Task {
  override def getTasks: Traversable[_ <: Task] = if (condition) Some(task) else None 
}

which will fail in dagr!

[tfenne]

I'm curious why your second example will fail? Both Some(task) and None extend Traversable[_ <: Task]. Are you saying dagr doesn't like it if you return nothing from getTasks? Maybe we should fix that?

[nh13]

Dagr doesn’t like tasks that return an empty sequence of tasks. I’ll look into it.

[nh13]

I think I also want the flatMap method off a Scatter to accept either a Scatter[_] or None. Hmmn.