Placeholder for feature parity with EmrEtlRunner

[alexanderdean]

Particularly around robustness.

This is a bit of an unfair ticket - it involves:

• Doing a detailed review of the EmrEtlRunner EMR invocation and monitoring code
• Identifying any missing features in Dataflow Runner
• Adding tickets into 0.2.0
• Closing this ticket and de-scheduling it from the 0.2.0 milestone

[BenFradet]

So far:

• EBS: Add support for EBS volume attachment to instance groups #4
• Apps: Add support for specifying applications to install on cluster  #6
• Ability to inspect step status and act on it (I don't really know it'll translate to df-runner yet)

[alexanderdean]

Ability to inspect step status and act on it (I don't really know it'll translate to df-runner yet)

I think this is fiddly but doable, but will need further thought...

[BenFradet]

Having given some thought to the last point, I'm not sure it really applies to df-runner since it seems really snowplow specific. What do you think @alexanderdean ?

[alexanderdean]

Hmm - I hear you. Let's break out the last point into a separate ticket "Explore options around ..." and put it into 0.3.0. This way gives us some more time to consider different options and whether we must have this in Dataflow Runner or if there is another way...