Provide for dependencies between tasks in a group

[mfischer-zd]

Tasks in a group sometimes need to be ordered to start up correctly.

For example, to support the Ambassador pattern, proxy containers (P[n]) used for outbound request routing by a dependent application may be started only after the dependent application (A) is started. This is because Docker needs to know the name of A to configure shared-container networking when launching P[n].

In the first approximation of the solution, ordering can be simple, e.g., by having the task list in a group be an array.

[ChrisHines]

+1

Batch jobs need task dependencies to manage data-flow between tasks.

I would expand this to allow dependencies between tasks in a job rather than just a task group. Data flow is not constrained to a single node.

[suyash-repo]

Any tentative date to add this enhancement feature in Nomad?

[dadgar]

@suyash1983: we don't have any dates yet but it will most likely be something that comes in 0.4

[blalor]

This is really going to be core to a number of use cases, like allowing EBS volumes to be mounted on a host before the application starts.

[c4milo]

or pulling down certificates and secrets from Vault before starting a given service.

[vrenjith]

+1
This will be a great addition for large enterprise cluster deployments

[steve-jansen]

+1

Running consul-template alongside an app...

[dadgar]

@steve-jansen You can run consul-template currently. We do this in our Nomad deployment. Essentially you have a script that runs consul-template which produces your config, and then runs your binary

[steve-jansen]

@dadgar nice, thanks for sharing. It's always great to hear how a company uses their own products. I imagine consul-template creates the env vars in the job config...

One catch for us, we aspire to have consul-template rewrite the config in response to changes in a Consul k/v, or more importantly, when Vault rotates a secret. We're trying to have consul-template signal the co-scheduled task to reload its config.

Indeed, my need for task dependencies is very narrow. In a perfect world for me, Nomad would have integration with Consul K/V and/or Vault. My need is config that updates dynamically for tasks. That would eliminate my need for task dependencies.

[dadgar]

@steve-jansen consul-template can supervise the process and restart/signal it whenever the config changes. @sethvargo to verify