Support marking nodes for maintenance

[tgross]

From https://www.joyent.com/blog/automatic-dns-updates-with-containerbuddy:

The problem is that if we simply remove the old container we'll have a period of lost traffic between the time we remove the container and the TTL expires. We need a way to signal Containerbuddy to mark the node for planned maintenance. I'll circle back to that in a revision to Containerbuddy and discuss this change in an upcoming post.

cc @misterbisson

[tgross]

Implementation thoughts:

• I'm going to update the internal API to use the term SendHeartbeat instead of WriteHealthCheck for clarity
• For initiating the action, we can have Containerbuddy accept a POSIX signal. SIGINT or SIGSTOP sound like the correct ones but if I had to hazard a guess there are probably bits of the Docker engine that are going to assume one or the other of these is available. So maybe using a SIGUSR1 would be better.
• When the signal is received, we should pause the heartbeat goroutine and send a MarkForMaintenance function call.
• The MarkForMaintenance function should be a new method of the DiscoveryService interface.
• In the Consul implementation, we can trigger an immediate marking of the node as "failed" by sending the appropriate API call to Consul (ref https://godoc.org/github.com/hashicorp/consul/api#Agent.FailTTL). On the next poll from a downstream client, that node will be removed from the list of heathy.

[misterbisson]

Agreed that SendHeartbeat seems more semantically correct for what's actually happening there.

[tgross]

#15 has been merged and I've got a release awaiting #19.

[tgross]

Included in release: https://github.com/joyent/containerbuddy/releases/tag/0.0.2-alpha