Default settings for Node removal are not optimal

[abronan]

Default settings for TTLs and Node removal on storage backends are not optimal and two annoying things could happen:

• It takes time using Consul and Etcd to detect a node failure, if the value of the TTL is high, swarm will see those nodes as alive until those entries are expired.
• This other scenario can happen:
  • Swarm agent registers onto discovery service
  • Manager sees the new entry: it is added to the list of known agents
  • Swarm agent fails
  • Manager removes the agent after the TTL expires
  • Swarm agent is back to life at the same time or directly after
  • It may take the Manager heartbeat time to register that node again although it's alive and working

One solution could be to reduce the default heartbeat and refresh time to a low value but at the risk of an increased pressure on the metadata storage cluster, especially if we use it for both discovery and metadata.