consul 0.8.0 and later failing to cluster

[rk295]

Hi,

I've just tried the stable/consul chart with ImageTag set to 0.8.0 (.1 and .2) and consul fails to form a cluster when deployed on a minikube test system.

The following errors are thrown by consul in the logs:

2017/05/12 17:54:15 [ERR] memberlist: Failed push/pull merge: Member 'youthful-heron-consul-2' has conflicting node ID '78788416-6a6d-1bf9-9d20-06f082e658f3' with this agent's ID from=172.17.0.11:37822
2017/05/12 17:54:15 [ERR] memberlist: Failed push/pull merge: Member 'youthful-heron-consul-1' has conflicting node ID '78788416-6a6d-1bf9-9d20-06f082e658f3' with this agent's ID from=172.17.0.10:35528

I'll do some research into why this might be and update this issue with any further info.

[rk295]

This is discussed more in hashicorp/consul#2877, looks like the -disable-host-node-id option was added in hashicorp/consul#2904.

I've gone ahead and modified a local copy of the chart, and can confirm this works with >=0.8.0.

The node-id is written to the persistent disk, meaning the node-id will survive restarts of a pod:

❯ kubectl get pods
NAME                    READY     STATUS    RESTARTS   AGE
hazy-whippet-consul-0   1/1       Running   0          2m
hazy-whippet-consul-1   1/1       Running   0          1h
hazy-whippet-consul-2   1/1       Running   0          1h
❯ kubectl exec -it hazy-whippet-consul-0 /bin/sh -- -c 'cat /var/lib/consul/node-id'
cc9f2e75-0b90-2533-40df-680826ca7b71
❯ kubectl delete pod hazy-whippet-consul-0
pod "hazy-whippet-consul-0" deleted
❯ kubectl exec -it hazy-whippet-consul-0 /bin/sh -- -c 'cat /var/lib/consul/node-id'
cc9f2e75-0b90-2533-40df-680826ca7b71

[lachie83]

This should be resolved. Closing