Quorum not being reached on machines with identical IDs

[abuehrle]

Running the network in Kubernetes on Bare Metal. Set the IPALLOC_RANGE in the weave-daemonset.yaml with the following:

containers:
        - name: weave
          env:
          - name: IPALLOC_RANGE
            value: 192.168.0.0/16
          image: weaveworks/weave-kube:latest
          imagePullPolicy: Always

Logs say for all three containers same collision:

INFO: 2017/02/02 20:24:09.535473 ->[147.75.100.177:56831|ea:ba:c8:b5:52:f9(kube-node-2.local.lan)]: connection shutting down due to error: local "ea:ba:c8:b5:52:f9(kube-node-2.local.lan)" and remote "ea:ba:c8:b5:52:f9(kube-node-1.local.lan)" peer names collision

I can see all of the weave containers in the container view (associated with each of the three hosts) as shown below:

But when I switch to the Weave Net view all I see is an error and status waiting for quorum:

[bboreham]

@abuehrle this looks like #2427, which should be fixed in the latest version which you are running.

Is it possible it's picking up persisted data from a 1.8 install?

[abuehrle]

This was the result of all nodes having the same machine-id. Fix was to run the following on each node:

rm /etc/machine-id
systemd-machine-id-setup

[weitzj]

Probably interesting to see how Scaleway (Online.net) does this in their own images:

systemd-machine-id-setup does not work and will always return the same id.

https://github.com/scaleway/image-voidlinux/blob/master/overlay-image-tools/usr/local/sbin/scw-gen-machine-id

#!/bin/sh
# description "generate a unique machine id"
# author "Scaleway <opensource@scaleway.com>"

if [ -f /etc/.regen-machine-id ]
then
	uuidgen > /etc/machine-id
	rm -f /etc/.regen-machine-id
fi

[weitzj]

Actually an upgrade to the above comment:

If you look at this diff https://github.com/scaleway/image-ubuntu/commit/d33d48a7e056b1e8a16cd129411872ff743f38fe it seems like you have to rm /etc/machine-id and rm /var/lib/dbus/machine-id

[bboreham]

Did you ever hear back from your machine provider how they expected users to deal with this, @abuehrle ?

[abuehrle]

Yes, they fixed a bug in the way they were provisioning machines, so this should no longer occur.