kube-control.connected state is removed on scale back

[jacekn]

As per subject kube-control.connected state is removed incorrectly during scale back.

To replicate start with fully deployed k8s bundle and:

juju add-unit kubernetes-master
# wait for the unit to deploy and then remove it
juju remove-unit kubernetes-master/1

Result:
kubernetes-worker units eventually end up in blocked state because of missing kube-control.connected state

Expected result:
kube-control.connected state is NOT removed since the relation is still in place

[tvansteenburgh]

Thanks, I'll fix this.

[johnsca]

Hrm. This relation isn't a peer relation, so it should not be in any way affected by scaling the master.

[johnsca]

The issue seems to be that KubeControlRequireer.scope is set to GLOBAL meaning that it expects to only ever have one kubes-master remote unit and if there are multiple, they all share the same conversation. I think this is partially an aspect of the conversation metaphor being inadequate and confusing (and there are plans to replace it with something simpler), but if the charms should be able to support scaling the master, then the scope needs to change and both the interface layer and charms updated to know how to deal with multiple masters being present.

[tvansteenburgh]

@johnsca Thanks for the research. Multi-master is definitely a thing, so we'll need to change the scope. Pretty sure this was a bug in the old kube-dns interface too, we just never noticed.