This old solution uses stable ip addresses instead of stable pod names to achieve persistent etcd peerURLs It creates an etcd cluster using 3 services and 3 statefulset with replica = 1.
You can use the
resources/etcd.yaml file of the above repo to create a 3 node etcd cluster inside a minikube node (it contains 3 services and 3 stateful sets). This file is generated by the template inside the
templates dir and the provided Makefile can be used to generate the resource from the template. Inside a production deployment you should edit the template adapting it to your needs (for example changing the persistent volume claims template definition to match your architecture) and then run
make to regenerate
Your clients should use the 3 created services ClusterIPs as the etcd endpoints. You can also create a dedicated client service with a label selector that will choose all the pods in the etcd cluster and use its cluster ip.
The etcd nodes network packets will pass through the kubernetes iptables based service proxy and not directly from pod to pod. This shouldn't add any major overhead (since services are used in a lot of cases) but to be sure it should be benchmarked.
If you want to launch a command (like
etcdctl) inside one of the etcd pods to communicate with the same cluster and you randomically see that the command takes some seconds before returning a result be sure to have enabled the right kubelet
hairpin-mode. Without it a pod won't be able to talk to itself using the service ip address (the client will then timeout and switch to another member in the provided etcd endpoints).