Why am I failing that check?

[erikbgithub]

Hi,

I'm following this guide in trying to set up a small etcd cluster with docker.

The following is my shell script:

#!/bin/bash
docker network rm etcd
docker network create etcd --subnet=172.28.0.0/16 --ip-range=172.28.0.0/16

docker stop etcd01
docker rm etcd01
docker run -d \
    --network="etcd" \
    --ip 172.28.0.4 \
    -v $(pwd)/etcd01:/etc/ssl/certs \
    -p 4001:4001 -p 2380:2380 -p 2379:2379 \
    --hostname etcd01 \
    --name etcd01 quay.io/coreos/etcd:latest \
    etcd \
    -name etcd01 \
    -advertise-client-urls http://172.28.0.4:2379,http://172.28.0.4:4001 \
    -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \
    -initial-advertise-peer-urls http://172.28.0.4:2380 \
    -listen-peer-urls http://0.0.0.0:2380 \
    -initial-cluster-token etcd01 \
    -initial-cluster etcd01=http://172.28.0.4:2380,etcd01=http://172.28.0.2:2380,etcd02=http://172.28.0.3:2380 \
    -initial-cluster-state new

docker stop etcd02
docker rm etcd02
docker run -d \
    --network="etcd" \
    --ip 172.28.0.2 \
    -v $(pwd)/etcd02:/etc/ssl/certs \
    -p 4001:4001 -p 2380:2380 -p 2379:2379 \
    --hostname etcd02 \
    --name etcd02 quay.io/coreos/etcd:latest \
    etcd \
    -name etcd02 \
    -advertise-client-urls http://172.28.0.2:2379,http://172.28.0.2:4001 \
    -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \
    -initial-advertise-peer-urls http://172.28.0.2:2380 \
    -listen-peer-urls http://0.0.0.0:2380 \
    -initial-cluster-token etcd02 \
    -initial-cluster etcd01=http://172.28.0.4:2380,etcd01=http://172.28.0.2:2380,etcd02=http://172.28.0.3:2380 \
    -initial-cluster-state new

docker stop etcd03
docker rm etcd03
docker run -d \
    --network="etcd" \
    --ip 172.28.0.3 \
    -v $(pwd)/etcd03:/etc/ssl/certs \
    -p 4001:4001 -p 2380:2380 -p 2379:2379 \
    --hostname etcd03 \
    --name etcd03 quay.io/coreos/etcd:latest \
    etcd \
    -name etcd03 \
    -advertise-client-urls http://172.28.0.3:2379,http://172.28.0.3:4001 \
    -listen-client-urls http://0.0.0.0:2379,http://0.0.0.0:4001 \
    -initial-advertise-peer-urls http://172.28.0.3:2380 \
    -listen-peer-urls http://0.0.0.0:2380 \
    -initial-cluster-token etcd03 \
    -initial-cluster etcd01=http://172.28.0.4:2380,etcd01=http://172.28.0.2:2380,etcd02=http://172.28.0.3:2380 \
    -initial-cluster-state new

The containers always die with logs like this:

2017-06-10 03:56:20.274281 I | etcdmain: etcd Version: 3.1.8
2017-06-10 03:56:20.277344 I | etcdmain: Git SHA: d267ca9
2017-06-10 03:56:20.277401 I | etcdmain: Go Version: go1.7.5
2017-06-10 03:56:20.277438 I | etcdmain: Go OS/Arch: linux/amd64
2017-06-10 03:56:20.277488 I | etcdmain: setting maximum number of CPUs to 2, total number of available CPUs is 2
2017-06-10 03:56:20.277537 W | etcdmain: no data-dir provided, using default data-dir ./etcd01.etcd
2017-06-10 03:56:20.277764 I | embed: listening for peers on http://0.0.0.0:2380
2017-06-10 03:56:20.277864 I | embed: listening for client requests on 0.0.0.0:2379
2017-06-10 03:56:20.278641 I | embed: listening for client requests on 0.0.0.0:4001
2017-06-10 03:56:20.405664 I | etcdmain: --initial-cluster must include etcd01=http://172.28.0.4:2380 given --initial-advertise-peer-urls=http://172.28.0.4:2380

For some reason the first container even worked for some time. So I probably added a spelling error somewhere or something. Sorry that I can't find it. Can you see it?

Btw. is it a bug that the final log message is of type info and not error as indicated in the code? Also I have to admit that I don't really understand it. At first it says one variable doesn't seem to contain the expected content, but then it shows the contents of another variable. Shouldn't it show what is currently in --initial-cluster instead so that I can check what the difference is?

Please also note that there need to be quite a few changes to get this to run at all. The guide is not really sufficient yet. For instance that it just expects to use static IPs but doesn't show how static IPs are set up with docker, or that the developer probably used another image (as mentioned in #8078).

All this is not to say that etcd would be bad. It's a pillar of modern cluster management and I'm very happy that it exists. That's why I want to learn to use it even outside of k8s. I'd appreciate any help.

[gyuho]

That's v2 documentation. Try https://github.com/coreos/etcd/blob/master/Documentation/op-guide/container.md?

[gyuho]

Oh, did you mean to pass etcd01=http://172.28.0.4:2380,etcd02?

Not etcd01=http://172.28.0.4:2380,etcd01?

[xiang90]

The advertise url for etcd01 is http://172.28.0.4:2380 (only one url), but in the initial cluster you provide http://172.28.0.4:2380,http://172.28.0.2:2380 (two urls). They do not match.

The error message is a little bit confusing when users provide the two urls separately. probably worth to fix.

[erikbgithub]

Thanks, yes. With all the numbers I couldn't see it anymore. This confusion is removed.

Another problem was that if one starts the three docker containers on a single host, the part -p 4001:4001 -p 2380:2380 -p 2379:2379 should only be done in one container not all three. Maybe more people may run into this, since at least the first time you just do it on one host to figure out how all the docker containers and the etcd client should talk to each other. Maybe this could be mentioned with a comment.

[gyuho]

if one starts the three docker containers on a single host, the part -p 4001:4001 -p 2380:2380 -p 2379:2379 should only be done in one container not all three.

Can you be more specific? If you run 3-node cluster in a single host with the same port, ports would conflict?

[erikbgithub]

Yes, one host, my laptop. Not technically reasonable. Just learning.

[gyuho]

Then I would just use different ports for each node.
Here's sample local cluster configuration https://github.com/coreos/etcd#running-a-local-etcd-cluster.

[veerendra2]

Getting same issue, still don't understand why

veeru@ultron:~/kube-hard-way$ etcd --version
etcd Version: 3.2.11
Git SHA: 1e1dbb2
Go Version: go1.8.5

veeru@ultron:~/kube-hard-way$ sudo /usr/local/bin/etcd \
>   --name ultron \
>   --cert-file=/etc/etcd/kubernetes.pem \
>   --key-file=/etc/etcd/kubernetes-key.pem \
>   --peer-cert-file=/etc/etcd/kubernetes.pem \
>   --peer-key-file=/etc/etcd/kubernetes-key.pem \
>   --trusted-ca-file=/etc/etcd/ca.pem \
>   --peer-trusted-ca-file=/etc/etcd/ca.pem \
>   --peer-client-cert-auth \
>   --client-cert-auth \
>   --initial-advertise-peer-urls https://192.168.1.100:2380 \
>   --listen-peer-urls https://192.168.1.100:2380 \
>   --listen-client-urls https://192.168.1.100:2379 \
>   --advertise-client-urls https://192.168.1.100:2379 \
>   --initial-cluster-token etcd-cluster-0 \
>   --initial-cluster ultron=http://192.168.1.100:2380 \
>   --initial-cluster-state new \
>   --data-dir=/var/lib/etcd
2018-03-15 02:04:00.485093 I | etcdmain: etcd Version: 3.2.11
2018-03-15 02:04:00.485123 I | etcdmain: Git SHA: 1e1dbb2
2018-03-15 02:04:00.485127 I | etcdmain: Go Version: go1.8.5
2018-03-15 02:04:00.485130 I | etcdmain: Go OS/Arch: linux/amd64
2018-03-15 02:04:00.485133 I | etcdmain: setting maximum number of CPUs to 8, total number of available CPUs is 8
2018-03-15 02:04:00.485164 N | etcdmain: the server is already initialized as member before, starting as etcd member...
2018-03-15 02:04:00.485179 I | embed: peerTLS: cert = /etc/etcd/kubernetes.pem, key = /etc/etcd/kubernetes-key.pem, ca = , trusted-ca = /etc/etcd/ca.pem, client-cert-auth = true
2018-03-15 02:04:00.485758 I | embed: listening for peers on https://192.168.1.100:2380
2018-03-15 02:04:00.485788 I | embed: listening for client requests on 192.168.1.100:2379
2018-03-15 02:04:00.507700 I | etcdmain: --initial-cluster must include ultron=https://192.168.1.100:2380 given --initial-advertise-peer-urls=https://192.168.1.100:2380

Trying to setup etcd in single node(my laptop). Any help would be appreciated