There is no local etcd, removing 'etcdctl set'.

[gosharplite]

No description provided.

[brendandburns]

@kelseyhightower thoughts?

[rjnagal]

@kelseyhightower can you take a look?

[erictune]

@AntonioMeireles can you take a look at this PR and say if it looks safe?

[erictune]

@pires any comment here?

[erictune]

@bussyjd this touches the same area as your #6890. Want to comment on this PR?

[pires]

To me it feels like we should have kept the single etcd in the master node, where apiserver runs and not a cluster like @AntonioMeireles implemented.

In this case, removing this line (or the entire entry is OK), because

flannel:
    interface: eth1
    etcd_endpoints: http://<master-private-ip>:4001

[bussyjd]

@erictune If a single master node is the way to go it then makes sense to get rid of this line IMO with @pires 's PR
I am sorry if I bring back an old discussion but running etcd only on the master feels like it might cause some issues in niche cases when the master is not available (flanneld nw conf update for example). It defeats the purpose of etcd's HA.
Now might be the right time to weight the pros and cons if it haven't been done before otherwise It's totally fine to discard #6555 and #6890 in favor of #6281

[pires]

@bussyjd @AntonioMeireles was the one who brought etcd HA to light, since I was in favor of a simpler approach. People here will tell you etcd is just an implementation detail that may change in the future and have that said, I am to believe they don't actually care about etcd best-practices. And I agree with this idea. So, to me, if you're not happy with a single etcd node cluster, perhaps it's time to assemble it outside of Kubernetes?

Anyway, #6281 works for me.

[bussyjd]

@pires Thank you for the info.

So, to me, if you're not happy with a single etcd node cluster, perhaps it's time to assemble it outside of Kubernetes?

Is that in any world a productive answer?

@erictune It is now clear that case #6281 is the way to go.

[pires]

Is that in any world a productive answer?

@bussyjd I'm sorry if it offended you in some way, but it's actually how I use it in production. Once again, etcd is just an implementation detail. How coupled do we want its high-availability to be to Kubernetes?

[AntonioMeireles]

[sorry lag, mutitasking too much]

several issues/angles at stake (and now we even have etcd2 to grasp as it is already in latest CoreOS alpha).

• one approach we could use is the old one were the minions are absolutelly dumb and have no etcd (1, 2, whatever) of any kind, with everything pointing to master. with that approach one will always have to "cheat" one way or another and make sure that in no circunstance the nodes have the chance to want/need etcd before master is up. [this is easy to trigger specially in Vagrant with fast boxes/ssds and without VAGRANT_NO_PARALLEL=true.
• another approach that @pires thinks is too much (its is, but handy for testing) is just have a local etcd cluster, spread among master and nodes.
• finally, the last approach is while having a single-node etcd-master running on master having on all the minions etcd in proxy/client mode pointing to master. this way and without further cheating even if they start well in advance the minions will wait orderly that etcd (and whatever deps on it) is up and ready in master. [this is what is being done in this master/node and works afaict flawlessly.] Also, from my understanding this approach is the recomended way by upstream of consuming etcd and should cover all use cases.

@pires et all - any objection for a PR that implements last option (IMHO the way to go), plus full etcd2 awareness, plus fixing kubelet/kube-register interaction in 0.15.x ? (as afaik this 3 issues are the ones currently affecting this ?]

[pires]

@AntonioMeireles I'm OK with depending on etcd2 (CoreOS 563.0.0) and follow the proxy approach.

[AntonioMeireles]

@pires can you please when you time hand pick the mods from my tree to yours (now that they are closer again :-) ) for "independent" testing :-) to see if this time we get rid of all corner cases :-). Anyway PR will available for discussion after my lunch :-)

[pires]

@AntonioMeireles working on the proxy thing right now.

[pires]

@AntonioMeireles done and can confirm it's working.

[AntonioMeireles]

@pires - thnks - working on PR.

[AntonioMeireles]

fellows,

submited #6973. the discussion/review should probably jump to there 👼

[brendandburns]

closing this as it appears to be obsoleted by #6973