Sync cluster bootstrap using etcd

[BugRoger]

This container allows to automatically bootstrap a cluster or distribute existing ceph config to new hosts. It makes use of etcd to store the inital configuration. It is especially cool when run on CoreOS and bootstrapping a cluster from nothing without manual intervention. It is compatible to the existing configuration mechanisms in the ceph-mon containers.

If this fits in the vision of ceph-docker I would be happy to keep on helping to maintain this config mechanism.

I'm using this on my own ceph cluster as can be seen here: https://github.com/BugRoger/d26a/tree/master/units/ceph

[Ulexus]

@BugRoger I quite like this idea, and it is looks very good to me. My only concern (and this as a CoreOS user, myself) is its dependency on etcd. For my part, I'd just add a note to the README indicating this dependency, but we should probably get @leseb 's opinion, on the matter.

[leseb]

@BugRoger Thanks a lot for pushing this. Actually that's part of the vision I have for ceph-docker.
I believe this is the big missing things in this repo, having an distributed entity to store config and keys is nice to have.
This was on my todo list :). Glad to see that this is here.

@Ulexus I understand your concern about the usage of etcd and it's fair. However I believe it has a sort of consensus now. Many big projects like Kubernetes are using it (Red Hat Atomic too).
So yes, somehow this is tied to CoreOS but nothing prevents you to use it.

I'd be really happy to see this in ceph-docker. I'll review that patch soon.

[leseb]

s/Aquire/Acquire/

[leseb]

@BugRoger Just reviewed that, the overall looks really good, I just left a couple of questions.

[leseb]

@Ulexus what do you think of adding etcdctl to ceph/base?

[Ulexus]

Absolutely

[Ulexus]

I'm pretty sure etcdctl 2 is compatible with etcd 0.4, so we should probably use it instead. CoreOS is still shipping with 0.4.7, though, so we should probably verify that (I'll do so when I get in to the office in a couple hours)

@leseb I think including etcdctl in base is a great idea. That would allow me to rewrite osd, Mon, etc, to (optionally) use etcd, as well. This feature was requested on my old repo, anyway.

[Ulexus]

Verified that etcdctl v2.0.4 works just fine with etcd v0.4.6 (current CoreOS beta, 584, and stable, 557.2). I don't have an alpha box running v0.4.7, but I can't imagine it not working with v0.4.7 if it works with v0.4.6. v0.4.7 is just a staging update for migration to 2.x.

[leseb]

@Ulexus Thanks for the quick tests :)

[Ulexus]

I don't see a reason not to commit this PR then update base with etcdctl v2.0.4, then update this component. Objections?

[Ulexus]

Well, except for the misspelling you found, anyway. :)

[leseb]

No objections unless @BugRoger wants to take care of that.
Since he didn't response to our comments, I believe we should merge this and rework what needs to be reworked :).

[Ulexus]

PR #29 updated etcdctl and moves it to ceph/base

[hunter]

A little late to the discussion. We'd started work on something similar using Consul for the KV. Any sense in using confd to make things a little more universal?

[leseb]

@hunter indeed confd will probably make the repo more universal.
If you have something really with confd I'd be happy to have a look at it.
Just created #34 so we can continue the discussion.

[BugRoger]

Thanks for considering this. And as I see you already fixed the initial comments. You folks are quick. 😄

[Ulexus]

@BugRoger Thanks for this; I just used it to rebuild my broken cluster. We should definitely integrate the other components to use it effectively.