ansible: add roles for contiv storage service

[mapuri]

this shall start ceph and volmaster/supervisor/plugin service in the cluster.

@erikh PTAL when you get a chance

[erikh]

I would recommend leaving this blank; volsupervisor is extremely chatty in debug mode.

[mapuri]

sure, will do.

[erikh]

ok, one last thing:

ceph mons are more like etcd than block storage, they are a consensus database. The OSDs should be numerous, but the mons should not, and be an odd number like zookeeper/etc. I think we should distribute a volmaster with each mon, and point other docker volplugin instances at those.

@shaleman might have some strong opinions on the architecture here too.

Here's some more info: http://docs.ceph.com/docs/v0.71/rados/configuration/mon-config-ref/

[mapuri]

@erikh

ceph mons are more like etcd than block storage, they are a consensus database. The OSDs should be numerous, but the mons should not, and be an odd number like zookeeper/etc. I think we should distribute a volmaster with each mon, and point other docker volplugin instances at those.

I see, so I think I will make the change to run ceph mon and osds on service-master hosts and just ceph-osds on service-worker hosts.

For etcd, I have seen it work ok with 2 nodes but I agree for consensus usually we need odd number of nodes and we should be able to achieve that by having odd number of hosts in service-master group once we are able to support it as I described in my comment above.

[erikh]

That sounds like a great plan. Do you want to merge this as-is or wait until this work is done?

[mapuri]

cool...yeah, I will update the diffs to incorporate the comment for removing the --debug flag for volsupervisor and running just ceph osds on service-worker.

Will update the diffs in a bit and ask for you to take one more look.

[mapuri]

@erikh this should be ready for another look.

I took care of the comments or removing the --debug flag for volsupervisor and running just ceph osds on service-worker.

Right now it comes with a caveat that the ceph mons and osds related host-groups need to be provisioned together in same run of the ansible playbook. I will open an issue to enhance ceph playbook for allowing incremental configuration of mons and osds

[erikh]

OK, assign the other issue to me unless it's a blocker for you, I would
like to tackle this.

On 5 Nov 2015, at 16:34, Madhav Puri wrote:

@erikh this should be ready for another look.

I took care of the comments or removing the --debug flag for
volsupervisor and running just ceph osds on service-worker.

Right now it comes with a caveat that the ceph mons and osds related
host-groups need to be provisioned together in same run of the ansible
playbook. I will open an issue to enhance ceph playbook for allowing
incremental configuration of mons and osds

---

Reply to this email directly or view it on GitHub:
#20 (comment)

[mapuri]

cool thanks Erik. I have created #22 to track this. It is not a blocker for me right now as in the cluster's vagrant setup (i.e. using the Vagrantfile) I am able to provision all the cluster nodes at once.

I will need it once I start integrating the new playbooks with the individual node commission/decommission logic in cluster manager, which I am thinking can be tackled after docker con rush is over.

I will merge this one now in a bit.

[mapuri]

oops looks like I forgot to update the diffs before merge :( ... I will revert this change and create a new PR with the updated change in a bit.