Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow adding a MDS to already deployed cluster #3599

Merged
merged 2 commits into from Apr 8, 2019
Merged

allow adding a MDS to already deployed cluster #3599

merged 2 commits into from Apr 8, 2019

Conversation

rishabh-d-dave
Copy link
Contributor

Add a playbook that creates a new MDS node as a part of already
deployed Ceph cluster and deploys MDS there.

@rishabh-d-dave rishabh-d-dave added DNM Do NOT merge feature labels Feb 12, 2019
@rishabh-d-dave rishabh-d-dave mentioned this pull request Feb 12, 2019
Merged
@rishabh-d-dave
Copy link
Contributor Author

TODO:

  • make add_mds work for container scenario

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Feb 12, 2019
@rishabh-d-dave
Copy link
Contributor Author

Attaching logs where dev-centos-non_container-add_mdss and dev-centos-container-add_mdss completed running successfully.

dev-centos-non_container-add_mdss-SUCCEEDED.log
dev-centos-container-add_mdss-SUCCEEDED.log
dev-centos-non_container-add_mdss.log
dev-centos-container-add_mdss.log

@guits guits changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Mar 26, 2019
@guits
Copy link
Collaborator

guits commented Mar 26, 2019

jenkins test dev-centos-non_container-add_mdss

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Mar 26, 2019
@guits guits changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Mar 26, 2019
@guits guits removed the DNM Do NOT merge label Mar 26, 2019
@guits guits added this to the v4.0 milestone Mar 26, 2019
guits
guits requested changes Mar 26, 2019
View reviewed changes
infrastructure-playbooks/add-mds.yml Outdated Show resolved Hide resolved
infrastructure-playbooks/add-mds.yml Outdated Show resolved Hide resolved
infrastructure-playbooks/add-mds.yml Outdated Show resolved Hide resolved
@dsavineau
Copy link
Contributor

out of my curiosity, does running the main playbook with --limit mdss would do the same than this standalone playbook ?

rishabh-d-dave
rishabh-d-dave commented Mar 26, 2019
View reviewed changes
tox.ini Outdated Show resolved Hide resolved
@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Mar 26, 2019 •
edited

out of my curiosity, does running the main playbook with --limit mdss would do the same than this standalone playbook ?

If the inventory file is not updated I think it would redploy MDS on mdss nodes but I can't be sure if the inventory files were updated it would have the same effect as running add-mdss.yml.

Originally, I did this mainly because I was following add-osd.yml and this README. But I am a little confused after reading it again. In contrast to what the README says, there are plenty instances where we are using roles -

$ grep -rnI "name: ceph-" infrastructure-playbooks/ | awk -F ':' '{print $1}' | uniq -c
      3 infrastructure-playbooks/shrink-osd.yml
      1 infrastructure-playbooks/shrink-osd-ceph-disk.yml
     11 infrastructure-playbooks/add-osd.yml
      3 infrastructure-playbooks/shrink-mon.yml
      4 infrastructure-playbooks/purge-docker-cluster.yml
     36 infrastructure-playbooks/switch-from-non-containerized-to-containerized-ceph-daemons.yml
      1 infrastructure-playbooks/storage-inventory.yml
      4 infrastructure-playbooks/rgw-standalone.yml
      1 infrastructure-playbooks/untested-by-ci/replace-osd.yml
     89 infrastructure-playbooks/rolling_update.yml
      3 infrastructure-playbooks/take-over-existing-cluster.yml
      4 infrastructure-playbooks/purge-cluster.yml

@guits @leseb Am I understanding it (the README) incorrectly?

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Mar 26, 2019 •
edited

@dsavineau @guits

Looks like it worked! I ran dev-centos-non_containerwith this patch to setup the initial cluster (see this attachment for output) and then reversed the earlier patch and applied this one and ran the same command again to add a mds node (see this one for output). Cluster looks fine to me -

[vagrant@mon0 ~]$ sudo ceph -k /var/lib/ceph/mon/ceph-mon0/keyring -n mon. status
  cluster:
    id:     efdb5a56-8018-41e3-a46f-d5e66638e57a
    health: HEALTH_WARN
            too few PGs per OSD (8 < min 30)
 
  services:
    mon: 1 daemons, quorum mon0 (age 28m)
    mgr: mon0(active, since 33m)
    mds: cephfs:1 {0=mds0=up:active}
    osd: 2 osds: 2 up (since 26m), 2 in (since 37m)
 
  data:
    pools:   2 pools, 16 pgs
    objects: 22 objects, 2.2 KiB
    usage:   27 GiB used, 35 GiB / 62 GiB avail
    pgs:     16 active+clean
 
[vagrant@mon0 ~]$

@rishabh-d-dave
Copy link
Contributor Author

jenkins test dev-centos-non_container-add_mdss

@rishabh-d-dave
Copy link
Contributor Author

@guits @dsavineau
There was only one error last time on CI. I don't think it was due to my patch.

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Apr 4, 2019
@guits guits changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Apr 4, 2019
@guits
Copy link
Collaborator

guits commented Apr 4, 2019

jenkins test dev-centos-non_container-switch_to_containers

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Apr 4, 2019
@rishabh-d-dave rishabh-d-dave changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Apr 4, 2019
@rishabh-d-dave
Copy link
Contributor Author

jenkins test dev-centos-non_container-switch_to_containers

@rishabh-d-dave rishabh-d-dave changed the title allow adding a MDS to already deployed cluster [skip ci]allow adding a MDS to already deployed cluster Apr 4, 2019
@guits guits changed the title [skip ci]allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Apr 4, 2019
@guits
Copy link
Collaborator

guits commented Apr 4, 2019

jenkins test dev-centos-container-add_mdss

@guits
Copy link
Collaborator

guits commented Apr 4, 2019

jenkins test dev-centos-non_container-add_mdss

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Apr 4, 2019
@guits guits changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Apr 8, 2019
@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-non_container-all_daemons

@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-container-all_daemons

@guits guits changed the title allow adding a MDS to already deployed cluster [skip ci] allow adding a MDS to already deployed cluster Apr 8, 2019
rishabh-d-dave and others added 2 commits April 8, 2019 13:05
@rishabh-d-dave
allow adding a MDS to already deployed cluster
03cd602 
Add a tox scenario that adds an new MDS node as a part of already
deployed Ceph cluster and deploys MDS there.

Signed-off-by: Rishabh Dave <ridave@redhat.com>
@guits @rishabh-d-dave
tests: run switch_to_containers against nautilus
665ccf0 
even on master, force the release to be nautilus.
this scenarios is failing because at multiple times this scenario is
actually downgrading the ceph version.
It might happen that the latest-master image is older than what was
deployed in the first step of the scenario (the RPM deployment).

Signed-off-by: Guillaume Abrioux <gabrioux@redhat.com>
(cherry picked from commit 41b0fa1)
@guits guits changed the title [skip ci] allow adding a MDS to already deployed cluster allow adding a MDS to already deployed cluster Apr 8, 2019
@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-container-all_daemons

@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-non_container-all_daemons

@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-non_container-add_mdss

@guits
Copy link
Collaborator

guits commented Apr 8, 2019

jenkins test dev-centos-container-add_mdss

@guits guits merged commit 4eb57ad into ceph:master Apr 8, 2019
@mergify mergify bot mentioned this pull request Apr 8, 2019
Merged
@rishabh-d-dave rishabh-d-dave deleted the add-mds branch April 22, 2019 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guits guits guits approved these changes

@leseb leseb Awaiting requested review from leseb

Assignees
No one assigned
Labels
backport-stable-4.0 feature
Projects
None yet
Milestone
v4.0
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@rishabh-d-dave @guits @dsavineau