Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.7] etcd migrate: instead of scaleup playbook etcd server should be started back #7313

Conversation

vrutkovs
Copy link
Member

@vrutkovs vrutkovs commented Feb 27, 2018

Backport of #7226
Replaces #7297

Verified to be working on 3.7 cluster

@openshift-ci-robot openshift-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Feb 27, 2018
roles:
- openshift_etcd_facts

- name: Re-configure etcd and bring the cluster up
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks okay to me. Please update with test results.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to add a few more vars, now migrates 3.7 cluster correctly

…ed back

master doesn't need to be restarted and etcd URLs updated as new etcd
nodes are being added. There is no need to run scaleup playbook, as etcd
nodes are already added to the cluster.

The migrate procedure now does the following:
* checks if the etcd data needs to be migrated
* makes etcd backup
* stops etcd services on all nodes except the first one
* migrates etcd data on first etcd node
* clears data on other etcd nodes
* updates etcd cluster configuration, updating ETCD_INITIAL_CLUSTER and
ETCD_INITIAL_CLUSTER_STATE
* starts the etcd service again one by one on etcd nodes

Now the only copy of migrated data is on the first etcd cluster, which
would replicate it to other nodes.

After migration is done master configs are updated and services
restarted if needed
@vrutkovs vrutkovs force-pushed the 3.7-migrate-start-etcd-and-check-health branch from b53a096 to 7e30cda Compare February 27, 2018 23:18
Copy link
Contributor

@michaelgugino michaelgugino left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Feb 28, 2018
@vrutkovs
Copy link
Member Author

/retest

1 similar comment
@vrutkovs
Copy link
Member Author

vrutkovs commented Mar 1, 2018

/retest

@vrutkovs
Copy link
Member Author

vrutkovs commented Mar 8, 2018

/test install

@openshift-ci-robot
Copy link

@vrutkovs: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/openshift-jenkins/logging 7e30cda link /test logging
ci/openshift-jenkins/extended_conformance_install_crio 7e30cda link /test crio
ci/openshift-jenkins/install 7e30cda link /test install

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@sdodson sdodson merged commit 5a60969 into openshift:release-3.7 Mar 8, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lgtm Indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants