Bug 1810469: The disaster recovery scripts need simplification #220

retroflexer · 2020-02-28T20:08:13Z

With the introduction of cluster-etcd-operator, much of the functionality in the old scripts is no longer needed. Therefore the scripts needed to be trimmed down and simplified. This PR attempts to achieve that goal.

bindata/etcd/etcd-member-remove.sh

bindata/etcd/etcd-restore-backup.sh

deads2k · 2020-03-02T14:12:24Z

bindata/etcd/etcd-restore-backup.sh

-    echo 'Path to the directory containing backup files is required: ./etcd-snapshot-restore.sh <path-to-backup>'
-    exit 1
+function usage {
+  echo 'Path to the directory containing backup files is required: ./etcd-snapshot-restore.sh <path-to-backup>'


this isn't descriptive enough for me. What content is expected here

Well, our current backup is expected to be in a directory containing two components:

etcd snapshot

Bundle of Static POD resources (which includes kube-apiserver's static pod resources, that may include encryption key)

Well, our current backup is expected to be in a directory containing two components:

etcd snapshot

Bundle of Static POD resources (which includes kube-apiserver's static pod resources, that may include encryption key)

don't tell me, write in the usage for everyone :)

Added to the usage.

bindata/etcd/etcd-restore-backup.sh

retroflexer · 2020-03-02T15:19:12Z

Currently, the scripts assume that the following variables come from etcd.env file:

${ASSET_DIR}
${CONFIG_FILE_DIR}
${ETCD_DATA_DIR_BACKUP}
${MANIFEST_DIR}
${MANIFEST_STOPPED_DIR}
${RESTORE_ETCD_POD_YAML}
${STATIC_POD}

and ${IMAGE} for downloading the etcdctl binary.

bindata/etcd/etcd-restore-backup.sh

alaypatel07 · 2020-03-05T18:53:30Z

bindata/etcd/etcd-snapshot-restore.sh

+  echo "etcd snapshot ${SNAPSHOT_FILE} does not exist."
+  exit 1
+fi
+


out of curiosity, why do we not care to check if BACKUP_FILE is not empty?

Well. We didn't have the size check in the past scripts :-). I was just trying to simplify the existing scripts.

alaypatel07 · 2020-03-05T18:56:59Z

bindata/etcd/etcd-snapshot-restore.sh

+
+
+# Restore static pod resources
+tar -C ${CONFIG_FILE_DIR} -xzf ${BACKUP_FILE} static-pod-resources


again just curious: should moving kube static pod be after etcd is moved few lines below?

That way when kube-apiserver is started it will have an etcd to talk to, or we have made an effort for etcd to exist :)

I can move it below. In the older scripts, we stopped kubelet, and restarted kubelet at the end. But based on David's comments, we let the kubelet be dirty.

retroflexer · 2020-03-05T21:30:04Z

/retest

hexfusion · 2020-03-05T22:49:17Z

/retest

retroflexer · 2020-03-05T23:34:28Z

/test e2e-gcp

openshift-ci-robot · 2020-03-05T23:34:39Z

@retroflexer: This pull request references Bugzilla bug 1810469, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.5.0) matches configured target release for branch (4.5.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1810469: The disaster recovery scripts need simplification

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

retroflexer · 2020-03-06T02:11:57Z

/retest

hexfusion · 2020-03-06T03:50:11Z

/retest

retroflexer · 2020-03-06T11:13:24Z

/retest

hexfusion · 2020-03-06T11:14:53Z

/lgtm

openshift-ci-robot · 2020-03-06T11:17:37Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hexfusion, retroflexer

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [hexfusion]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

openshift-bot · 2020-03-06T12:01:26Z

/retest

Please review the full test history for this PR and help us cut down flakes.

hexfusion · 2020-03-06T12:42:41Z

/retest

retroflexer · 2020-03-06T13:45:03Z

/retest

openshift-ci-robot · 2020-03-06T14:50:57Z

@retroflexer: The following test failed, say /retest to rerun all failed tests:

Test name	Commit	Details	Rerun command
ci/prow/e2e-azure	`0531dc4`	link	`/test e2e-azure`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

hexfusion · 2020-03-06T15:56:33Z

/cherrypick release-4.4

openshift-cherrypick-robot · 2020-03-06T15:56:34Z

@hexfusion: once the present PR merges, I will cherry-pick it on top of release-4.4 in a new PR and assign it to you.

In response to this:

/cherrypick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-03-06T15:59:42Z

@retroflexer: All pull requests linked via external trackers have merged. Bugzilla bug 1810469 has been moved to the MODIFIED state.

In response to this:

Bug 1810469: The disaster recovery scripts need simplification

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-cherrypick-robot · 2020-03-06T16:00:27Z

@hexfusion: #220 failed to apply on top of branch "release-4.4":

.git/rebase-apply/patch:287: trailing whitespace.
# ./etcd-snapshot-restore.sh $path-to-backup 
.git/rebase-apply/patch:1575: trailing whitespace.
# ./etcd-snapshot-restore.sh $path-to-backup 
warning: 2 lines add whitespace errors.
Using index info to reconstruct a base tree...
M	bindata/etcd/openshift-recovery-tools
A	pkg/etcdenvvar/envvarcontroller.go
A	pkg/etcdenvvar/etcd_env.go
M	pkg/operator/etcd_assets/bindata.go
M	pkg/operator/scriptcontroller/scriptcontroller.go
M	pkg/operator/starter.go
Falling back to patching base and 3-way merge...
Auto-merging pkg/operator/targetconfigcontroller/etcd_env.go
Auto-merging pkg/operator/starter.go
CONFLICT (content): Merge conflict in pkg/operator/starter.go
Auto-merging pkg/operator/scriptcontroller/scriptcontroller.go
Auto-merging pkg/operator/etcd_assets/bindata.go
CONFLICT (content): Merge conflict in pkg/operator/etcd_assets/bindata.go
CONFLICT (modify/delete): pkg/etcdenvvar/envvarcontroller.go deleted in HEAD and modified in Simplify the disaster recovery scripts. Version Simplify the disaster recovery scripts of pkg/etcdenvvar/envvarcontroller.go left in tree.
CONFLICT (modify/delete): bindata/etcd/openshift-recovery-tools deleted in Simplify the disaster recovery scripts and modified in HEAD. Version HEAD of bindata/etcd/openshift-recovery-tools left in tree.
Removing bindata/etcd/etcd-restore-backup.sh
error: Failed to merge in the changes.
Patch failed at 0001 Simplify the disaster recovery scripts

In response to this:

/cherrypick release-4.4

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 28, 2020

retroflexer changed the title ~~Scripts simplification~~ [wip]: Scripts simplification Feb 28, 2020

openshift-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 28, 2020

openshift-ci-robot requested review from alaypatel07 and mfojtik February 28, 2020 20:08

retroflexer commented Feb 28, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Outdated Show resolved Hide resolved

deads2k reviewed Feb 28, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Outdated Show resolved Hide resolved

deads2k reviewed Feb 28, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Outdated Show resolved Hide resolved

deads2k reviewed Feb 28, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Outdated Show resolved Hide resolved

deads2k reviewed Feb 28, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Show resolved Hide resolved

retroflexer force-pushed the scripts-simplification branch from 4e12940 to 5093343 Compare March 2, 2020 10:37

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 2, 2020

retroflexer force-pushed the scripts-simplification branch from 5093343 to 63e7ef9 Compare March 2, 2020 13:51

deads2k reviewed Mar 2, 2020

View reviewed changes

bindata/etcd/etcd-member-remove.sh Show resolved Hide resolved

deads2k reviewed Mar 2, 2020

View reviewed changes

bindata/etcd/etcd-restore-backup.sh Outdated Show resolved Hide resolved

deads2k reviewed Mar 2, 2020

View reviewed changes

bindata/etcd/etcd-restore-backup.sh Outdated Show resolved Hide resolved

deads2k reviewed Mar 2, 2020

View reviewed changes

bindata/etcd/etcd-restore-backup.sh Outdated Show resolved Hide resolved

retroflexer force-pushed the scripts-simplification branch 2 times, most recently from 253a5f1 to 1c67ced Compare March 2, 2020 15:02

retroflexer force-pushed the scripts-simplification branch from 1c67ced to 1c97918 Compare March 2, 2020 15:53

openshift-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 2, 2020

retroflexer force-pushed the scripts-simplification branch from 1c97918 to 40896ff Compare March 3, 2020 11:33

retroflexer changed the title ~~[wip]: Scripts simplification~~ Scripts simplification Mar 4, 2020

openshift-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 4, 2020

hexfusion reviewed Mar 4, 2020

View reviewed changes

bindata/etcd/etcd-restore-backup.sh Outdated Show resolved Hide resolved

retroflexer force-pushed the scripts-simplification branch from 40896ff to 9dd3672 Compare March 4, 2020 19:47

retroflexer force-pushed the scripts-simplification branch from 00c5f41 to c879118 Compare March 5, 2020 18:25

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Mar 5, 2020

alaypatel07 reviewed Mar 5, 2020

View reviewed changes

retroflexer force-pushed the scripts-simplification branch from c879118 to 4a3e968 Compare March 5, 2020 19:11

retroflexer closed this Mar 5, 2020

retroflexer reopened this Mar 5, 2020

Simplify the disaster recovery scripts

0531dc4

retroflexer force-pushed the scripts-simplification branch from 4a3e968 to 0531dc4 Compare March 5, 2020 23:36

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Mar 6, 2020

openshift-merge-robot merged commit 173b86c into openshift:master Mar 6, 2020

retroflexer mentioned this pull request Mar 6, 2020

[release-4.4] Bug 1810723: The disaster recovery scripts need simplification #241

Merged

retroflexer deleted the scripts-simplification branch June 2, 2021 00:01

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Bug 1810469: The disaster recovery scripts need simplification #220

Bug 1810469: The disaster recovery scripts need simplification #220

retroflexer commented Feb 28, 2020 •

edited

deads2k Mar 2, 2020

retroflexer Mar 2, 2020

deads2k Mar 4, 2020

retroflexer Mar 4, 2020

retroflexer commented Mar 2, 2020 •

edited

alaypatel07 Mar 5, 2020

retroflexer Mar 5, 2020

alaypatel07 Mar 5, 2020 •

edited

retroflexer Mar 5, 2020

retroflexer commented Mar 5, 2020

hexfusion commented Mar 5, 2020

retroflexer commented Mar 5, 2020

openshift-ci-robot commented Mar 5, 2020

retroflexer commented Mar 6, 2020

hexfusion commented Mar 6, 2020

retroflexer commented Mar 6, 2020

hexfusion commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

openshift-bot commented Mar 6, 2020

hexfusion commented Mar 6, 2020

retroflexer commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

hexfusion commented Mar 6, 2020

openshift-cherrypick-robot commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

openshift-cherrypick-robot commented Mar 6, 2020



		# Restore static pod resources
		tar -C ${CONFIG_FILE_DIR} -xzf ${BACKUP_FILE} static-pod-resources

Bug 1810469: The disaster recovery scripts need simplification #220

Bug 1810469: The disaster recovery scripts need simplification #220

Conversation

retroflexer commented Feb 28, 2020 • edited

deads2k Mar 2, 2020

Choose a reason for hiding this comment

retroflexer Mar 2, 2020

Choose a reason for hiding this comment

deads2k Mar 4, 2020

Choose a reason for hiding this comment

retroflexer Mar 4, 2020

Choose a reason for hiding this comment

retroflexer commented Mar 2, 2020 • edited

alaypatel07 Mar 5, 2020

Choose a reason for hiding this comment

retroflexer Mar 5, 2020

Choose a reason for hiding this comment

alaypatel07 Mar 5, 2020 • edited

Choose a reason for hiding this comment

retroflexer Mar 5, 2020

Choose a reason for hiding this comment

retroflexer commented Mar 5, 2020

hexfusion commented Mar 5, 2020

retroflexer commented Mar 5, 2020

openshift-ci-robot commented Mar 5, 2020

retroflexer commented Mar 6, 2020

hexfusion commented Mar 6, 2020

retroflexer commented Mar 6, 2020

hexfusion commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

openshift-bot commented Mar 6, 2020

hexfusion commented Mar 6, 2020

retroflexer commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

hexfusion commented Mar 6, 2020

openshift-cherrypick-robot commented Mar 6, 2020

openshift-ci-robot commented Mar 6, 2020

openshift-cherrypick-robot commented Mar 6, 2020

retroflexer commented Feb 28, 2020 •

edited

retroflexer commented Mar 2, 2020 •

edited

alaypatel07 Mar 5, 2020 •

edited