Skip to content

Adds the backup and restore guide for the Operator #432

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 17, 2022
Merged

Adds the backup and restore guide for the Operator #432

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions manage_quay/master.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,9 @@ include::modules/mirroring-tag-patterns.adoc[leveloffset=+2]
include::modules/mirroring-working-with.adoc[leveloffset=+2]
include::modules/mirroring-recommend.adoc[leveloffset=+2]

include::modules/backing-up-and-restoring-intro.adoc[leveloffset=+1]
include::modules/backing-up-red-hat-quay.adoc[leveloffset=+2]
include::modules/restoring-red-hat-quay.adoc[leveloffset=+2]

:context: manage_quay

Expand Down
4 changes: 4 additions & 0 deletions modules/backing-up-and-restoring-intro.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[[backing-up-and-restoring-intro]]
= Backing up and restoring Red Hat Quay on an OpenShift Container Platform deployment

Use the content within this section to back up and restore {productname} on an OpenShift Container Platform deployment.
212 changes: 212 additions & 0 deletions modules/backing-up-red-hat-quay.adoc
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,212 @@
[[backing-up-red-hat-quay]]
== Backing up Red Hat Quay

This procedure is exclusively for OpenShift Container Platform and NooBaa deployments.

.Prerequisites

* A {productname} deployment on OpenShift Container Platform.


.Procedure


. Backup the `QuayRegistry` custom resource by exporting it:
+
[source,terminal]
----
$ oc get quayregistry <quay-registry-name> -n <quay-namespace> -o yaml > quay-registry.yaml
----

. Edit the resulting `quayregistry.yaml` and remove the status section and the following metadata fields:
+
[source,yaml]
----
metadata.creationTimestamp
metadata.finalizers
metadata.generation
metadata.resourceVersion
metadata.uid
----

. Backup the managed keys secret:
+
[NOTE]
====
If you are running a version older than Red Hat Quay 3.7.0, this step can be skipped. Some secrets are automatically generated while deploying Quay for the first time. These are stored in a secret called `<quay-registry-name>-quay-registry-managed-secret-keys` in the QuayRegistry namespace.
====
+
[source,terminal]
----
$ oc get secret -n <quay-namespace> <quay-registry-name>-quay-registry-managed-secret-keys -o yaml > managed-secret-keys.yaml
----

. Edit the the resulting `managed-secret-keys.yaml` file and remove all owner references. Your `managed-secret-keys.yaml` file should look similar to the following:
+
[source,yaml]
----
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: <quayname>-quay-registry-managed-secret-keys
namespace: <quay-namespace>
data:
CONFIG_EDITOR_PW: <redacted>
DATABASE_SECRET_KEY: <redacted>
DB_ROOT_PW: <redacted>
DB_URI: <redacted>
SECRET_KEY: <redacted>
SECURITY_SCANNER_V4_PSK: <redacted>
----
+
All information under the `data` property should remain the same.

. Backup the current Quay configuration:
+
[source,terminal]
----
$ oc get secret -n <quay-namespace> $(oc get quayregistry <quay-registry-name> -n <quay-namespace> -o jsonpath='{.spec.configBundleSecret}') -o yaml > config-bundle.yaml
----

. Scale down the Quay the Quay Operator:
+
[source,terminal]
----
$ oc scale --replicas=0 deployment $(oc get deployment -n <quay-operator-namespace> |awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace>
----

. Scale down the Quay namespace:
+
[source,terminal]
----
$ oc scale --replicas=0 deployment $(oc get deployment -n <quay-namespace> -l quay-component=quay -o jsonpath='{.items[0].metadata.name}') -n <quay-namespace>
----

. Wait for the `registry-quay-app` pods to disappear. You can check their status by running the following command:
+
[source,terminal]
----
$ oc get pods -n <quay-namespace>
----
+
Example output:
+
[source,terminal]
----
registry-quay-config-editor-77847fc4f5-nsbbv 1/1 Running 0 9m1s
registry-quay-database-66969cd859-n2ssm 1/1 Running 0 6d1h
registry-quay-mirror-758fc68ff7-5wxlp 1/1 Running 0 8m29s
registry-quay-mirror-758fc68ff7-lbl82 1/1 Running 0 8m29s
registry-quay-redis-7cc5f6c977-956g8 1/1 Running 0 5d21h
----

. Identify the Quay PostgreSQL pod name:
+
[source,terminal]
----
$ oc get pod -l quay-component=postgres -n <quay-namespace> -o jsonpath='{.items[0].metadata.name}'
----
+
Exampe output:
[source,terminal]
----
quayregistry-quay-database-59f54bb7-58xs7
----

. Obtain the Quay database name:
+
[source,terminal]
----
$ oc -n <quay-namespace> rsh $(oc get pod -l app=quay -o NAME -n <quay-namespace> |head -n 1) cat /conf/stack/config.yaml|awk -F"/" '/^DB_URI/ {print $4}'
quayregistry-quay-database
----

. Download a backup database:
+
[source,terminal]
----
$ oc exec quayregistry-quay-database-59f54bb7-58xs7 -- /usr/bin/pg_dump -C quayregistry-quay-database > backup.sql
----

. Decode and export the `AWS_ACCESS_KEY_ID`:
+
[source,terminal]
----
$ export AWS_ACCESS_KEY_ID=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_ACCESS_KEY_ID}' |base64 -d)
----

. Decode and export the `AWS_SECRET_ACCESS_KEY_ID`:
+
[source,terminal]
----
$ export AWS_SECRET_ACCESS_KEY=$(oc get secret -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.AWS_SECRET_ACCESS_KEY}' |base64 -d)
----

. Create a new directory and copy all blobs to it:
+
[source,terminal]
----
$ mkdir blobs
----
+
[source,terminal]
+
----
$ aws s3 sync --no-verify-ssl --endpoint https://$(oc get route s3 -n openshift-storage -o jsonpath='{.spec.host}') s3://$(oc get cm -l app=noobaa -n <quay-namespace> -o jsonpath='{.items[0].data.BUCKET_NAME}') ./blobs
----


[NOTE]
====
You can also use link:https://rclone.org/[rclone] or link:https://s3tools.org/s3cmd[sc3md] instead of the AWS command line utility.
====

. Scale up the Quay the Quay Operator:
+
[source,terminal]
----
$ oc scale --replicas=1 deployment $(oc get deployment -n <quay-operator-namespace> |awk '/^quay-operator/ {print $1}') -n <quay-operator-namespace>
----

. Scale up the Quay namespace:
+
[source,terminal]
----
$ oc scale --replicas=1 deployment $(oc get deployment -n <quay-namespace> -l quay-component=quay -o jsonpath='{.items[0].metadata.name}') -n <quay-namespace>
----

. Check the status of the Operator:
+
[source,terminal]
----
$ oc get quayregistry <quay-registry-name> -n <quay-namespace> -o yaml
----
+
Example output:
+
[source,yaml]
----
apiVersion: quay.redhat.com/v1
kind: QuayRegistry
metadata:
...
name: example-registry
namespace: <quay-namespace>
...
spec:
components:
- kind: quay
managed: true
...
- kind: clairpostgres
managed: true
configBundleSecret: init-config-bundle-secret
status:
configEditorCredentialsSecret: example-registry-quay-config-editor-credentials-fg2gdgtm24
configEditorEndpoint: https://example-registry-quay-config-editor-quay-enterprise.apps.docs.gcp.quaydev.org
currentVersion: 3.7.0
lastUpdated: 2022-05-11 13:28:38.199476938 +0000 UTC
registryEndpoint: https://example-registry-quay-quay-enterprise.apps.docs.gcp.quaydev.org
0 5d21h
----
Loading