Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update backup method #1375

Merged
merged 8 commits into from
May 9, 2024
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
107 changes: 106 additions & 1 deletion docs/creating-claudie-backup/creating-claudie-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,111 @@ These are the only services that will have a PVC attached to it, the other are s

## Backing up Claudie

### Manual backup

Claudie is already deployed on an existing Management Cluster and at least 1 Input Manifest has been applied.

Create a directory where the backup of the state will be stored.

```bash
mkdir claudie-backup
```

Put your Claudie inputmanifests into the created folder.
bernardhalas marked this conversation as resolved.
Show resolved Hide resolved
Despire marked this conversation as resolved.
Show resolved Hide resolved

We will now back up the state of the respective input manifests from MongoDB and MinIO.

```bash
kubectl get pods -n claudie

NAME READY STATUS RESTARTS AGE
ansibler-6f4557cf74-b4dts 1/1 Running 0 18m
builder-5d68987c86-qdfd5 1/1 Running 0 18m
claudie-operator-6d9ddc7f8b-hv84c 1/1 Running 0 18m
context-box-5d75bfffc6-d9qfm 1/1 Running 0 18m
create-table-job-ghb9f 0/1 Completed 1 18m
dynamodb-6d65df988-c626j 1/1 Running 0 18m
kube-eleven-556cfdfd98-jq6hl 1/1 Running 0 18m
kuber-7f8cd4cd89-6ds2w 1/1 Running 0 18m
make-bucket-job-9mjft 0/1 Completed 0 18m
minio-0 1/1 Running 0 18m
minio-1 1/1 Running 0 18m
minio-2 1/1 Running 0 18m
minio-3 1/1 Running 0 18m
mongodb-6ccb5f5dff-ptdw2 1/1 Running 0 18m
scheduler-676bd4468f-6fjn8 1/1 Running 0 18m
terraformer-66c6f67d98-pwr9t 1/1 Running 0 18m
```

To backup state from MongoDB execute the following command

```bash
kubectl exec -n claudie mongodb-<your-mongdb-pod> -- sh -c 'mongoexport --uri=mongodb://$MONGO_INITDB_ROOT_USERNAME:$MONGO_INITDB_ROOT_PASSWORD@localhost:27017/claudie -c inputManifests --authenticationDatabase admin' > claudie-backup/inputManifests
bernardhalas marked this conversation as resolved.
Show resolved Hide resolved
```

Next we need to backup the state from MinIO. Port-forward the MinIO service so that it is accessible from localhost.

```bash
kubectl port-forward -n claudie svc/minio 9000:9000
```

Setup an alias for the [mc](https://min.io/docs/minio/linux/reference/minio-mc.html) command line tool.

```bash
mc alias set claudie-minio http://127.0.0.1:9000 <ACCESSKEY> <SECRETKEY>
```

!!! note "Provide the access and secret key for minio. The default can be found in the github repository in the `manifests/claudie/minio/secrets` folder. If you have not changed them, we strongly encourage you to do so!"

Download the state into the backup folder

```bash
mc mirror claudie-minio/claudie-tf-state-files ./claudie-backup
```

You now have everything you need to restore your input manifests to a new management cluster.

!!! warning "These files will contain your credentials, DO NOT STORE THEM OUT IN THE PUBLIC!"

To restore the state on your new management cluster you can follow these commands. We expect that your default `kubeconfig` points to the new Management Cluster, if it does not, you can override it in the following commands using `--kubeconfig ./path-to-config`.

Copy the collection into the MongoDB pod.

```bash
kubectl cp ./claudie-backup/inputManifests mongodb-<your-mongodb-pod>:/tmp/inputManifests -n claudie
```

Import the state to MongoDB.

```bash
kubectl exec -n claudie mongodb-<your-mongodb-pod> -- sh -c 'mongoimport --uri=mongodb://$MONGO_INITDB_ROOT_USERNAME:$MONGO_INITDB_ROOT_PASSWORD@localhost:27017/claudie -c inputManifests --authenticationDatabase admin --file /tmp/inputManifests'
```

!!! note "Don't forget to delete the `/tmp/inputManifests` file"

Port-forward the MinIO service and import the backed up state.

```bash
mc cp --recursive ./claudie-backup/<your-folder-name-downloaded-from-minio> claudie-minio/claudie-tf-state-files
```

You can now apply your Claudie inputmanifests which will be immediately in the `DONE` stage. You can verify this with

```bash
kubectl get inputmanifests -A
```

Now you can make any new changes to your inputmanifests on the new management cluster and the state will be re-used.

!!! note "The secrets for the clusters, namely kubeconfig and cluster-metadata, are re-created after the workflow with the changes has finished."

!!! note "Alternatively you may also use any GUI clients for MongoDB and Minio for more straightforward backup of the state. All you need to backup is the bucket `claudie-tf-state-files` in MinIO and the collection `inputManifests` from MongoDB"


### Using Velero

!!! warning "Velero does not support HostPath volumes. If the PVCs in your management cluster are attached to such volumes, the backup will not work. In this case, use the above backup method."

All resources that are deployed or created by Claudie can be identified with the following label:

```
Expand Down Expand Up @@ -83,4 +188,4 @@ kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/
velero restore create --from-backup claudie-backup
```

Once all resources are restored, you should be able to deploy new input manifests and also modify existing infrastructure without any problems.
Once all resources are restored, you should be able to deploy new input manifests and also modify existing infrastructure without any problems.
2 changes: 2 additions & 0 deletions manifests/claudie/crd/claudie.io_inputmanifests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ metadata:
controller-gen.kubebuilder.io/version: v0.11.3
creationTimestamp: null
name: inputmanifests.claudie.io
labels:
app.kubernetes.io/part-of: claudie
spec:
group: claudie.io
names:
Expand Down