Skip to content

This repo contains example bash scripts and yaml files for backing up and restoring OpenShift clusters with OADP

License

Notifications You must be signed in to change notification settings

IBM/openshift-oadp-examples

Repository files navigation

OADP Blog Resources

This repository contains example files and scripts for backing up and restoring an OpenShift cluster. The resources provided here are intended to help you understand the process and customize it for your environment.

Table of Contents

DPA, Backups, and Cloud Credential

The dpa-backups-cloud-credential directory contains a file for the Data Protection Applications (DPA), example backup files and a cloud-credenial file.

  • oadp-dpa.yaml
    This YAML file configures backup and snapshot locations, specifying the necessary details for storing and managing data protection. Kopia is an open-source tool used for creating, managing, and restoring backups. In the context of the OADP, kopia serves as the uploader type for the nodeAgent, which means it is the mechanism responsible for efficiently transferring and storing backup data.

    Note you can have multiple backup locations specified in the DPA, you would need to duplicate lines 8-22 and update for a second location.

  • ibmcos-bucket-creds.json This file needs to be updated to contain the HMAC credentials from your COS bucket. Refer back to the blog for more details on how to retrieve your credentials and create the credentials on the cluster.

  • backup-csi.yaml
    This YAML file configures a backup resource named backup-name-csi in the openshift-adp namespace. It specifies dpa-name-1 as the volume snapshot location and excludes the openshift-adp and openshift-storage namespaces from the backup. The backup uses volume snapshots (snapshotVolumes: true) and is stored in backup-location-name-from-dpa. It has a TTL of 72000 hours, meaning it will be deleted after 3000 days.

  • backup-filesystem.yaml
    This YAML file configures a backup resource named backup-name-filesystem in the openshift-adp namespace. It specifies dpa-name-1 as the volume snapshot location and excludes the openshift-adp and openshift-storage namespaces from the backup and defaults to file system backups for all volumes. It enables volume snapshots and their data movement to be stored in backup-location-name-from-dpa. It has a TTL of 72000 hours, meaning it will be deleted after 3000 days.

Getting Started

  1. Clone this repository:

    git clone https://github.com/IBM/openshift-oadp-examples.git
    cd openshift-oadp-examples
  2. Review and modify the example files as necessary to fit your OpenShift environment.

  3. You will need to authenticate to your cluster through the command-line.

    To log in to an OpenShift cluster using the command line, follow these steps:

    1. Install the OpenShift CLI (oc): If you haven't already, you need to download and install the OpenShift CLI (oc). You can find the installation instructions here.

    2. Obtain the Login Command: You need the login command provided by your OpenShift cluster administrator or found in the OpenShift web console. This typically includes the API server URL and a token or username and password.

    3. Run the Login Command: Use the provided login command in your terminal. The command should look like the following:

    • Log in with this token:
      oc login --token=sha256..... --server=https.....
  4. Execute the main script to perform a restore.

Restore Files

The restores directory contains YAML files necessary for restoring your OpenShift cluster. These files define the state of the cluster resources to be restored, such as deployments, services, and persistent volume claims.

  • restore-1-example.yaml
    In summary, this YAML file is configuring a restore operation named restore-1-example in the openshift-adp namespace, restoring from a backup named backup-name, and including specific namespaces (namespace-1 and namespace-2) and resources (project, secrets, routes). Additionally, it specifies a timeout for operations and uses a ConfigMap named resource-modifier for resource modification during the restore process.

  • restore-2-example-csi.yaml
    This YAML file defines a restore operation named restore-2-example-csi in the openshift-adp namespace. It will restore resources from a backup named backup-name, specifically targeting resources in namespace-3 and including persistent volumes, persistent volume claims, and volume snapshots. The operation has a timeout of 20 minutes per item and will restore persistent volumes as well.

  • restore-2-example-filesystem.yaml
    This YAML file defines a restore operation named restore-2-example-filesystem in the openshift-adp namespace. It will restore resources from a backup named backup-name, focusing on the namespace-3 namespace and restoring only pods, persistent volumes, and persistent volume claims. Each item operation in the restore process has a timeout limit of 20 minutes.

Resource Modifier

The resource-modifier directory includes a configuration file used to modify resources. These can be used to change resource configurations to fit the target environment. This file is an example that can be used if you need to update a route or a secret.

The provided resource modifier file demonstrates how to apply patches to secrets and routes. This example serves as a template for basic modifications, showing the standard method for patching these resources.

Limitations

In our experience, not all secrets can be patched using a resource modifier. For some complex secrets, a script was used to achieve the desired modifications. This script offers a more flexible approach to handle intricate scenarios. The script is provided in this repo search-replace-secrets.sh.

  • resource-modifier.yaml:
    This YAML file defines a Kubernetes ConfigMap named resource-modifier in the openshift-adp namespace, containing rules to modify specific resources. It updates a secret's stringData.url field in the secret-namespace-here namespace and replaces a value in a route's path in the route-namespace-here namespace.

Unmounted PVs

The unmounted-pvs directory contains a file related to mounting a Persistent Volume (PV) if it is not mounted. This step is crucial for accessing persistent data during the backup and restore process. If a PV is not mounted, it will not be backed up.

  • mounting-pod.yaml: YAML file to create and mount the required volumes that are not mounted.

Scripts

The scripts directory contains various utility scripts used throughout the restore process and are referenced in the main-script.sh. These scripts automate repetitive tasks and ensure consistency.

  • restore-utilities.sh
    This script contains various task that are needed multiple times in the main script.

  • delete-pods.sh
    Delete pods in a given namespace.

  • delete-secrets.sh:
    Delete secrets of a certain type in a given namespace.

  • search-replace-secrets.sh:
    Search and replace the old cluster name and updates with the new cluster name in a given namesapce. Multiple secrets can be patched at a time, need to be seperated by a comma.
    For example: "secret1,secret2,secret3,secret4,secret5"

Main Script

The main-script.sh is the primary script orchestrating the entire restore process. It calls other scripts and manages the workflow.

  • You are able to run this script in a dry run mode, so you are able to test the script without making any real changes to the cluster. This is a parameter that will need to be specified when running the script.
  • Specify using filesystem or CSI. If you are using filesystem set to true, if using CSI set to false. This is also a parameter that will be specified when running the script.

This script ensures that resources are modified appropriately, and restores are executed in a sequence.

Commands to run the main scrips are below:

This example command will run the script in dry run mode for CSI. Change DRY_RUN to false if you wish to run the script for real. Make sute to update the command with your cluster URL.

DRY_RUN=true USE_FILESYSTEM=false ./main-script.sh https://.cluster-name-123b456bb0fc123456d8bb0dff2739ca-0000.us-east.containers.appdomain.cloud backup-name

This command will run the script in dry run mode for Filesystem. Change DRY_RUN to false if you wish to run the script for real. Make sute to update the command with your cluster URL.

DRY_RUN=true USE_FILESYSTEM=true ./main-script.sh https://.cluster-name-123b456bb0fc123456d8bb0dff2739ca-0000.us-east.containers.appdomain.cloud backup-name

To get your cluster URL

  1. Sign-in to IBM Cloud account
  2. On the left hand side, open the Navigation Menu
  3. Navigate to Openshift > Clusters
  4. Select cluster
  5. Make sure you are on the Overview tab
  6. Scroll down till you find the Ingress subdomain
  7. Make sure to add https://

About

This repo contains example bash scripts and yaml files for backing up and restoring OpenShift clusters with OADP

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages