diff --git a/docs/en/solutions/How_to_migrate_harbor_registry_pvc_storage_to_s3.md b/docs/en/solutions/How_to_migrate_harbor_registry_pvc_storage_to_s3.md new file mode 100644 index 00000000..eea9d1ad --- /dev/null +++ b/docs/en/solutions/How_to_migrate_harbor_registry_pvc_storage_to_s3.md @@ -0,0 +1,223 @@ +--- +products: + - Alauda DevOps +kind: + - Solution +--- + +# Harbor Registry Storage Migration: PVC to S3 + +## Issue + +This guide provides step-by-step instructions for migrating Harbor registry data from PVC (Persistent Volume Claim) storage to S3-compatible storage. This migration helps improve scalability and reduces storage management overhead. + +## Environment + +This solution is compatible with Alauda Build of Harbor v2.12.z. + +## Resolution + +### Prerequisites + +Before starting the migration, ensure you have: + +- **Important**: A fully deployed Harbor instance with `read-only mode` enabled. To enable read-only mode, Navigate to Harbor web `Administration → Configuration → System Settings → Repository Read Only`. +- **Important**: Since Harbor needs to be set to read-only mode during migration, it's recommended to simulate this process in a test environment first, evaluate the migration time, and allocate sufficient maintenance window. +- An S3-compatible storage service (MinIO, Ceph, AWS S3, etc.) with appropriate access credentials. +- A pre-created S3 bucket for storing Harbor registry data. +- Download and sync the rclone migration tool image to your internal registry for use in subsequent steps: + +```txt +# Download URL for China Region +https://cloud.alauda.cn/attachments/knowledge/337969938/rclone-amd64.tgz +https://cloud.alauda.cn/attachments/knowledge/337969938/rclone-arm64.tgz + +# Download URLs for Other Regions +https://cloud.alauda.io/attachments/knowledge/337969938/rclone-amd64.tgz +https://cloud.alauda.io/attachments/knowledge/337969938/rclone-arm64.tgz +``` + +### S3 Region Configuration + +#### How to Determine the Correct Region + +Please refer to your S3 provider's official documentation to determine the correct region for your specific service. Most providers will have this information available in their console, dashboard, or documentation. + +### Migration Process + +#### Migrate Registry Data to S3 + +This section describes how to migrate existing Harbor registry data from PVC to S3 storage using rclone. The migration process includes: + +1. **Data Synchronization**: Copy all registry data from PVC to S3 +2. **Data Verification**: Verify the integrity of migrated data + +Execute the following script to perform the migration: + +```bash +export S3_HOST=http://xxxxx:xxx # S3 storage endpoint +export S3_PROVIDER=Minio # Configure based on S3 type. Supported providers: Minio, Ceph, AWS, etc. Refer to: https://rclone.org/docs/#configure +export S3_KEY_ID=xxxx +export S3_ACCESS_KEY=xxxxx +export S3_BUCKET=harbor # Create this bucket in S3 beforehand +export S3_REGION=us-east-1 # If S3 doesn't have regions, this is not needed. If it exists, configure it and add region = $S3_REGION in the config below +export SYNC_IMAGE=rclone/rclone:1.71.0 # Replace with your internal registry image +export HARBOR_REGISTRY_PVC=xxxxx +export HARBOR_NS=xxxxx + +cat>sync-and-check-s3.yaml< + REGISTRY_STORAGE_S3_SECRETKEY: +kind: Secret +metadata: + name: s3-secret + namespace: # Replace with your Harbor namespace +type: Opaque +``` + +Add the following content to the Harbor resource (note that storage configurations other than registry must be preserved): + +```yaml +apiVersion: operator.alaudadevops.io/v1alpha1 +kind: Harbor +metadata: + name: harbor +spec: + helmValues: + persistence: + enabled: true +# Add the following content + imageChartStorage: + disableredirect: true + s3: + existingSecret: s3-secret # an secret for S3 accesskey and secretkey + bucket: harbor # Storage bucket created in S3 cluster + region: us-east-1 # S3 region (required for AWS S3, optional for MinIO/Ceph) + regionendpoint: http://xxxxx # S3 cluster access address, note that the access port must be included + v4auth: true + type: s3 +# END +``` + +### Verification and Testing + +After completing the configuration update, verify that the migration was successful by testing Harbor functionality: + +1. **Test Docker Operations**: Log in to Harbor locally and verify that docker push/pull operations work correctly +2. **Check Storage**: Confirm that new images are being stored in the S3 bucket +3. **Verify Existing Images**: Ensure that previously migrated images can still be pulled successfully