PX Backup 2.7.0 Sign off

[kshithijiyer-px]

---

PXB Sign-off-Template
Following are the tasks which has to be completed by following teams

NOTE: This issue should not be closed until PX-Backup build/version pushed to IBM Cloud Catalog

Sign-off process Step 1:

Portworx Team:

• Planned PX-Backup Release Version: https://docs.portworx.com/release-notes/portworx-backup-on-prem
• Lists of Tests executed:
• Screenshots/Summary of Test Execution:
• IKS/ROKS Support Matrix for new version: IKS: 1.29.x, 1.28.8, 1.27.x ROKS: 4.15.x, 4.14.x, 4.13.x, 4.12.x
• Steps for existing users to migrate to new versions: Same as 2.5.1
• Bug fixes V/S TestCase mapping:
• Please attach logs from both the PX-Backup and stork(source and destination cluster) components that were used during the execution of the tests described above:

IBM Team:

• Planned PX-Backup Release Version:
• Provide information about the IKS cluster(including BOM version) where integration tests were conducted:
• Provide information about the ROKS cluster(including BOM version) where integration tests were conducted:
• Torpedo Test Result's Details from IKS Cluster:
• Torpedo Test Result's Details from ROKS Cluster:
• IBM Team need to verify tasks and sign-off PX_Backup build

Sign-off process Step 2:

NOTE: Once above tasks completed and marked as done including sign-off then PX-Backup and IBM Team need to complete following tasks as well

PX-Backup Team:

• Push sign-off PX-Backup build to IBM Cloud Catalog in one region i.e us-east or so
• Verify torpedo tests(via automation) on default IKS and ROKS version - In case of test failure build has to be rollback and discussed with IBM Team.
• Push sign-off PX-Backup build to all IBM Cloud Catalog regions.

IBM Team:

• Run PX-Backup automated tests on one region for IKS and ROKS default version
• In case of tests failure discuss with PX-Backup team and take action accordingly
• Close this issue in case of success test result.

[kshithijiyer-px]

The test cases which have failed are due to PB-6896 PB-6819 which are Known issues which have been release noted in our documentation: https://docs.portworx.com/release-notes/portworx-backup-on-prem#known-issues

[kshithijiyer-px]

Testcase details have attached: px_backup_2.7.0_system_test.csv

[ambiknai]

Test Suite Passed !!
BasicBackupCreation | CancelAllRunningRestoreJobs | ScaleMongoDBWhileBackupAndRestore | BasicSelectiveRestore | BackupMultipleNsWithSameLabel 
PX-BACKUP-VERSION: 2.7.0
 
k8s: default (1.29.4 )

[ambiknai]

Test Suite Passed !!

BasicBackupCreation | CancelAllRunningRestoreJobs | ScaleMongoDBWhileBackupAndRestore | BasicSelectiveRestore | BackupMultipleNsWithSameLabel 
PX-BACKUP-VERSION: 2.7.0
 
ocp: default

ROKS: 4.14.24_openshift

[ambiknai]

@kshithijiyer-px Torpedo test on 4.15 cluster failed with connection error. Did you create cluster by disabling secure by default and then trigger tests

[kshithijiyer-px]

@kshithijiyer-px Torpedo test on 4.15 cluster failed with connection error. Did you create cluster by disabling secure by default and then trigger tests

Yes @ambiknai the images aren't push to catalog yet hence we would need to disable secure by default to allow to pull from docker.io.

[ambiknai]

@kshithijiyer-px Alright. When can we expect the images in catalog. Is 2.7.0 push planned in IBM Catalog once the images are in place.

[Balachandar-Pan]

@ambiknai Test catalog is updated and verified with ROKS 4.15.14_1537 with secure enabled via upgrade scenarios from PX-B 2.6.0 to 2.7.0.

[root@ip-10-13-8-162 ~]# helm ls -A
NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION
px-backup pxb270 2 2024-06-04 05:04:05.41204968 +0000 UTC deployed px-central-2.7.0 2.7.0

[root@ip-10-13-8-162 ~]# kubectl -n pxb270 get po
NAME READY STATUS RESTARTS AGE
alertmanager-px-backup-alertmanager-0 2/2 Running 0 30h
alertmanager-px-backup-alertmanager-1 2/2 Running 0 30h
prometheus-operator-6fc9bb4db-p7tbv 1/1 Running 0 30h
prometheus-px-backup-dashboard-prometheus-0 2/2 Running 0 30h
prometheus-px-backup-dashboard-prometheus-1 2/2 Running 0 30h
px-backup-5cfc5858d7-lhxrv 1/1 Running 0 30h
pxc-backup-mongodb-0 1/1 Running 0 30h
pxc-backup-mongodb-1 1/1 Running 0 30h
pxc-backup-mongodb-2 1/1 Running 0 30h
pxcentral-apiserver-6ccffdbbd6-f2rsb 1/1 Running 0 30h
pxcentral-backend-7c977b978b-jl4pq 1/1 Running 0 30h
pxcentral-frontend-648b698dd7-lwrn6 1/1 Running 0 30h
pxcentral-keycloak-0 1/1 Running 0 30h
pxcentral-keycloak-postgresql-0 1/1 Running 0 31h
pxcentral-lh-middleware-85dcd8559c-hzh7h 1/1 Running 0 30h
pxcentral-mysql-0 1/1 Running 0 30h
pxcentral-post-install-hook-5btt7 0/1 Completed 0 30h

[root@ip-10-13-8-162 ~]# kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
10.240.0.52 Ready master,worker 6d2h v1.28.9+8ca71f7 10.240.0.52 10.240.0.52 Red Hat Enterprise Linux 8.9 (Ootpa) 4.18.0-513.24.1.el8_9.x86_64 cri-o://1.28.6-5.rhaos4.15.gita02fb1e.el8
10.240.128.51 Ready master,worker 6d2h v1.28.9+8ca71f7 10.240.128.51 10.240.128.51 Red Hat Enterprise Linux 8.9 (Ootpa) 4.18.0-513.24.1.el8_9.x86_64 cri-o://1.28.6-5.rhaos4.15.gita02fb1e.el8
10.240.64.58 Ready master,worker 6d2h v1.28.9+8ca71f7 10.240.64.58 10.240.64.58 Red Hat Enterprise Linux 8.9 (Ootpa) 4.18.0-513.24.1.el8_9.x86_64 cri-o://1.28.6-5.rhaos4.15.gita02fb1e.el8
[root@ip-10-13-8-162 ~]#

pxb270_image version.txt

[ambiknai]

@Balachandar-Pan Thank you for sharing this result. May I know if I can execute same from torpedo. Is secure by default tests supported in torpedo test suite. Few days back when I ran the tests, it failed for 4.15

[kshithijiyer-px]

@ambiknai to run torpedo with Secure by Default enabled we need to push the app images to custom ICR repo and then Pass it to torpedo by using -e INTERNAL_DOCKER_REGISTRY=docker.pwx.dev.purestorage.com -e IMAGE_PULL_SERVER=docker.pwx.dev.purestorage.com -e IMAGE_PULL_USERNAME=**** -e IMAGE_PULL_PASSWORD=**** parameters. We haven't tried this approach we'll need time till next week as we are stuck with this release to build and get a pipeline ready with these changes.

@ambiknai If you have bandwidth can you please try in parallel and let us know if this approach works?

CC: @trenukarya-px

[ambiknai]

@kshithijiyer-px

What I understand is we have to push TORPEDO_IMG=portworx/torpedo-backup:master to icr.io.
Unfortunately we are not allowed to manually push any of the images to icr.io registry . We need an approved pipeline which we do not have as of now for pushing any of the portworx images. Could you post an update here once test is done from your end.

I know for PXE, images are pushed, to icr.io/ext/portworx by PX team.

[kshithijiyer-px]

@ambiknai We have already completed the tests which are documented in #22 (comment) and #22 (comment). Are you looking for something else as well?

[ambiknai]

@kshithijiyer-px I was talking about the approach you mentioned here