[TEST] Harvester v1.2.2 - Rancher v2.8.4 Integration Test

[albinsun]

Prerequisites

1. Setup 3 nodes Harvester v1.2.2 (QEMU/KVM, 8C/16G/500G per node)
2. Create a new storage class apart from default one.
3. Create 2 networks on different NICs: mgmt-vlan1 and bond-vlan1
4. Create 2 VMs with extra disk and data with md5sum: 1 running and 1 stopped
5. Take VM backups and snapshots on 2 VMs

Rancher Integration Test

v2.8.4

1. Setup Rancher v2.8.4 (Helm with v1.28.9+k3s1 on 2C/4G QEMU/KVM)
2. Import Harvester to Rancher
3. Create single node RKE2 cluster
   • harvester_e2e_tests/integrations/test_9_rancher_integration.py
   • Scale up to 2 nodes
   • Scale down to 1 node
4. Create single node RKE1 cluster
   • harvester_e2e_tests/integrations/test_9_rancher_integration.py
   • Scale up to 2 nodes
   • Scale down to 1 node
5. Create single node RKE2 cluster via Terraform
   • harvester_e2e_tests/integrations/test_z_terraform_rancher.py

v2.8.3 -> v2.8.4

1. Setup Rancher v2.8.3 (Helm with v1.28.9+k3s1 on 2C/4G QEMU/KVM)
2. Import Harvester to Rancher
3. Create single node RKE2 cluster
   • harvester_e2e_tests/integrations/test_9_rancher_integration.py
4. Upgrade Rancher to v2.8.4
   • Cluster still Active
   • Nginx deployment and LB still works
   • Scale up to 2 nodes
   • Scale down to 1 node

Reference

1. v1.3.0 Release Testing - Three node Harvester with Rancher #1150
2. v1.2.0 Release Testing - Rancher with fresh Harvester installation #900
3. https://github.com/rancher/rancher/releases/tag/v2.8.4

[albinsun]

Prerequisites

1. Setup 3 nodes Harvester v1.2.2 (QEMU/KVM, 8C/16G/500G per node)

   

2. Create a new storage class apart from default one.
3. Create 2 networks on different NICs: mgmt-vlan1 and bond-vlan1

   

4. Create 2 VMs with extra disk and data with md5sum: 1 Running and 1 Off

   • running w/ mgmt-vlan1

     
     

   • stopped w/ bond-vlan1

     
     

   • Both with extra data and md5sum

     

5. 🔴 Take VM backups and snapshots on 2 VMs

   • Backup ❌

     

     [BUG] Fail to backup a Stopped/Off VM due to volume error state harvester#5841

   • Snapshot

     

[albinsun]

Rancher Integration - v2.8.4

1. Setup Rancher v2.8.4 (Helm with v1.28.9+k3s1 on 2C/4G QEMU/KVM)

   

2. 🟡 Create single node RKE2 cluster (v1.28.9+rke2r1)

   • harvester_e2e_tests/integrations/test_9_rancher_integration.py::TestRKE2

     
     

     • test_import_harvester
     • test_create_rke2
     • test_create_pvc
     • test_csi_deployment
     • test_delete_deployment
     • test_deploy_nginx
     • test_load_balancer_service[1-dhcp]
     • test_load_balancer_service[1-pool]
   • Scale up to 2 nodes

     
     

   • Scale down to 1 node ❌

     

     Known [BUG] Fail to scale down RKE2 Harvester node driver cluster harvester#4358

3. 🟢 Create single node RKE1 cluster (v1.28.9)

   • harvester_e2e_tests/integrations/test_9_rancher_integration.py::TestRKE1

     
     

     • test_import_harvester
     • test_create_rke1
     • test_create_pvc
     • test_cloud_provider_chart
     • test_deploy_nginx
     • test_load_balancer_service[1-dhcp]
     • test_load_balancer_service[1-pool]
     • test_csi_driver_chart
     • test_csi_deployment
     • test_delete_deployment
   • Scale up to 2 nodes

     
     

   • Scale down to 1 node

     
     

4. 🟢 Create single node RKE2 cluster via Terraform

   • terraform version

     Terraform v1.8.4
     on linux_amd64
     + provider registry.terraform.io/rancher/rancher2 v3.1.1
     

   • harvester_e2e_tests/integrations/test_z_terraform_rancher.py

     

     • test_import_harvester
     • test_create_cloud_credential
     • test_create_machine_config
     • test_create_rke2_cluster
     • test_delete_rke2_cluster
     • test_delete_machine_config
     • test_delete_cloud_credential

[albinsun]

Rancher Integration - v2.8.3 -> v2.8.4

1. Setup Rancher v2.8.3 (Helm with v1.28.9+k3s1 on 2C/4G QEMU/KVM)

   

2. 🟢 Create single node RKE2 cluster (v1.28.9+rke2r1)

   • harvester_e2e_tests/integrations/test_9_rancher_integration.py::TestRKE2

     
     

     • test_import_harvester
     • test_create_rke2
     • test_create_pvc
     • test_csi_deployment
     • test_delete_deployment
     • test_deploy_nginx
     • test_load_balancer_service[1-dhcp]
     • test_load_balancer_service[1-pool]

3. 🟡 Upgrade Rancher to v2.8.4

   • Upgrade successfully

     

   • Cluster still Active

     

   • Nginx deployment and LB still works

     
     

   • Scale up to 2 nodes

     
     
     

   • Scale down to 1 node ❌

     

     Known [BUG] Fail to scale down RKE2 Harvester node driver cluster harvester#4358

[albinsun]

Provide test result on harvester-v1.2.2 with

1. rancher-v2.8.4 (detail)
2. rancher-v2.8.3 upgrade to rancher-v2.8.4 (detail)

Issues hit:

1. [BUG] Fail to backup a Stopped/Off VM due to volume error state harvester#5841
   • Only affect offline VM backup
2. [BUG] Fail to scale down RKE2 Harvester node driver cluster harvester#4358
   • Known scale down issue
   • Source issue is planned to fix in rancher-v2.9
     [BUG] Scaling down etcd machine pool can cause multiple machines to be deleted unintentionally rancher/rancher#42582

cc. @khushboo-rancher

[albinsun]

FYR, harvester-cloud-provider upgrade fail.

• [BUG] Fail to Upgrade harvester-cloud-provider on RKE2 cluster (rancher-v2.8.4) harvester#5873