[RELEASE] 1.5.1

[innobead]

What's the task? Please describe.
Action items for releasing v1.5.1

Describe the items of the task (DoD, definition of done) you'd like.

• Pre-Release
  • Regression test plan (manual) - @khushboo-rancher
  • Run e2e regression for pre-GA milestones (install, upgrade) - @yangchiu
  • Run security testing of container images for pre-GA milestones - @yangchiu
  • Verify longhorn chart PR to ensure all artifacts are ready for GA (install, upgrade) @chriscchien
  • Run core testing (install, upgrade) for GA build from the previous patch and the last patch of the previous feature release (1.4.3, 1.5.0). - @roger-ryao
• Release
  • Release longhorn/chart from the release branch to publish to ArtifactHub
  • Release note
    • Deprecation note
    • Upgrade notes including highlighted notes, deprecation, compatible changes, and others impacting the current users
• Post-Release
  • Create a new release branch of manager/ui/tests/engine/longhorn instance-manager/share-manager/backing-image-manager when creating the RC1
  • Update https://github.com/longhorn/longhorn/blob/master/deploy/upgrade_responder_server/chart-values.yaml @PhanLe1010
  • Add another request for the rancher charts for the next patch release (1.5.1) @rebeccazzzz
• Rancher charts: verify the chart is able to install & upgrade - @khushboo-rancher
  • rancher/image-mirrors update @weizhe0422 (@PhanLe1010 )
    • docs(longhorn): Add longhorn 1.5.1 images rancher/image-mirror#418
  • rancher/charts 2.7 branches for rancher marketplace @weizhe0422 (@PhanLe1010)
    • dev-2.7: Add Longhorn 1.5.1 chart with dev-2.7 rancher/charts#2797

cc @longhorn/qa @longhorn/dev

[yangchiu]

Run security testing of container images for pre-GA milestones:

v1.5.1-rc1: 37 HIGH, 0 CRITICAL

[yangchiu]

Run e2e regression for pre-GA milestones (install, upgrade):

v1.5.1-rc1:
amd64: 4 failures
arm64: 3 failures
amd64 upgrade: 2 failures
arm64 upgrade: 3 failures

[innobead]

Run e2e regression for pre-GA milestones (install, upgrade):

v1.5.1-rc1: amd64: 4 failures arm64: 3 failures amd64 upgrade: 2 failures arm64 upgrade: 3 failures

@yangchiu are there any issues causing these failures in core tests?

cc @longhorn/qa

[yangchiu]

Run e2e regression for pre-GA milestones (install, upgrade):
v1.5.1-rc1: amd64: 4 failures arm64: 3 failures amd64 upgrade: 2 failures arm64 upgrade: 3 failures

@yangchiu are there any issues causing these failures in core tests?

cc @longhorn/qa

Need more time to investigate each failed test case one by one to know it's flaky or real issue.

[yangchiu]

Summarize all test cases failed in #6274 (comment) here:

• test_ha.py::test_inc_restoration_with_multiple_rebuild_and_expansion
  • [BUG] Test case test_inc_restoration_with_multiple_rebuild_and_expansion failed on SLES #6089
• test_ha.py::test_engine_image_not_fully_deployed_perform_dr_restoring_expanding_volume
  • [BUG] Volume gets stuck in an unknown state forever if created in an engine not fully deployed environment #6131
• test_ha.py::test_engine_image_not_fully_deployed_perform_replica_scheduling
  • [BUG] Volume gets stuck in an unknown state forever if created in an engine not fully deployed environment #6131
• test_node.py::test_node_eviction_multiple_volume
  • unknown
  • https://ci.longhorn.io/job/public/job/v1.5.x/job/v1.5.x-longhorn-tests-sles-arm64/56/testReport/junit/tests/test_node/test_node_eviction_multiple_volume/
  • rerun this test case for 50 times, unable to reproduce, the reproducibility is still unknown
• test_csi.py::test_csi_mount_volume_online_expansion
  • [BUG] The second time PVC online expansion always fail on SLES #6076
• test_infra.py::test_offline_node
  • probably flaky, it waits for k8s nodes down after stop the aws instances, but it timeouts
  • https://ci.longhorn.io/job/public/job/v1.5.x/job/v1.5.x-longhorn-tests-sles-amd64/62/testReport/junit/tests/test_infra/test_offline_node/

[roger-ryao]

• test_infra.py::test_offline_node

  • probably flaky, it waits for k8s nodes down after stop the aws instances, but it timeouts
  • https://ci.longhorn.io/job/public/job/v1.5.x/job/v1.5.x-longhorn-tests-sles-amd64/62/testReport/junit/tests/test_infra/test_offline_node/

Hi @yangchiu,
I had previously submitted an enhancement PR (longhorn/longhorn-tests#1197) for test_infra.py::test_offline_node, but it still occasionally fails.

[chriscchien]

Verify longhorn chart PR to ensure all artifacts are ready for GA (install, upgrade)

• longhorn/longhorn release: 1.5.1 #6348 (comment)
• longhorn/charts release: 1.5.1 charts#125 (comment)

[roger-ryao]

Run core testing (install, upgrade) for GA build from the previous patch and the last patch of the previous feature release (1.4.3, 1.5.0):

• amd64: 0 fail
• arm64: 1 fail
• [TEST] Fix flaky test case test_snapshot_prune_with_backing_image and test_snapshot_with_backing_image #6349
• amd64 upgrade from v1.4.3: 0 fail
• arm64 upgrade from v1.4.3: 0 fail
• amd64 upgrade from v1.5.0: 0 fail
• arm64 upgrade from v1.5.0: 1 fail
• [TEST] Fix flaky test case test_default_storage_class_syncup #6141

[innobead]

@rebeccazzzz @weizhe0422 @PhanLe1010 @khushboo-rancher @roger-ryao Please help with the post items.

[weizhe0422]

UPDATE: Two PRs have been created covering generating mirrors (rancher/image-mirror#418) and Rancher charts files (rancher/charts#2797), I will follow PR comments. Also, I update the task status.