Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[backport v1.3] [BUG] upgrade will stuck without any error meessage if the checksum is invalid #5543

Closed
harvesterhci-io-github-bot opened this issue Apr 10, 2024 · 6 comments
Assignees
Labels
area/upgrade kind/bug Issues that are defects reported by users or that we know have reached a real release need-reprioritize not-require/test-plan Skip to create a e2e automation test issue reproduce/always Reproducible 100% of the time severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact)
Milestone

Comments

@harvesterhci-io-github-bot

backport the issue #5480

@harvesterhci-io-github-bot harvesterhci-io-github-bot added area/upgrade kind/bug Issues that are defects reported by users or that we know have reached a real release need-reprioritize not-require/test-plan Skip to create a e2e automation test issue reproduce/always Reproducible 100% of the time severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact) labels Apr 10, 2024
@harvesterhci-io-github-bot harvesterhci-io-github-bot added this to the v1.3.1 milestone Apr 10, 2024
@harvesterhci-io-github-bot
Copy link
Author

harvesterhci-io-github-bot commented Apr 30, 2024

Pre Ready-For-Testing Checklist

  • If labeled: require/HEP Has the Harvester Enhancement Proposal PR submitted?
    The HEP PR is at:

  • Where is the reproduce steps/test steps documented?
    The reproduce steps/test steps are at:

    1. Test version with incorrect isoChecksum
      After applying the following version.yaml, the console should return error message like admission webhook "validator.harvesterhci.io" denied the request: invalid isoChecksum asdf since asdf isn't in sha512 format.
      apiVersion: harvesterhci.io/v1beta1
      kind: Version
      metadata:
        name: master
        namespace: harvester-system
      spec:
        isoURL: http://192.168.100.1:8000/harvester-v1.3.0-amd64.iso
        isoChecksum: asdf
    2. Test version with correct isoChecksum and run upgrade
    • Apply the correct version.yaml to K8s
    • Click the upgrade button in harvester dashboard
    • Everything should works fine
    1. Test vm image
    • Go to harvester UI images and create image with mismatch SHA512 checksum
    • The vm image should download 4 times and then failed with error message like Failed to process sync file: the expected checksum ... doesn't match the file actual checksum ...
  • Is there a workaround for the issue? If so, where is it documented?
    The workaround is at:

  • Have the backend code been merged (harvester, harvester-installer, etc) (including backport-needed/*)?
    The PR is at: Fix upgrade stuck when using incorrect isoChecksum in version CR (backport #5498) #5700

    • Does the PR include the explanation for the fix or the feature?

    • Does the PR include deployment change (YAML/Chart)? If so, where are the PRs for both YAML file and Chart?
      The PR for the YAML change is at:
      The PR for the chart change is at:

  • If labeled: area/ui Has the UI issue filed or ready to be merged?
    The UI issue/PR is at:

  • If labeled: require/doc, require/knowledge-base Has the necessary document PR submitted or merged?
    The documentation/KB PR is at:

  • If NOT labeled: not-require/test-plan Has the e2e test plan been merged? Have QAs agreed on the automation test case? If only test case skeleton w/o implementation, have you created an implementation issue?

    • The automation skeleton PR is at:
    • The automation test case PR is at:
  • If the fix introduces the code for backward compatibility Has a separate issue been filed with the label release/obsolete-compatibility?
    The compatibility issue is filed at:

@irishgordo irishgordo self-assigned this May 16, 2024
@irishgordo
Copy link

Validation Failed

It seems on v1.3-head (05/17):

rancher@harvester-node-0:~> sudo cat /etc/os-release 
NAME="SLE Micro"
VERSION="5.4"
VERSION_ID="5.4"
PRETTY_NAME="Harvester v1.3.0"
ID="sle-micro-rancher"
ID_LIKE="suse"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:suse:sle-micro-rancher:5.4"
VARIANT="Harvester"
VARIANT_ID="Harvester-v1.3-20240307"
GRUB_ENTRY_NAME="Harvester v1.3.0"

That this still is failing.
On an upgrade path, single node from v1.3.0 -> v1.3-head.
With an upgrade crd:

apiVersion: harvesterhci.io/v1beta1
kind: Version
metadata:
  name: v13-head
  namespace: harvester-system
spec:
  isoURL: http://192.168.9.163:5000/harvester-v1.3-amd64.iso
  isoChecksum: asdf

Applied.
As it does still just "hang" without any message about the upgrade being errored due to the sha512sum checksum of the iso being incorrect.

Screenshot from 2024-05-17 15-28-17
Screenshot from 2024-05-17 15-27-05
Screenshot from 2024-05-17 15-26-23
Screenshot from 2024-05-17 15-21-14

@irishgordo irishgordo removed their assignment May 17, 2024
@brandboat
Copy link
Contributor

gentle ping @irishgordo, though #5700 was merged in branch v1.3, it was not included in v1.3.0 release. Perhaps you can try upgrade from v1.3-head to master

@irishgordo irishgordo self-assigned this May 20, 2024
@irishgordo
Copy link

@brandboat - thanks for that mention 😄 👍
Testing from v1.3-head ( Version: master-b6185038-head ) to Master I was able to validate this.
Screenshot from 2024-05-20 11-24-53

I'll go ahead and close this out 👍 😄

@irishgordo
Copy link

Re-Opening, need to revalidate due to:
harvester/harvester-installer#737

@irishgordo
Copy link

This looks great - thanks again @brandboat :

╭─mike at suse-workstation-team-harvester in /media/mike/ubuntu-nfs-share/isos
╰─○ kubectl create -f master-head-version.yaml 
Error from server (BadRequest): error when creating "master-head-version.yaml": admission webhook "validator.harvesterhci.io" denied the request: invalid isoChecksum dogs-are-great
╭─mike at suse-workstation-team-harvester in /media/mike/ubuntu-nfs-share/isos
╰─○ 

On Version: v1.3-274e20ac-head -> master-head

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/upgrade kind/bug Issues that are defects reported by users or that we know have reached a real release need-reprioritize not-require/test-plan Skip to create a e2e automation test issue reproduce/always Reproducible 100% of the time severity/2 Function working but has a major issue w/o workaround (a major incident with significant impact)
Projects
None yet
Development

No branches or pull requests

3 participants