Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Syncer container crashes when full sync tries to delete CNS volumes #2347

Closed
skogta opened this issue Apr 19, 2023 · 2 comments · Fixed by #2348
Closed

Syncer container crashes when full sync tries to delete CNS volumes #2347

skogta opened this issue Apr 19, 2023 · 2 comments · Fixed by #2348
Assignees
@skogta
Labels
kind/bug Categorizes issue or PR as related to a bug. release-3.0.1-candidate

Comments

@skogta
Copy link
Contributor

skogta commented Apr 19, 2023 •
edited
Loading

Is this a BUG REPORT or FEATURE REQUEST?:
BUG REPORT

/kind bug

What happened:
During full sync if CSI deletes volumes with no corresponding PV in the cluster, we run into nil pointer exception as volumeMigrationService is not initalised.
See this line in full sync code. We try to delete migration CR even if there are no in-tree volumes in the setup. Right now, we are only initialising volumeMigrationService when there is an in-tree volume in the setup.

What you expected to happen:
Volumes on CNS for which there is no PV on the cluster should have been deleted. While the volumes did get deleted, the syncer container also crashed with nil pointer exception.

How to reproduce it (as minimally and precisely as possible):

  1. Create a CNS volume on VC.
  2. Wait for full sync to delete it in 2 full sync cycles.

Anything else we need to know?:
No

Environment:

  • csi-vsphere version: 3.0
  • vsphere-cloud-controller-manager version:
  • Kubernetes version: 1.26
  • vSphere version: vSphere Client version 8.0.1.10000
  • Kernel (e.g. uname -a): Linux k8s-control-806-1680773753 5.4.0-66-generic #74-Ubuntu SMP Wed Jan 27 22:54:38 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
@k8s-ci-robot k8s-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Apr 19, 2023
@skogta
Copy link
Contributor Author

skogta commented Apr 19, 2023

/assign skogta

skogta added a commit to skogta/vsphere-csi-driver that referenced this issue Apr 19, 2023
@skogta
Fix issue kubernetes-sigs#2347 bu initialising volume migration servi…
b6efe7d 
…ce if it is a single VC deployment.
@skogta skogta mentioned this issue Apr 19, 2023
Merged
@skogta
Copy link
Contributor Author

skogta commented Apr 19, 2023

Fix: #2348

k8s-ci-robot pushed a commit that referenced this issue Apr 19, 2023
@skogta
Fix issue #2347 bu initialising volume migration service if it is a s…
68bdbb4 
…ingle VC deployment. (#2348)
skogta added a commit to skogta/vsphere-csi-driver that referenced this issue Apr 20, 2023
@skogta
Cherry-pick PR 2348
871fd99 
Fix issue kubernetes-sigs#2347 bu initialising volume migration service if it is a single VC deployment. (kubernetes-sigs#2348)
k8s-ci-robot pushed a commit that referenced this issue Apr 20, 2023
@skogta
Cherry pick PR 2348 and 2327 (#2349)
2198f0d 
* Cherry-pick PR 2327
Delete stale volume Info CRs only after it is validated for 2 full sync cycles. (#2327)

* Cherry-pick PR 2348
Fix issue #2347 bu initialising volume migration service if it is a single VC deployment. (#2348)
rajguptavm pushed a commit to rajguptavm/vsphere-csi-driver that referenced this issue Jul 20, 2023
@skogta @rajguptavm
Fix issue kubernetes-sigs#2347 bu initialising volume migration servi…
fe62ba1 
…ce if it is a single VC deployment. (kubernetes-sigs#2348)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@skogta skogta

Labels
kind/bug Categorizes issue or PR as related to a bug. release-3.0.1-candidate
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Initialise volume migration service for full sync in single VC setup skogta/vsphere-csi-driver
3 participants
@k8s-ci-robot @divyenpatel @skogta