Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

microk8s.reset doesn't clean up volumes #488

Closed
knkski opened this issue May 28, 2019 · 9 comments
Closed

microk8s.reset doesn't clean up volumes #488

knkski opened this issue May 28, 2019 · 9 comments
Labels
kind/feature New feature or request triage/good-first-issue Good for newcomers

Comments

@knkski
Copy link
Contributor

knkski commented May 28, 2019

I attempted to remove the microk8s snap, and it hung on Save data of snap "microk8s" in automatic snapshot set. After investigating a bit, it looks like microk8s.reset doesn't clean up volumes, and I had many of them left over:

$ sudo du -sh /var/snap/microk8s/common/
37G	/var/snap/microk8s/common/

With paths like this: /var/snap/microk8s/common/default-storage/kubeflow-database-a6dbbcb2-mariadb-0-pvc-a75a4c3b-6b61-11e9-bf3c-e86a6466b483/

After running sudo rm -rf /var/snap/microk8s/common/*, removing the microk8s snap worked great.
@ktsakalozos
Copy link
Member

@knkski this is somewhat expected. We are not removing persistent volumes unless you do a microk8s.disable storage and answer "Yes" to the respective question. The reasn for not cleaning up persistent volumes without confirmation is that you may have some data you may want to keep. Having said that, we may want to have a flag on the microk8s.reset to clean persistent volumes as well. What do you think?

@ktsakalozos ktsakalozos added kind/feature New feature or request triage/good-first-issue Good for newcomers labels May 29, 2019
@knkski
Copy link
Contributor Author

knkski commented May 29, 2019

@ktsakalozos: A flag on microk8s.reset sounds perfect. For comparison, juju has a --destroy-storage on kill-controller that does the same thing

@TomHellier
Copy link

I am facing this issue as well, I think it would be nice for a warning to show up here .e.g.

Save data of snap "microk8s" in automatic snapshot set #2
WARN: persistent data exists in /var/snap/microk8s/common/, run microk8s.disable storage to remove

@knkski
Copy link
Contributor Author

knkski commented Jun 5, 2019

@ktsakalozos: I tried disabling storage (after having just done a microk8s.reset), and got this:

$ microk8s.disable storage
Disabling default storage
storageclass.storage.k8s.io "microk8s-hostpath" deleted
Error from server (NotFound): error when deleting "/home/knkski/snap/microk8s/522/tmp/temp.storage.yaml": deployments.extensions "hostpath-provisioner" not found
Failed to disable storage

@ktsakalozos
Copy link
Member

Did you microk8s.disable storage after a microk8s.reset?

Could you do a microk8s.enable storage and then a microk8s.disable storage? I suspect the microk8s.reset removed the storage daemon resulting in the error you got.

@knkski
Copy link
Contributor Author

knkski commented Jun 5, 2019

@ktsakalozos: Looks like microk8s.enable storage, then microk8s.disable storage worked, thanks! Do you think it would make sense to open a new bug for adding in some output like this?

Failed to disable storage, is the storage daemon running? You may need to run `microk8s.enable storage` first.

@ktsakalozos
Copy link
Member

open a new bug for adding in some output

Yes, that would be a great entry level patch! Go ahead with creating an issue with the suggested improvement.

rzr added a commit to CrossStream/microk8s that referenced this issue Aug 27, 2019
@rzr
scripts: Add destroy-storage option to reset tool
cb7e95b 
This will help for cleaning  volume

- Reintroduce cluster_clean removed by @ktsakalozos (8173dce)

Relate-to: canonical#488
Change-Id: I04a13f060b1c4beda9dd5006c435e5b2be12c385
Signed-off-by: Philippe Coval <p.coval@samsung.com>
rzr added a commit to CrossStream/microk8s that referenced this issue Aug 27, 2019
@rzr
scripts: Add destroy-storage option to reset tool
3a15513 
This will help for cleaning  volume

- Reintroduce cluster_clean removed by @ktsakalozos (8173dce)

Relate-to: canonical#488
Change-Id: I04a13f060b1c4beda9dd5006c435e5b2be12c385
Signed-off-by: Philippe Coval <p.coval@samsung.com>
@rzr rzr mentioned this issue Aug 27, 2019
Merged
rzr added a commit to CrossStream/microk8s that referenced this issue Sep 3, 2019
@rzr
scripts: Add destroy-storage option to reset tool
c5f4619 
This will help for cleaning volumes

Relate-to: canonical#488
Change-Id: I04a13f060b1c4beda9dd5006c435e5b2be12c385
Signed-off-by: Philippe Coval <p.coval@samsung.com>
ktsakalozos pushed a commit that referenced this issue Sep 12, 2019
@rzr @ktsakalozos
scripts: Add destroy-storage option to reset tool (#616)
541f444 
This will help for cleaning volumes

Relate-to: #488
Change-Id: I04a13f060b1c4beda9dd5006c435e5b2be12c385
Signed-off-by: Philippe Coval <p.coval@samsung.com>
@rzr
Copy link
Contributor

rzr commented Oct 1, 2019

I guess this issue can be closed since release of v1.16.0 (920) ?

microk8s.reset --destroy-storage
du -csh /var/snap/microk8s/common/*
40K     /var/snap/microk8s/common/run
176M    /var/snap/microk8s/common/var
176M    total

Relate-to: #616

@ktsakalozos
Copy link
Member

Yes, thank you @rzr

ianbibby pushed a commit to ianbibby/microk8s that referenced this issue Jan 10, 2020
@rzr @ianbibby
scripts: Add destroy-storage option to reset tool (canonical#616)
8ee89c6 
This will help for cleaning volumes

Relate-to: canonical#488
Change-Id: I04a13f060b1c4beda9dd5006c435e5b2be12c385
Signed-off-by: Philippe Coval <p.coval@samsung.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/feature New feature or request triage/good-first-issue Good for newcomers
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rzr @TomHellier @ktsakalozos @knkski