New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix uninstall failure when Btrfs is used as the storage driver #296

Open
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
2 participants
@satoru-takeuchi
Copy link

satoru-takeuchi commented Jan 28, 2019

snap remove microk8s fails as follows if the storage driver is Btrfs.

$ sudo snap remove microk8s
error: cannot perform the following tasks:
- Remove data for snap "microk8s" (383) (remove /var/snap/microk8s/common/var/lib/docker/btrfs/subvolumes/<id>: operation not permitted)

It's because Btrfs subvolumes can't be deleted by rmdir() syscall.
Although this problem disappear if the linux kernel version is
v4.18 or more, it's better to fix it for older kernel users.

Satoru Takeuchi
fix uninstall failure when Btrfs is used as the storage driver
`snap remove microk8s` fails as follows if the storage driver is Btrfs.

```
$ sudo snap remove microk8s
error: cannot perform the following tasks:
- Remove data for snap "microk8s" (383) (remove /var/snap/microk8s/common/var/lib/docker/btrfs/subvolumes/<id>: operation not permitted)
```

It's because Btrfs subvolumes can't be deleted by rmdir() syscall.
Although this problem disappear if the linux kernel version is
v4.18 or more, it's better to fix it for older kernel users.
@ktsakalozos

This comment has been minimized.

Copy link
Collaborator

ktsakalozos commented Jan 31, 2019

Hi @satoru-takeuchi

Can you please show me how you setup MicroK8s with btrfs so I can reproduce your setup? We try to keep MicroK8s self-contained meaning that we want binaries such as btrfs to be shipped with the snap.

Thank you for the PR

@satoru-takeuchi

This comment has been minimized.

Copy link
Author

satoru-takeuchi commented Jan 31, 2019

Thank you for your response, @ktsakalozos

I setup microk8s just by the following command with btrfs root filesystem.

$ mount | grep btrfs
/dev/sda1 on / type btrfs (rw,relatime,ssd,space_cache,subvolid=257,subvol=/@) 
/dev/sda1 on /home type btrfs (rw,relatime,ssd,space_cache,subvolid=258,subvol=/@home)
$ sudo snap install microk8s --classic
microk8s v1.13.2 from Canonical✓ installed
$ microk8s.docker info | grep btrfs
Storage Driver: btrfs
WARNING: No swap limit support

However, I guess you're not a Btrfs user. To make Btrfs as a docker' storage driver
from non-btrfs users, the following operations are needed.

https://docs.docker.com/storage/storagedriver/btrfs-driver/

As for microk8s, we should replace "/var/lib/docker" with "/var/snap/microk8s/common/var/lib/docker/".

@ktsakalozos

This comment has been minimized.

Copy link
Collaborator

ktsakalozos commented Jan 31, 2019

Indeed I am not on btrfs.

In https://github.com/ubuntu/microk8s/pull/296/files#diff-5e0254f4e4d4ae7c6097bbc7feda139aR15 how do we know btrfs is on our path? If we were to use the full path how would we know it it the same across all distros. Would it work if we put btrfs in the snap? We need to add it in this list: https://github.com/ubuntu/microk8s/blob/master/snapcraft.yaml#L114

@satoru-takeuchi

This comment has been minimized.

Copy link
Author

satoru-takeuchi commented Jan 31, 2019

In https://github.com/ubuntu/microk8s/pull/296/files#diff-5e0254f4e4d4ae7c6097bbc7feda139aR15 how
do we know btrfs is on our path?

I didn't consider that, sorry.

If we were to use the full path how would we know it it the same across all distros.

I see. So I should use just "btrfs" rather than full path, .../btrfs).

Would it work if we put btrfs in the snap? We need to add it in this list:
https://github.com/ubuntu/microk8s/blob/master/snapcraft.yaml#L114

I have one question about it.

How can I install btrfs command by setting stage-packages by distribution
independent way?

For example, btrfs commandis included in "btrfs-tools" package on Ubuntu16.04 and
"btrfs-progs" package on Ubuntu 18.04. However, as I read on the snapcraft's spec,
packages written in stage-packages are installed by distribution's package manager.
If my understanding is correct, when I add "btrfs-tools", it doesn't work on Ubuntu
18.04 and if I add "btrfs-progs", it doen't work on Ubuntu 16.04.

https://docs.snapcraft.io/snapcraft-yaml-reference/4276

@ktsakalozos

This comment has been minimized.

Copy link
Collaborator

ktsakalozos commented Jan 31, 2019

The MicroK8s snap is based on Ubuntu 16.04 (we may move to 18.04 at some point in the future). This means that is you add "btrfs-tools" in the staging packages btrfs will be in the snap under ${SNAP}. So in your patch you can access the btrfs client using the full path ${SNAP}/...path inside the snap.../btrfs. We should also make sure docker is indeed using the btrfs tools shipped with the snap. We do use the right paths in https://github.com/ubuntu/microk8s/blob/master/microk8s-resources/wrappers/run-docker-with-args but lets double check. Also an FYI, we will be swapping dockerd with containerd in future releases.

@satoru-takeuchi

This comment has been minimized.

Copy link
Author

satoru-takeuchi commented Jan 31, 2019

Thank you for giving me the detailed explanation. I'll fix my PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment