[rootless] driver "btrfs" failed to remove root filesystem: Failed to destroy btrfs snapshot /home/<USER>/.local/share/docker/btrfs/subvolumes for <ID>: operation not permitted

[AkihiroSuda]

Workaround

Create the ~/.config/docker/daemon.json with the following content and run systemctl --user restart docker

{"storage-driver": "fuse-overlayfs"}

---

Description

btrfs driver is not really expected to be supported for rootless, but it is automatically chosen by default when the host filesystem is btrfs (e.g. on Fedora 33 Workstation), and it does not work actually.

Steps to reproduce the issue:

1. Install Fedora 33 Workstation (aka "Desktop Edition")
2. Install Docker 20.10
3. Disable SELInux, at least for iptables (Rootless mode doesn't start on Fedora 32 with SELinux enabled (but works on CentOS 8.2): "can't open lock file /run/xtables.lock: Permission denied" #41230): sudo dnf install -y policycoreutils-python-utils && sudo semanage permissive -a iptables_t
4. docker-rootless-setuptool.sh install
5. export DOCKER_HOST=$XDG_RUNTIME_DIR/docker.sock
6. Make sure docker info shows btrfs as the storage driver
7. docker run --rm busybox true

Describe the results you received:

$ docker run --rm busybox true
ERRO[0000] Error waiting for container: container 17f95f5c7b026d9af2367b3676382ec1cc9c7c36e61e93ade87c30f86ce6ace1: driver "btrfs" failed to remove root filesystem: Failed to destroy btrfs snapshot /home/suda/.local/share/docker/btrfs/subvolumes for a9644718eaa311671fddd7440b275f97ad3642e21397cc7bd02244d5c9b0e81d: operation not permitted 

Describe the results you expected:
It should work with btrfs, or it should chose fuse-overlayfs, not btrfs.
The latter one is my expected behavior, but the former one might be more ideal.

Additional information you deem important (e.g. issue happens only occasionally):
Fedora 33 Server users are unlikely to be affected because it does not use btrfs by default

Output of docker version:

$ docker version
Client: Docker Engine - Community
 Version:           20.10.0
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        7287ab3
 Built:             Tue Dec  8 19:00:39 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.0
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       eeddea2
  Built:            Tue Dec  8 18:58:12 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker info:

$ $ docker info
Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.4.2-docker)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 2
 Server Version: 20.10.0
 Storage Driver: btrfs
  Build Version: Btrfs v5.9 
  Library Version: 102
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc version: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
  rootless
  cgroupns
 Kernel Version: 5.9.11-200.fc33.x86_64
 Operating System: Fedora 33 (Workstation Edition)
 OSType: linux
 Architecture: x86_64
 CPUs: 2
 Total Memory: 3.809GiB
 Name: localhost.localdomain
 ID: WMPE:OZUM:S6ZX:YRMZ:J2LY:J5DJ:7QNJ:IPJ2:QAMV:TE4J:W5U2:AWSY
 Docker Root Dir: /home/suda/.local/share/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No kernel memory TCP limit support
WARNING: No oom kill disable support
WARNING: No cpuset support
WARNING: Support for cgroup v2 is experimental

[AkihiroSuda]

user_subvol_rm_allowed mount option may work (https://twitter.com/kazenoasagi/status/1332668195107344385)

[AkihiroSuda]

Being fixed in #42203