Trying to set up ceph in docker with -s btrfs. The container cannot be removed:
% docker rm 43f1e8078e9b
Error: container_delete: Cannot destroy container 43f1e8078e9b: Driver btrfs failed to remove root filesystem 43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00: Failed to destroy btrfs snapshot: directory not empty
2014/02/12 18:39:10 Error: failed to remove one or more containers
Well, that's true, there are some snapshots:
% sudo btrfs subvol list -ao /var/lib/docker|grep 43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00
ID 686 gen 8878 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00-init
ID 687 gen 8958 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00
ID 711 gen 8949 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-0/current
ID 719 gen 8950 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-1/current
ID 726 gen 8934 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-1/snap_2332
ID 727 gen 8935 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-0/snap_2909
ID 728 gen 8949 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-0/snap_2911
ID 729 gen 8950 top level 5 path btrfs/subvolumes/43f1e8078e9b400863efc8fb650dcf37bf9a9bdb30e6107c65fa70eeb3f88d00/var/lib/ceph/osd/ceph-1/snap_2334
I guess the correct behaviour here is to remove those on container removal. Non-privileged container shouldn't break the docker state.
@farcaller Creating subvolumes in the container isn't something we support in any way at this point.
Yes, I'm not sure there is anything that we can do on the dockerside when there are live subvolumes inside the container. You should probably have a script that cleans up the volumes when the container is stopped. Maybe an email on the mailing list is a better place for this?
Well yes, apparently this is more an issue of btrs, given the fact that creating subvol doesn't require any privileges (as opposed to removing it).
I wouldn't say this is a btrfs issue. In my eyes there exists no hook to execute something when docker destroy or stop the container.
Docker should removes everything related or created by the container like subvolumes created inside of the container filesystem.