Overlay allows deleting directories that aren't empty

[ibuildthecloud]

I know this is not a Docker bug, this is in the kernel itself. I'm only filing here as I expect it will gain the attention of somebody who know where/how to get this fixed.

Test case

Dockerfile

FROM ubuntu
RUN mkdir /empty

docker build -t test .

docker run -i test bash << "EOF"
rmdir /empty
mkdir /empty
touch /empty/a
rmdir /empty && echo "I'm a bug"
EOF

The last line should fail but it doesn't.

docker version; docker info
Client version: 1.6.0
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): 4749651
OS/Arch (client): linux/amd64
Server version: 1.6.0
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): 4749651
OS/Arch (server): linux/amd64
Containers: 9
Images: 808
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Kernel Version: 4.0.1-040001-generic
Operating System: Ubuntu 14.10
CPUs: 4
Total Memory: 7.504 GiB
Name: inotmac
ID: FPSO:Z7SO:BZJ3:IXH4:DNKV:UPZQ:KLP2:IRXW:5CBY:PBDM:HAKO:UOC7
Username: ibuildthecloud
Registry: [https://index.docker.io/v1/]

[ibuildthecloud]

@GordonTheTurtle is this a generic message or did you determine that my issue lacks info? Do I have to follow your template?

#ECANNOTCOMPUTE

[jessfraz]

Lol you are fine

[jessfraz]

There seem to be a few weird overlay bugs wrt Unix sockets, installing pip lol etc
ping @vbatts

[vbatts]

@ibuildthecloud interesting that fails on ubuntu's 4.0.1
The same steps fails on fedora20

vbatts@valse ~/tmp (master *) $ docker build -t test .
Sending build context to Docker daemon 2.048 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
latest: Pulling from ubuntu
e9e06b06e14c: Pull complete 
a82efea989f9: Pull complete 
37bea4ee0c81: Pull complete 
07f8e8c5e660: Already exists 
ubuntu:latest: The image you are pulling has been verified. Important: image verification is a tech preview feature and should not be relied on to provide security.
Digest: sha256:8126991394342c2775a9ba4a843869112da8156037451fc424454db43c25d8b0
Status: Downloaded newer image for ubuntu:latest
 ---> 07f8e8c5e660
Step 1 : RUN mkdir /empty
 ---> Running in ea2b4971b9ee
 ---> 3adf0ef44a5b
Removing intermediate container ea2b4971b9ee
Successfully built 3adf0ef44a5b
vbatts@valse ~/tmp (master *) $ docker run -i test bash << "EOF"                                                                                                                                                     
> rmdir /empty
> mkdir /empty
> touch /empty/a
> rmdir /empty && echo "I'm a bug"
> EOF
rmdir: failed to remove '/empty': Directory not empty
vbatts@valse ~/tmp (master *) $ docker info
Containers: 2
Images: 5
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Kernel Version: 3.19.5-100.fc20.x86_64
Operating System: Fedora 20 (Heisenbug)
CPUs: 4
Total Memory: 11.43 GiB
Name: valse.usersys.redhat.com
ID: RSJF:6G2K:WRHU:VG4Y:OUFZ:EZDT:UZMQ:5EGF:YGMA:6ADA:BW5B:VN5H
Username: vbatts
Registry: [https://index.docker.io/v1/]

[vbatts]

rawr. i just tested again on 4.0.2-300.fc22.x86_64 and the bug is exposed. so, regression in the kernel.

[vbatts]

further, this now bunk container can not be restarted.

vbatts@noyee ~ (master) $ docker start 3                                                                                                                                  
Error response from daemon: Cannot start container 3: Error getting container 37b677109f2e7da10a9c3a05b24b2a6e0ac8fa972ce2bba7f093aca65da2aaa7 from driver overlay: error creating overlay mount to /var/lib/docker/overlay/37b677109f2e7da10a9c3a05b24b2a6e0ac8fa972ce2bba7f093aca65da2aaa7/merged: file exists
FATA[0000] Error: failed to start one or more containers 

I've just boiled this down to a reproduction outside docker:

base=$(mktemp -d)
echo ${base}

## make an overlay fs setup
mkdir -p ${base}/{upper,work,merged,mnt,lower}

## make an empty directory in the base layer
mkdir ${base}/lower/empty

## mount the base layer
sudo mount \
        -t overlay \
        -o rw,lowerdir=${base}/lower/,workdir=${base}/work/,upperdir=${base}/upper/ \
        ${base}/merged/ ${base}/mnt/

## create a whiteout
rmdir ${base}/mnt/empty

## add the dir back (on top of the whiteout)
mkdir -p ${base}/mnt/empty
touch ${base}/mnt/empty/file.txt

## now rm the non-empty directory
rmdir ${base}/mnt/empty && echo "bug here!"

## cleanup
sudo umount ${base}/mnt
sudo find ${base}

[vbatts]

starting with reporting it against fedora's kernel, https://bugzilla.redhat.com/show_bug.cgi?id=1220915

[vbatts]

upstream fix done and added to the 4.0 stable tree
torvalds/linux@d377c5e

[calavera]

I can confirm that this has already been fixed in upstream:

d@test:~$ sudo docker run -i test ash << "EOF"
rmdir /empty
mkdir /empty
touch /empty/a
rmdir /empty && echo "I'm a bug"
EOF

rmdir: '/empty': Directory not empty
d@test:~$ sudo docker info
Containers: 1
Images: 2
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.2-040102-generic
Operating System: Ubuntu 14.04.2 LTS
CPUs: 1
Total Memory: 489 MiB
Name: test
ID: 6WL7:6M3T:ER6K:QBLJ:7DL4:GKO2:4L5N:ZGQD:FRN7:O3S6:D76A:M5J5

Closing this issue. 🎉

[vbatts]

Great. Another one bites the dust.

/me dances
On Jul 16, 2015 20:23, "David Calavera" notifications@github.com wrote:

Closed #13108 #13108.

—
Reply to this email directly or view it on GitHub
#13108 (comment).