Can not remove directory inside container created during image build

[pinkal-vansia]

Description
I am not able to remove non empty directory in docker container. Directory is created as part of image build.

Steps to reproduce the issue:

1. Dockerfile

FROM ubuntu
RUN mkdir -p /usr/local/bin/test
RUN echo "test" > /usr/local/bin/test/test.txt

2. Build Image

docker build -t deletetest .

3. Run container

docker run --rm -it deletetest bash

Describe the results you received:
try to delete directory

root@d284c6c5e4f6:/# cd /usr/local/bin
root@d284c6c5e4f6:/usr/local/bin# rm -rf test/
rm: cannot remove 'test/': Directory not empty

Additional information you deem important (e.g. issue happens only occasionally):

[centos@test]$ docker build -t testdelete .
Sending build context to Docker daemon 2.048 kB
Step 1/3 : FROM ubuntu
 ---> f49eec89601e
Step 2/3 : RUN mkdir -p /usr/local/bin/test
 ---> Running in 1483f6170161
 ---> 0a41a1f634ea
Removing intermediate container 1483f6170161
Step 3/3 : RUN echo "test" > /usr/local/bin/test/test.txt
 ---> Running in 4f27c36a0a05
 ---> 39019e424b60
Removing intermediate container 4f27c36a0a05
Successfully built 39019e424b60
[centos@test]$ docker run -it testdelete bash
root@d284c6c5e4f6:/# cd /usr/local/bin
root@d284c6c5e4f6:/usr/local/bin# ll
total 4
drwxr-xr-x.  3 root root   17 Feb 23 08:30 ./
drwxr-xr-x. 10 root root 4096 Jan 24 09:27 ../
drwxr-xr-x.  2 root root   21 Feb 23 08:30 test/
root@d284c6c5e4f6:/usr/local/bin# rm -rf test/
rm: cannot remove 'test/': Directory not empty
root@d284c6c5e4f6:/usr/local/bin# ll test/
ls: cannot access 'test/test.txt': No such file or directory
total 0
drwxr-xr-x. 1 root root 21 Feb 23 08:37 ./
drwxr-xr-x. 1 root root 17 Feb 23 08:30 ../
??????????? ? ?    ?     ?            ? test.txt

Output of docker version:

[centos@test]$ docker version
Client:
 Version:      1.13.0
 API version:  1.25
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:55:28 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.0
 API version:  1.25 (minimum version 1.12)
 Go version:   go1.7.3
 Git commit:   49bf474
 Built:        Tue Jan 17 09:55:28 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

[centos@test]$ docker info
Containers: 10
 Running: 0
 Paused: 0
 Stopped: 10
Images: 201
Server Version: 1.13.0
Storage Driver: overlay
 Backing Filesystem: xfs
 Supports d_type: false
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 03e5862ec0d8d3b3f750e19fca3ee367e13c090e
runc version: 2f7393a47307a16f8cee44a37b262e8b81021e3e
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 3.10.0-514.6.1.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 6.896 GiB
Name: <removed>
ID: BQID:S2LK:OB74:DLPS:ICCZ:I2AI:L4WX:EMNM:2OQP:O7NN:FQ6F:SJJA
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Working on M3 large EC2 instance

[mountkin]

To use overlayfs over xfs, you have to format the xfs filesystem with mkfs.xfs -n ftype=1 /path/to/dev
See https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/7.2_Release_Notes/technology-preview-file_systems.html

[thaJeztah]

Thanks @mountkin, yes that's the cause of this issue. Docker 1.13 adds a warning if the filesystem is not formatted correctly, but will refuse to start without ftype=1 in future; see https://github.com/docker/docker/blob/v1.13.1/docs/deprecated.md#backing-filesystem-without-d_type-support-for-overlayoverlay2 (and #27433)

The output of docker info shows if the backing filesystem has this option enabled here;

Supports d_type: false

I'm closing this issue, because this is a duplicate of #27358, but feel free to comment after I closed.