Cannot remove docker volume

[mindaugasrukas]

Output of docker version:

$ docker version
Client:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      darwin/amd64

Server:
 Version:      1.10.3
 API version:  1.22
 Go version:   go1.5.3
 Git commit:   20f81dd
 Built:        Thu Mar 10 21:49:11 2016
 OS/Arch:      linux/amd64

Output of docker info:

$ docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 47
Server Version: 1.10.3
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 295
 Dirperm1 Supported: true
Execution Driver: native-0.2
Logging Driver: json-file
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 4.1.19-boot2docker
Operating System: Boot2Docker 1.10.3 (TCL 6.4.1); master : 625117e - Thu Mar 10 22:09:02 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 996.1 MiB
Name: default
ID: XWWF:P22A:CMF2:PGQC:M4DA:2TEH:ZTCG:VIPU:DO2M:IMCC:Q5I3:QQ6T
Debug mode (server): true
 File Descriptors: 10
 Goroutines: 22
 System Time: 2016-04-06T12:05:22.62834683Z
 EventsListeners: 0
 Init SHA1: 
 Init Path: /usr/local/bin/docker
 Docker Root Dir: /mnt/sda1/var/lib/docker
Username: mindaugas
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox

Additional environment details (AWS, VirtualBox, physical, etc.):
VirtualBox Version 5.0.16 r105871

Steps to reproduce the issue:

1. Create files:
   docker-compose-1.yml:

app:
  container_name: app.container
  image: busybox
  volumes_from:
  - data1
  entrypoint: "top"
data1:
  container_name: data1.container
  build: .
  dockerfile: Dockerfile-1

docker-compose-2.yml:

app:
  volumes_from:
  - data2
data2:
  container_name: data2.container
  build: .
  dockerfile: Dockerfile-1

Dockerfile-1:

FROM busybox
ADD . /tmp/data
VOLUME ["/tmp/data"]
CMD ["true"]

1. Run containers:

docker-compose -f docker-compose-1.yml -f docker-compose-2.yml up -d

1. Remove containers:

docker rm -f -v $(docker ps -aq)

Describe the results you received:

$ docker rm -f -v $(docker ps -aq)
73ff8a6d5203
6883f51cd6fd
...

$ docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES

$ docker volume ls
DRIVER              VOLUME NAME
local               fc12bdcfce09a647bdd0015d41635154b86352447c07fc5104fc72a958be9538

$ docker volume rm fc12bdcfce09a647bdd0015d41635154b86352447c07fc5104fc72a958be9538
Error response from daemon: Conflict: remove fc12bdcfce09a647bdd0015d41635154b86352447c07fc5104fc72a958be9538: volume is in use - [ad009b5b5c088c4d91917f14b05b9f6f174badcb82c7f603734550b36c20a120]

$ docker rm ad009b5b5c088c4d91917f14b05b9f6f174badcb82c7f603734550b36c20a120
Failed to remove container (ad009b5b5c088c4d91917f14b05b9f6f174badcb82c7f603734550b36c20a120): Error response from daemon: No such container: ad009b5b5c088c4d91917f14b05b9f6f174badcb82c7f603734550b36c20a120

$ docker-machine ssh
Boot2Docker version 1.10.3, build master : 625117e - Thu Mar 10 22:09:02 UTC 2016
Docker version 1.10.3, build 20f81dd

docker@default:~$ sudo /etc/init.d/docker restart
cat: can't open '/var/run/docker.pid': No such file or directory
Need TLS certs for default,127.0.0.1,10.0.2.15,192.168.99.100
-------------------

docker@default:~$ exit

$ docker volume rm fc12bdcfce09a647bdd0015d41635154b86352447c07fc5104fc72a958be9538
fc12bdcfce09a647bdd0015d41635154b86352447c07fc5104fc72a958be9538

$ docker volume ls
DRIVER              VOLUME NAME

Describe the results you expected:
Successfully removed all docker volume

Additional information you deem important (e.g. issue happens only occasionally):
the same issue on my colleague computer (linux) where docker runs locally.

[thaJeztah]

@mindaugasrukas are you still seeing this issue on the current release?

[mindaugasrukas]

Do you mean on version v1.12.3? Need to check. @mariusgrigaitis please check it.

[mariusgrigaitis]

Still able to replicate it. Tried it on Docker for Mac. Version 1.12.3-beta30.1 (13946)

docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 32
Server Version: 1.12.3
Storage Driver: overlay2
 Backing Filesystem: extfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.30-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 7.785 GiB
Name: moby
ID: ZVWY:FDR5:3FAX:5XNK:ANS3:36ZJ:3PUT:2A3I:FAKG:5CST:4NMP:B4KV
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 22
 Goroutines: 29
 System Time: 2016-12-01T07:46:58.635991911Z
 EventsListeners: 1
No Proxy: *.local, 169.254/16
Username: mariusgrigaitis
Registry: https://index.docker.io/v1/
Experimental: true
Insecure Registries:
 127.0.0.0/8

 /tmp/test  docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
 /tmp/test  docker volume ls
DRIVER              VOLUME NAME
 /tmp/test  docker-compose -f docker-compose-1.yml -f docker-compose-2.yml up -d

Building data1
Step 1 : FROM busybox
latest: Pulling from library/busybox
56bec22e3559: Pull complete
Digest: sha256:29f5d56d12684887bdfa50dcd29fc31eea4aaf4ad3bec43daf19026a7ce69912
Status: Downloaded newer image for busybox:latest
 ---> e02e811dd08f
Step 2 : ADD . /tmp/data
 ---> b1e589dfbf83
Removing intermediate container f1923d38877f
Step 3 : VOLUME /tmp/data
 ---> Running in a82cf5c926cf
 ---> 8a84e08eba25
Removing intermediate container a82cf5c926cf
Step 4 : CMD true
 ---> Running in fe6376a637e1
 ---> 3b31afdf8709
Removing intermediate container fe6376a637e1
Successfully built 3b31afdf8709
WARNING: Image for service data1 was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Building data2
Step 1 : FROM busybox
 ---> e02e811dd08f
Step 2 : ADD . /tmp/data
 ---> Using cache
 ---> b1e589dfbf83
Step 3 : VOLUME /tmp/data
 ---> Using cache
 ---> 8a84e08eba25
Step 4 : CMD true
 ---> Using cache
 ---> 3b31afdf8709
Successfully built 3b31afdf8709
WARNING: Image for service data2 was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
Creating data1.container
Creating data2.container
Creating app.container
 /tmp/test  docker rm -f -v $(docker ps -aq)

391d7e0a24d3
2f73eafeac04
a7d1c4c52b06
 /tmp/test  docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
 /tmp/test  docker volume ls
DRIVER              VOLUME NAME
local               3edc4ee826d7b7563b16e35b5313396c8d001a0f46cc6f7b11bf6765888e5e3e
 /tmp/test  docker volume rm 3edc4ee826d7b7563b16e35b5313396c8d001a0f46cc6f7b11bf6765888e5e3e
Error response from daemon: Unable to remove volume, volume still in use: remove 3edc4ee826d7b7563b16e35b5313396c8d001a0f46cc6f7b11bf6765888e5e3e: volume is in use - [391d7e0a24d3491cbf085bdcf0d3d438cd411fffb8f2c563d9ea096e0928d273]

[yongtang]

This issue could be reproduced with the most recent master (1.14-dev).

The issue is that, the app.container will actually takes two volumes --volumes-from data1.container --volumes-from data2.container.

However, both volumes will have the same mount point /tmp/data. When app.container is started, one mount point (volume) will be overridden by the other mount point (volume). As a result, volume rm will not be able to dereference.

Without docker-compose, a similar behavior could be emulated with:

$ cat Dockerfile
FROM busybox
VOLUME ["/tmp/data"]
$ docker build -t vimage .
$ docker run --name=data1 vimage true
$ docker run --name=data2 vimage true
$ docker run --name=app --volumes-from=data1 --volumes-from=data2 -d busybox top
$ docker rm -f -v $(docker ps -aq)
$ docker volume ls
$ docker volume rm ...

I think it makes sense to return an error in case duplicate mount point exist.

Created a PR #29563 for that.

/cc @cpuguy83