docker rm -f reports 'Driver devicemapper failed to remove root filesystem' #795

grahamwhaley · 2017-04-05T16:03:08Z

I got the following error (when trying to run https://github.com/01org/cc-oci-runtime/blob/master/tests/metrics/density/docker_memory_usage.sh.in under https://github.com/01org/ciao/tree/master/testutil/ciao-down (which runs Ubuntu under a VM)):
Error response from daemon: Driver devicemapper failed to remove root filesystem 15d6062951f42e2a00206847c3846a7dbec38e31341d5640344a328515647c71: Device is Busy

After some digging I managed to re-create the issue by running:

 sudo docker run --name fred -tid busybox sh
 sudo docker run --name fred -tid busybox sh
 sudo docker rm -f fred john

Interesting points:

it appears you have to run docker as root to make this happen
you have to try to delete more than one container in a single command (individual docker rm -f calls work fine)
if you switch the storage driver on docker to overlay (adding --storage-driver=overlay to the systemd file), then the problem does not manifest
I tried this with runc, and it seemed to work

The text was updated successfully, but these errors were encountered:

grahamwhaley · 2017-04-05T16:03:51Z

@devimc fyi, don't know if you want to take this one or not

grahamwhaley · 2017-04-05T16:57:14Z

Just to add some more info. re-confirmed with a clean ciao-down image, so that is using:

Docker version 1.12.1, build 23cf638
cc-oci-runtime version: 2.1.2 commit: 1778df4

devimc · 2017-04-27T13:41:48Z

@grahamwhaley does this issue only occurs in ciao-down? if so could you report it to ciao-down maintainers ?

grahamwhaley · 2017-04-27T14:42:02Z

@devimc good question - I dunno - but I should be in a position to try it out I think. I'll give it a try and get back here...

grahamwhaley · 2017-04-27T14:51:10Z

@devimc And let me set my Clear Linux box up for devicemapper (if I can), as it is using overlay, so of course that worked....

grahamwhaley · 2017-04-27T15:52:27Z

H @devimc Looks like I reproduced this on Clear Linux 14740 with cc-oci-runtime v2.1.1 by setting the --storage-driver=devicemapper in the systemd docker file (away from the Clear Linux default of overlay).

devimc · 2017-04-27T16:31:29Z

@grahamwhaley thanks, I'm going to dig on this issue

grahamwhaley · 2017-05-03T17:30:16Z

Query was 'does this happen with runc and devicemapper' - I will check...

grahamwhaley · 2017-05-04T19:05:14Z

Update. Tested with Docker 1.2.1, CC 2.1.4 where the problem can be seen with --runtime=cor, and it does not happen with --runtime=runc.

GabyCT · 2017-05-05T20:25:53Z

@grahamwhaley and @jcvenegas , I also noticed this issue
Error response from daemon: Driver devicemapper failed to remove root filesystem ba63c5a01d9aac3e4b9371a81455dd041ae6fb2763f66e7bf230ced30a31ae4e: failed to remove device 85bfa90b9e766e6c8b6f0892fa03a45d4e233c166438718e6df23bbb6d29e0b8: Device is Busy

the configuration that I have is the following
cc-oci-runtime version: 2.1.5
spec version: 1.0.0-rc1
commit: 1a25508
Docker version 17.04.0-ce, build 4845c56
Centos 7.0

That happen when I am trying to remove a container that has the status of exited
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b924d366f2d6 gabyct/nuttcp "bash -c '/root/nu..." 5 minutes ago Exited (1) 4 minutes ago nuttcp-client

GabyCT · 2017-05-05T20:29:55Z

May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.188935544-04:00" level=debug msg="devmapper: DeleteDevice START(hash=a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9 syncDelete=false)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.188954805-04:00" level=debug msg="devmapper: issueDiscard START(device: a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)."
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.188962788-04:00" level=debug msg="devmapper: activateDeviceIfNeeded(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.262399052-04:00" level=debug msg="devmapper: issueDiscard END(device: a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)."
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.262440787-04:00" level=debug msg="devmapper: deactivateDevice START(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.262506815-04:00" level=debug msg="devmapper: removeDevice START(docker-253:0-1837118-a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.283833555-04:00" level=debug msg="devmapper: removeDevice END(docker-253:0-1837118-a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.283857748-04:00" level=debug msg="devmapper: deactivateDevice END(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.288824217-04:00" level=debug msg="devmapper: unregisterDevice(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.290739633-04:00" level=debug msg="devmapper: DeleteDevice END(hash=a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9 syncDelete=false)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.290884836-04:00" level=debug msg="devmapper: DeleteDevice START(hash=a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init syncDelete=false)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.290902072-04:00" level=debug msg="devmapper: issueDiscard START(device: a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)."
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.290913320-04:00" level=debug msg="devmapper: activateDeviceIfNeeded(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.365751216-04:00" level=debug msg="devmapper: issueDiscard END(device: a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)."
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.365789624-04:00" level=debug msg="devmapper: deactivateDevice START(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.365889762-04:00" level=debug msg="devmapper: removeDevice START(docker-253:0-1837118-a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.389828431-04:00" level=debug msg="devmapper: removeDevice END(docker-253:0-1837118-a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.389866955-04:00" level=debug msg="devmapper: deactivateDevice END(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.396662825-04:00" level=debug msg="devmapper: unregisterDevice(a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init)"
May 05 08:23:38 localhost.localdomain dockerd[8666]: time="2017-05-05T08:23:38.398666828-04:00" level=debug msg="devmapper: DeleteDevice END(hash=a5529ebd17e292dfb629be5e01c25893e81c7ef35084772cba457a1c0e8062d9-init syncDelete=false)"
May 05 08:24:23 localhost.localdomain dockerd[8666]: time="2017-05-05T08:24:23.957736061-04:00" level=debug msg="clean 2 unused exec commands"
May 05 08:24:52 localhost.localdomain dockerd[8666]: time="2017-05-05T08:24:52.818157529-04:00" level=debug msg="Calling GET /_ping"

GabyCT · 2017-05-08T15:42:05Z

@jcvenegas even using the overlay option in centos 7, the error continues

jcvenegas · 2017-05-08T16:28:52Z

@GabyCT Still same message : "Error response from daemon: Driver devicemapper failed to remove root filesystem ID: Device is Busy" ?

GabyCT · 2017-05-08T16:29:48Z

@jcvenegas yes, the same error

jcvenegas · 2017-05-08T16:33:47Z

@GabyCT thanks for confirm , would not expect to see device-mapper issues with overlay. I'll take a look

GabyCT · 2017-05-08T22:07:45Z

@jcvenegas should we increase to P1 this bug? currently this is affecting pnp network measurements

GabyCT · 2017-05-08T22:12:14Z

by the way, I tested using runc and the problem is not there even using device mapper or overlay

jcvenegas · 2017-05-09T13:52:30Z

@GabyCT if this is blocker for the CI yes, or skip the new test that are triggering the issue

GabyCT · 2017-05-09T14:21:02Z

@jcvenegas the issue is that is not a particular test, the problem is present when we are running all the tests

jcvenegas · 2017-05-09T16:04:42Z

@GabyCT what is the last known version that worked for CentOS ?

jcvenegas · 2017-05-09T16:14:50Z

I tried to reproduce the issue something:

I could not reproduce it using cc-oci-runtime.sh ( probably a race condition ? )
I only can reproduce the issue the first time I run docker ( /var/lib/docker clean )

jcvenegas · 2017-05-17T14:49:24Z

@grahamwhaley @GabyCT I can reproduce the steps from @grahamwhaley description only with docker 1.12 using docker 17.05 and cc-oci-runtiem 2.1.8 I cannot reproduce the issue anymore.

grahamwhaley · 2017-05-30T17:59:04Z

@jcvenegas OK. I normally see this when running the runtime density metrics tests as they 'rm' a list of containers all in one go, and have to be run as root. Sooo, I'm happy to leave this here open for a bit and see if this either does or does not re-occur when we start running those tests on 17.05 and above.
Failing that, the real path to close this would be a bisect on docker versions or a trawl of the docker history/changes I suspect - not enticing...

** Changes - Devicemapper storage - version: Update container image to 16020 - obs: Update packaging to include kernel config changes for xfs - cc-oci-runtime.sh: Enable debug by default - configure: Add warning when kernel or image is not found - proxy: Better error message for too long socket path - tests: metrics: formalise the metrics reporting methods - build: Adds option to install kernel and image ** Shortlog 271223a tests: Skip docker cp integration test for devicemapper storage 9b91166 tests: Fix functional tests to include workload directory 66cbd66 tests: Fix state tests and add check for workload_dir. b8ffdb1 tests: Fix mounts test for unmounting volume. 97922b5 storage: Pass drive name to hyperstart with the newcontainer command dd13e32 storage: Pass block device to qemu if found. b95748f state: Store the block storage device file system type and index 2d783ae storage: Add function to check the storage for container rootfs e05e368 mount: Add function to return the device name and file system type b1768e8 mount: Add function to get the mount point given a path f9c0d36 storage: Add function to check if device is a devicemapper device 2c8374d mount: Add function to get underlying device for a path. 69abef6 state: Update the rootfs and workload dir config from state f202ba4 namespace: Do not join mount namespace while unmounting. 9d8a083 mounts: Unmount the container rootfs 61b8b8b proxy: Pass hyperstart fsmap array with newcontainer command. 27f25fc proxy: Pass shared directory to hyperstart. 0af8583 state: Store the workload dir and rootfs mount to state. 53dc5ff state: Add the host_path and mnt_dir field to state. 4bb2d17 mounts: Bind mount container rootfs to workload directory 65d5eb4 config: Add a new field "workload_dir" to config for the worload path fb9f94e mounts: Factorise the pod function for bind-mouting rootfs 5249cb4 9pfs: Dynamically add 9pfs options to qemu. 0ef2f8c version: Update container image to version 16020 b1fcd2a obs: Update packaging to include kernel config changes for xfs 567127f cc-oci-runtime.sh: Enable debug by default 053ea8e tests: Fix make discheck running proxy d10dc46 configure: Add warning when kernel or image is not found c543dcb proxy: Better error message for too long socket path 5b83071 tests: metrics: density: Use the new metrics reporting function 35de59e tests: metrics: Update README with details of new infra 9a4daf9 tests: metrics: Add script to save results into csv 827347d tests: common: Add new method for saving results 68c2673 build: Adds option to install kernel and image ** Compatibility with Docker Clear Containers 2.2.0 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.0 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.0 requires at least Clear Linux containers image [16020][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.0 requires at least Clear Linux Containers kernel 4.9.33-74 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16020/clear/clear-16020-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1003 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

* Release 2.2.0 ** Changes - versions: update kernel to linux-container-4.9.34-75 - shim: Add command line option to show version information - storage: Perform sync operation before vm is shutdown - tests: Use nginx:1.13.0 for swarm tests - proxy: Lift the numbers of opened file descriptors limit - CI: Enforce checkcommits. - Fix unnecessary bash-isms in installation script ** Shortlog 3eeda89 versions: update kernel to linux-container-4.9.34-75 4e8f155 version: Add command line option to show version information 21b874a storage: Perform sync operation before vm is shutdown 191e9f8 tests: Use nginx:1.13.0 for swarm tests 2f22de2 proxy: Lift the numbers of opened file descriptors limit 55eba02 CI: Enforce checkcommits. 6566262 Fix unnecessary bash-isms in installation script ** Compatibility with Docker Clear Containers 2.2.0 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.0 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.0 requires at least Clear Linux containers image [16160][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.0 requires at least Clear Linux Containers kernel 4.9.34-75 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16160/clear/clear-16160-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1022 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

* Release 2.2.2 ** Changes - versions: update kernel to linux-container-4.9.34-75 - shim: Add command line option to show version information - storage: Perform sync operation before vm is shutdown - tests: Use nginx:1.13.0 for swarm tests - proxy: Lift the numbers of opened file descriptors limit - CI: Enforce checkcommits. - Fix unnecessary bash-isms in installation script ** Shortlog 3eeda89 versions: update kernel to linux-container-4.9.34-75 4e8f155 version: Add command line option to show version information 21b874a storage: Perform sync operation before vm is shutdown 191e9f8 tests: Use nginx:1.13.0 for swarm tests 2f22de2 proxy: Lift the numbers of opened file descriptors limit 55eba02 CI: Enforce checkcommits. 6566262 Fix unnecessary bash-isms in installation script ** Compatibility with Docker Clear Containers 2.2.1 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.1 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.1 requires at least Clear Linux containers image [16160][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.1 requires at least Clear Linux Containers kernel 4.9.34-75 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16160/clear/clear-16160-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1022 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

* Release 2.2.2 ** Changes - artifacts: Remove OBS common artifacsts between 2.x and 3.x - versions: Update kernel and image versions - Tests: Change nginx image for swarm tests - install: Extract autoconf-archive install to separate script ** Shortlog ef8144e artifacts: Remove OBS common artifacsts between 2.x and 3.x 73e28fe versions: Update kernel and image versions 3a9e3d7 Tests: Change nginx image for swarm tests 82cfb1f install: Extract autoconf-archive install to separate script ** Compatibility with Docker Clear Containers 2.2.2 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.2 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.2 requires at least Clear Linux containers image [16180][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.2 requires at least Clear Linux Containers kernel 4.9.35-76 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16180/clear/clear-16180-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1033 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

* Release 2.2.3 ** Changes - tests: Dockerfile for Nginx for swarm tests - delete: Fail if try to delete a non-stopped container. ** Shortlog 91aacdf tests: Dockerfile for Nginx for swarm tests 5121969 delete: allow destroy_pod fail if vm die 29f8aa2 tests: functional: check delete fails without kill a51537c delete: Delete container only if state is STOPPED ** Compatibility with Docker Clear Containers 2.2.3 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.3 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.3 requires at least Clear Linux containers image [16180][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.3 requires at least Clear Linux Containers kernel 4.9.35-76 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16180/clear/clear-16180-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1040 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

** Changes - tests: Dockerfile for Nginx for swarm tests - delete: Fail if try to delete a non-stopped container. ** Shortlog 91aacdf tests: Dockerfile for Nginx for swarm tests 5121969 delete: allow destroy_pod fail if vm die 29f8aa2 tests: functional: check delete fails without kill a51537c delete: Delete container only if state is STOPPED ** Compatibility with Docker Clear Containers 2.2.3 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.3 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.3 requires at least Clear Linux containers image [16180][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.3 requires at least Clear Linux Containers kernel 4.9.35-76 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16180/clear/clear-16180-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1040 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

** Changes - versions: Update image to version 16670 - OBS: add dependencies' versions - documentation: ciao-down prepare is now ciao-down create ** Shortlog cbdfd1d versions: Update image to version 16670 2c13fb1 OBS: add dependencies' versions 68d00dc documentation: Add the -u flag when go getting ciao-down 0c9d6bb documentation: ciao-down prepare is now ciao-down create ** Compatibility with Docker Clear Containers 2.2.4 is compatible with Docker v17.05.0-ce ** OCI Runtime Specification Clear Containers 2.2.4 support the OCI Runtime Specification [1.0.0-rc5][ocispec] ** Clear Linux Containers image Clear Containers 2.2.4 requires at least Clear Linux containers image [16670][clearlinuximage] ** Clear Linux Containers Kernel Clear Containers 2.2.4 requires at least Clear Linux Containers kernel 4.9.35-76 * Installation - [Centos][centos] - [Ubuntu][ubuntu] - [Fedora][fedora] - [Clear Linux][clearlinux] * Issues & limitations - Qemu segfault (free(): invalid pointer) running dnf install intel#669 - DNS Resolution in Swarm intel#854 - docker rm -f reports 'Driver devicemapper failed to remove root filesystem' intel#795 compatibility with docker 17.06 intel#1021 [centos]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Centos-7.md [ubuntu]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Ubuntu.md [fedora]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-Fedora.md [clearlinux]: https://github.com/01org/cc-oci-runtime/blob/master/documentation/Installing-Clear-Containers-on-ClearLinux.md [clearlinuximage]: https://download.clearlinux.org/releases/16670/clear/clear-16670-containers.img.xz [ocispec]: https://github.com/opencontainers/runtime-spec/releases/tag/v1.0.0-rc5 Fixes: intel#1050 Signed-off-by: Jose Carlos Venegas Munoz <jose.carlos.venegas.munoz@intel.com>

grahamwhaley · 2017-08-15T19:02:11Z

Hi @devimc @GabyCT @jcvenegas - is this still a problem we see? did/do we have any more info on why etc.? I only ask as it is still listed over in
clearcontainers/tests#381
https://github.com/clearcontainers/tests/pull/381/files#diff-51e815c00956126952247d59c8a047bdR76

for instance...

GabyCT · 2017-08-16T21:47:50Z

@grahamwhaley this is not present in CC 3.0 with the new agent, however, in CC 2.x is still happening

grahamwhaley · 2017-08-17T08:50:19Z

thanks @GabyCT OK, as the tests in https://github.com/clearcontainers/tests/tree/master/metrics should work for both CC3.x and CC2.x, I think we'll leave the warnings in there for the minute, and pointing at the CC2.x repo to indicate they are CC2.x related.

grahamwhaley mentioned this issue Apr 5, 2017

metrics: Fix CC memory footprint calculation #785

Merged

devimc self-assigned this Apr 27, 2017

jcvenegas added the type:bug label Apr 27, 2017

jcvenegas added the P2 label May 3, 2017

jcvenegas assigned grahamwhaley May 3, 2017

GabyCT assigned jcvenegas May 8, 2017

GabyCT mentioned this issue May 23, 2017

Error response from daemon: Driver devicemapper failed to remove root filesystem clearcontainers/runtime#163

Closed

jcvenegas mentioned this issue Jun 1, 2017

Driver overlay failed to remove root filesystem (docker 17.05) #936

Open

This was referenced Jun 8, 2017

Release: 2.1.11 #970

Merged

Release 2.1.12 #993

Merged

jcvenegas mentioned this issue Jun 22, 2017

release: 2.2.0 #1005

Merged

jcvenegas mentioned this issue Jun 29, 2017

release: 2.2.1 #1025

Merged

jcvenegas mentioned this issue Jul 6, 2017

release: version 2.2.2 #1037

Merged

jcvenegas mentioned this issue Jul 13, 2017

Release 2.2.3 #1041

Merged

egernst added the CC 2.1 label Jul 18, 2017

jcvenegas mentioned this issue Jul 28, 2017

Release 2.2.4 #1052

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docker rm -f reports 'Driver devicemapper failed to remove root filesystem' #795

docker rm -f reports 'Driver devicemapper failed to remove root filesystem' #795

grahamwhaley commented Apr 5, 2017

grahamwhaley commented Apr 5, 2017

grahamwhaley commented Apr 5, 2017

devimc commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

devimc commented Apr 27, 2017

grahamwhaley commented May 3, 2017

grahamwhaley commented May 4, 2017

GabyCT commented May 5, 2017

GabyCT commented May 5, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 8, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 8, 2017

GabyCT commented May 8, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 9, 2017

GabyCT commented May 9, 2017

jcvenegas commented May 9, 2017

jcvenegas commented May 9, 2017

jcvenegas commented May 17, 2017

grahamwhaley commented May 30, 2017

grahamwhaley commented Aug 15, 2017

GabyCT commented Aug 16, 2017

grahamwhaley commented Aug 17, 2017

docker rm -f reports 'Driver devicemapper failed to remove root filesystem' #795

docker rm -f reports 'Driver devicemapper failed to remove root filesystem' #795

Comments

grahamwhaley commented Apr 5, 2017

grahamwhaley commented Apr 5, 2017

grahamwhaley commented Apr 5, 2017

devimc commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

grahamwhaley commented Apr 27, 2017

devimc commented Apr 27, 2017

grahamwhaley commented May 3, 2017

grahamwhaley commented May 4, 2017

GabyCT commented May 5, 2017

GabyCT commented May 5, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 8, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 8, 2017

GabyCT commented May 8, 2017

GabyCT commented May 8, 2017

jcvenegas commented May 9, 2017

GabyCT commented May 9, 2017

jcvenegas commented May 9, 2017

jcvenegas commented May 9, 2017

jcvenegas commented May 17, 2017

grahamwhaley commented May 30, 2017

grahamwhaley commented Aug 15, 2017

GabyCT commented Aug 16, 2017

grahamwhaley commented Aug 17, 2017