Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Containeri is staing in state " Removal In Progress" for 12 weeks #3434

Closed
yuklia opened this issue May 9, 2016 · 20 comments
Closed

Containeri is staing in state " Removal In Progress" for 12 weeks #3434

yuklia opened this issue May 9, 2016 · 20 comments

Comments

@yuklia
Copy link

yuklia commented May 9, 2016

docker ps --all
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS                PORTS               NAMES
9adaaacc30d3        3d93fd21072f        "/bin/bash"         12 weeks ago        Removal In Progress                       9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_9adaaacc30_web
$ docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 35
Server Version: 1.11.1
Storage Driver: aufs
 Root Dir: /var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 146
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge null host
Kernel Version: 3.19.0-58-generic
Operating System: Ubuntu 14.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 11.65 GiB
Name: yuklia
ID: X2GJ:JAQ3:MU3L:44E6:5O45:LHAY:RLRR:FQ25:PZ55:3SDP:KPY5:BQ6H
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Username: yuklia
Registry: https://index.docker.io/v1/

docker rm -f 9adaaacc30d3 makes no sense

@yuklia yuklia changed the title Container stayed in state " Removal In Progress" for 12 weeks Containeri is staing in state " Removal In Progress" for 12 weeks May 9, 2016
@liamawhite
Copy link

have you tried forcing removal?

docker rm -f 9adaaacc30d3

@yuklia
Copy link
Author

yuklia commented May 10, 2016

@liamawhite yes, it doesn't help

@shin-
Copy link

shin- commented May 10, 2016

Hm, I don't think this is a docker-compose issue. However, the way I would proceed is

  1. Stop the docker daemon (sudo service docker stop)
  2. Remove the remaining container files sudo rm -rf /var/lib/docker/containers/9adaaacc30d3<TAB>
  3. Start the docker daemon (sudo service docker start).

HTH

@shin-
Copy link

shin- commented May 23, 2016

I'm closing this issue now, if you're still experiencing trouble feel free to comment again.

@shin- shin- closed this as completed May 23, 2016
@shahmaulikn
Copy link

I have created Cronjob for removing docker containers, I found same "Removal in Progress" .

@karanp24
Copy link

karanp24 commented Oct 3, 2017

I am facing this same issue in docker 17.09-ce , when I manually try to remove the container i get the folllowing error
Error response from daemon: unable to remove filesystem for 872a4f65532abd823760e8bdce43c614bc171570b8d6cf3264c3deb274cf01b5: remove /var/lib/docker/containers/872a4f65532abd823760e8bdce43c614bc171570b8d6cf3264c3deb274cf01b5/shm: device or resource busy
If the daemon is restarted then these exited/Removal In Progress containers disappear.As this is my production system I cant take chance of restarting daemon again and again. Any suggestions how can we overcome this behavior.

@r2p2
Copy link

r2p2 commented Mar 4, 2018

Same here, this issue should not stay closed. Docker version 18.02.0-ce.

@dogrod
Copy link

dogrod commented Mar 5, 2018

Same problem, it confused me for a long time, please fix it. docker version 17.12.0-ce.

@Irytacja
Copy link

18.05.0-ce
Cannot force remove container from /var/liv/docker/containers neither remove it with force remove via docker rm -f

@xu-song
Copy link

xu-song commented Sep 30, 2018

+1

root@11:~# systemctl stop docker
root@11:~# rm -rf /var/lib/docker/containers/f0e2078...5/
rm: cannot remove '/var/lib/docker/containers/f0e...5/mounts/shm': 
Device or resource busy

@FritschAuctores
Copy link

FritschAuctores commented Nov 12, 2018

It also works this
docker rmi $(docker images -a -q) -f
docker rm $(docker ps -a -f status=exited -q)
restart docker

use it careful !

@rahman13dec4u
Copy link

same issue in latest stable version of docker-ce..this is causing a lots of issue now...whenever we are deleting a container it goes in "Removal In Progress" and only option to fix is to reboot the server..even manual docker service stop hangs.

Can someone please let us know the solution or when can this be fixed.

@jerryleooo
Copy link

One of our servers has such issue, I compared it with other servers and find the difference is we use "devicemapper" as Storage Driver. I am investigating this and will update anything new here.

@AaronDMarasco-VSI
Copy link

@jerryleooo devicemapper is deprecated as of 18.09, so I doubt you're going to get closure on this ticket. I haven't seen the problem since migrating to overlay2 (CentOS 7).

@jerryleooo
Copy link

@AaronDMarasco-VSI I think so. The server with issue is CentOS 7 with 3.10.0-327 kernel and the file system is xfs. We found it is a little frustrated to change it to overlay2 due to the xfs file system. Right now we simply remove it from production servers and will change the file system later.

@AaronDMarasco-VSI
Copy link

@jerryleooo I am using overlay2 on xfs with 3.10.0-957.1.3.el7.x86_64 with no problems. I was lucky to have some extra space in my LVM when I moved it from the raw devicemapper partitions (forget the details, but one was sparse allocated and the other was metadata).

@4sskick
Copy link

4sskick commented Aug 5, 2019

Hm, I don't think this is a docker-compose issue. However, the way I would proceed is

  1. Stop the docker daemon (sudo service docker stop)
  2. Remove the remaining container files sudo rm -rf /var/lib/docker/containers/9adaaacc30d3<TAB>
  3. Start the docker daemon (sudo service docker start).

HTH

You've saved my day! this method help me, thank you!

@jorioux
Copy link

jorioux commented Aug 27, 2019

Hm, I don't think this is a docker-compose issue. However, the way I would proceed is

  1. Stop the docker daemon (sudo service docker stop)
  2. Remove the remaining container files sudo rm -rf /var/lib/docker/containers/9adaaacc30d3<TAB>
  3. Start the docker daemon (sudo service docker start).

HTH

For Windows, the path for the containers directory is:
C:\ProgramData\docker\containers

@crixster
Copy link

Hm, I don't think this is a docker-compose issue. However, the way I would proceed is

  1. Stop the docker daemon (sudo service docker stop)
  2. Remove the remaining container files sudo rm -rf /var/lib/docker/containers/9adaaacc30d3<TAB>
  3. Start the docker daemon (sudo service docker start).

HTH

Thank God ! finally a solution! I was trying to remove plex via portainer and i accidently clicked on refreshed page. It was stuck and status was showing removal in progress but the container was dead. So far this is the only method that worked

@eduncan911
Copy link

For the record on pop-os 22.04 (aka ubuntu 22.04), and using docker install via CLI (not package manager version), I run into this problem of Removal In Progress when I have to run some docker containers as root during some build processes (for microcontrollers and qemu emu).

The layer/container files become owned by root, instead of my userspace - and docker daemon can't clean them.
Perhaps it's a security? That since my uid didn't create the container file (was going back and forth between --privledged and running as root), but did originally start the process, my uid-within-docker isn't given the rights to delete.

I've always had to sudo rm --rf /var/lib/docker/containers/<ID> between stopping the daemon on those particular containers. I try to avoid raw embedded development on my desktops without VMs (no buildroot/yocto). But in this particular occasion, on this particular machine, i often run into the Removal In Progress and it's the same containers I'm fracking with root on for the loopback devices and low-level stuff, as well as emulations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests