Docker driver "overlay2" failed to remove root filesystem

[GeorgeV220]

Is there an existing issue for this?

• I have searched the existing issues before opening this issue.

Current Behavior

When I try to start my server I get the error that the Docker driver "overlay2" failed to remove root filesystem. If I restart the dedicated server, everything will work properly until 2-3 restarts are made on the game server.
Console logs:

container@pterodactyl~ Server marked as offline...
[Pterodactyl Daemon]: Checking server disk space usage, this could take a few seconds...
[Pterodactyl Daemon]: Updating process configuration files...
[Pterodactyl Daemon]: Ensuring file permissions are set correctly, this could take a few seconds...
container@pterodactyl~ Server marked as starting...
container@pterodactyl~ Server marked as offline...
container@pterodactyl~ Error Event [e5a97eed-f3d5-48db-8be3-80c8b41075ce]: environment/docker: failed to remove container during pre-boot: Error response from daemon: container 24773d7e124a62e9f4e4facf5134175db8126516bb4ad9e51201be896a41fbe6: driver "overlay2" failed to remove root filesystem: unlinkat /var/lib/docker/overlay2/c334e3332baee0f79e92ced2282f38f9e12e11020b2239ad78ca095e91203e91/merged: device or resource busy
[Pterodactyl Daemon]: ---------- Detected server process in a crashed state! ----------
[Pterodactyl Daemon]: Exit code: 137
[Pterodactyl Daemon]: Out of memory: false
[Pterodactyl Daemon]: Checking server disk space usage, this could take a few seconds...
[Pterodactyl Daemon]: Updating process configuration files...
[Pterodactyl Daemon]: Ensuring file permissions are set correctly, this could take a few seconds...
container@pterodactyl~ Server marked as starting...
[Pterodactyl Daemon]: Server crash was detected but an error occurred while handling it.
container@pterodactyl~ Server marked as offline...
[Pterodactyl Daemon]: ---------- Detected server process in a crashed state! ----------
[Pterodactyl Daemon]: Exit code: 137
[Pterodactyl Daemon]: Out of memory: false
[Pterodactyl Daemon]: Aborting automatic restart, last crash occurred less than 60 seconds ago.

OS: CentOS 7.9
Kernel Version: 5.15.5-1.el7.elrepo.x86_64
Docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.6.3-docker)
  scan: Docker Scan (Docker Inc., v0.9.0)

Server:
 Containers: 3
  Running: 0
  Paused: 0
  Stopped: 3
 Images: 3
 Server Version: 20.10.11
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
 runc version: v1.0.2-0-g52b36a2
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.15.5-1.el7.elrepo.x86_64
 Operating System: CentOS Linux 7 (Core)
 OSType: linux
 Architecture: x86_64
 CPUs: 12
 Total Memory: 62.81GiB
 Name: <>
 ID: GUEI:KXYA:EAOF:2ZPG:LWS6:6ZHN:LS52:H3OO:UGKS:YA5B:ZUMY:TYWF
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Expected Behavior

Start the server normally

Steps to Reproduce

I just did 2-3 reboots of the game server and it shows the error, I do not know what causes it

Panel Version

1.6.6

Wings Version

1.5.3

Error Logs

https://ptero.co/ihofutypec
https://bin.ptdl.co/xxqdy/

[GeorgeV220]

The problem was that the docker mounted to a cPanel jailshell

/home/georgev22martexn/var/lib/docker/overlay2/XXX/merged

so when closing the game server, the docker (or wings I'm not sure) could not find the mountpoint that should normally be on

/var/lib/docker/overlay2/XXX/merged

[ajdevtechgithub]

I am also experiencing the same issue, but as we verified that docker is not mounted on the jailed shell and it is on the normal shell

ERRO[0070] Error waiting for container: container 6f14155449c87fc68854c0972e3a696fc05aa3165220913c1caf82b3e843f619: driver "overlay2" failed to remove root filesystem: unlinkat /var/lib/docker/overlay2/325b8027b7ac2807ee6fe6293c33d5c1e5a82dddb1e8fafb212a56640aa9cf2c/merged: device or resource busy

We tried all way but only reboot seems worked for one time, not cagefs is active on this.