New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker container hangs #1116

Closed
adacey opened this Issue Jan 9, 2017 · 11 comments

Comments

Projects
None yet
3 participants
@adacey

adacey commented Jan 9, 2017

Expected behaviour

Docker container process to continue running.

Actual behaviour

Docker container linuxserver/sabnzbd becomes hung with a zombie process. The container can bee accessed by docker exec but killing the process is unsuccessful. The container also cannot be stopped when using docker stop or docker kill. The only option is to completely restart docker and restart all containers.

Information

Docker issue:

56EFECB3-1FA1-4F3C-87E4-731B61B784CE

Steps to reproduce the behaviour

  1. Start sabnzbd container (linuxserver/sabnzbd)
  2. Wait. As far as I can tell, it seems to be a problem with unrar getting hung while unpacking a file, but the sabnzbd process itself also becomes unresponsive and unkillable.
@adacey

This comment has been minimized.

adacey commented Jan 9, 2017

Output from diagnostics:

Docker for Mac: version: 1.12.5 (3e6f00c1d)
OS X: version 10.12.2 (build: 16C67)
logs: /tmp/56EFECB3-1FA1-4F3C-87E4-731B61B784CE/20170109-111207.tar.gz
[OK] vmnetd
[OK] dns
[OK] driver.amd64-linux
[OK] virtualization VT-X
[OK] app
[OK] moby
[OK] system
[OK] moby-syslog
[OK] db
[OK] env
[OK] virtualization kern.hv_support
[OK] slirp
[OK] osxfs
[OK] moby-console
[OK] logs
[OK] docker-cli
[OK] menubar
[OK] disk

@adacey

This comment has been minimized.

adacey commented Jan 9, 2017

$ docker info
Containers: 13
Running: 9
Paused: 0
Stopped: 4
Images: 16
Server Version: 1.12.5
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 63
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge null host overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.39-moby
Operating System: Alpine Linux v3.4
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 9.743 GiB
Name: moby
ID: AXFJ:5QL7:6UE3:U4QR:N6EC:LAQF:RXIY:PLFZ:JK77:DH6M:PFTJ:GVGY
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 73
Goroutines: 71
System Time: 2017-01-09T12:44:45.048483506Z
EventsListeners: 1
No Proxy: *.local, 169.254/16
Registry: https://index.docker.io/v1/
WARNING: No kernel memory limit support
Insecure Registries:
127.0.0.0/8

@adacey

This comment has been minimized.

adacey commented Jan 9, 2017

This seems to point to an issue with the Hyperkit machine itself hanging on a filesystem. If I attach to the docker socket with screen to troubleshoot it I find that ls and df are hanging.

/ # uname -a
Linux moby 4.4.39-moby #1 SMP Fri Dec 16 07:34:12 UTC 2016 x86_64 Linux
/ # df -h
Filesystem Size Used Available Use% Mounted on
tmpfs 4.9G 158.0M 4.7G 3% /
tmpfs 997.6M 740.0K 996.9M 0% /run
cgroup_root 10.0M 0 10.0M 0% /sys/fs/cgroup
dev 10.0M 0 10.0M 0% /dev
shm 4.9G 604.0K 4.9G 0% /dev/shm
/dev/vda2 59.0G 1.3G 54.7G 2% /var
osxfs 465.0G 131.6G 333.1G 28% /Users

I've left df running like that for over 5 minutes now and the prompt still hasn't returned. I am mounting an AFP share from my NAS via /Volumes/media which is used on numerous containers (Plex, Sonarr, Couchpotato, Sabnzbd, Transmission) and it seems to work without issue except for sabnzbd.

@dsheets

This comment has been minimized.

Contributor

dsheets commented Jan 9, 2017

Could you please post the specific command you are executing in the sabnzbd case?

There have been a number of improvements to relevant subsystems that have been published on the Beta channel, too. If you are comfortable running the Beta channel, you could give it a try and see if the problem has already been fixed. Please note that this will entail removing all of your containers and images when changing channels.

@adacey

This comment has been minimized.

adacey commented Jan 9, 2017

I'm creating the sabnzbd container and then just starting it with docker start sabnzbd. The openvpn network is from dperson/openvpn-client on a separate bridge network (vpn) and is used by my sabnzbd, transmission, sonarr, and couchpotato containers. I also have an nginx container running that's linked to the vpn network via docker network connect providing reverse proxy access to the containers on the vpn network.

docker create --name=sabnzbd \
--net=container:openvpn \
-v /Users/Shared/docker/containers/sabnzbd/config:/config \
-v /Volumes/media/downloads:/downloads \
-e PGID=20 -e PUID=501 \
-e TZ=Europe/London \
linuxserver/sabnzbd

I'm not opposed to upgrading to the beta channel given that it looks like 1.13 is pretty far along on release candidates. How easily could I switch back to stable after 1.13 is released or would I need to recreate my containers again?

@adacey

This comment has been minimized.

adacey commented Jan 10, 2017

I might have tentatively found a fix, awaiting further testing. I realised that I shouldn't have bothered with mounting the downloads directory from the NAS drive, it makes far more sense for that to be on the local filesystem and then have Sonarr or Couchpotato take care of copying over the completed downloads to the NAS. It was just a holdover from migrating over from my Synology NAS that had been running everything before I'd switched to Docker on my Mac Mini.

@adacey

This comment has been minimized.

adacey commented Jan 12, 2017

No luck. Stayed stable for a day or so but then crashed again overnight, and on restart it crashed again within 5 minutes. Still seems to be hit during yenc decoding or unrar. I tried a docker stop first to see if that would stop the container this time as I still had the sabnzbd web gui when the hang after restarting docker happened. Still seeing the same hanging on df and ls when I attach to hyperkit with screen.

@dsheets

This comment has been minimized.

Contributor

dsheets commented Jan 17, 2017

What version did you use for these latest tests? If at all possible, instructions on how to reproduce the issue would help immensely. Removing variables would also be very useful. Do you observe the issue even when not using AFP file systems? Does it happen when executing unrar or similar alone outside of sabnzbd?

@adacey

This comment has been minimized.

adacey commented Jan 18, 2017

I'd removed any AFP shares from the sabnzbd container (they weren't needed anyway) and still had the same issue. Upgrading to the beta channel has cleared up the issue though.

@dsheets

This comment has been minimized.

Contributor

dsheets commented Jan 19, 2017

Glad the beta channel fixes the problem for you! A synchronizing stable release for 1.13.0 should be available shortly which should also work. Thanks for using Docker for Mac!

@dsheets dsheets closed this Jan 19, 2017

@TrueOsiris

This comment has been minimized.

TrueOsiris commented Nov 8, 2017

I have this exact same issue on the current unRaid server with native docker and linuxserver/sonarr
I've mounted a samba share (to a freenas share) to this container (via the host)
The share remains accessible without issues. (I don't know from the docker)
I can "docker exec -it sonarr /bin/bash" but every command there remains pending.
Docker version 1.12.6, build 6b644ec

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment