Java process hangs on inside container #18502

Closed
stremlenye opened this Issue Dec 8, 2015 · 13 comments

Comments

Projects
None yet
8 participants
@stremlenye

Description:
Java process hangs on if started inside container.
That happens randomly but even if you run

java

or

java -version

To be precise commands are executed properly end prints what them supposed to print, but doesn't exit or fail. Logs are empty, jstack fails to connect to VM, ps -A shows java <defunct>.

docker version: Docker version 1.9.1, build a34a1d5
docker info:

Docker version 1.9.1, build a34a1d5

MacBook-Pro:~ user$ docker info

Containers: 4

Images: 13

Server Version: 1.9.1

Storage Driver: aufs

 Root Dir: /mnt/sda1/var/lib/docker/aufs

 Backing Filesystem: extfs

 Dirs: 21

 Dirperm1 Supported: true

Execution Driver: native-0.2

Logging Driver: json-file

Kernel Version: 4.1.13-boot2docker

Operating System: Boot2Docker 1.9.1 (TCL 6.4.1); master : cef800b - Fri Nov 20 19:33:59 UTC 2015

CPUs: 1

Total Memory: 996.2 MiB

Name: test

ID: YFBV:XZLM:67RC:XQ7H:3IJO:URTR:T4EU:ATIY:2Q3K:7RIS:E73M:DUBF

Debug mode (server): true

 File Descriptors: 12

 Goroutines: 19

 System Time: 2015-12-08T16:51:00.78015091Z

 EventsListeners: 0

 Init SHA1: 

 Init Path: /usr/local/bin/docker

 Docker Root Dir: /mnt/sda1/var/lib/docker

Labels:

 provider=virtualbox

uname -a: Darwin MacBook-Pro.local 15.0.0 Darwin Kernel Version 15.0.0: Sat Sep 19 15:53:46 PDT 2015; root:xnu-3247.10.11~1/RELEASE_X86_64 x86_64

Environment details: VirtualBox

How reproducible:

Steps to Reproduce:

  1. Run container in attached mode
  2. Install Java 8
  3. Run java couple of times till get hanged on

Actual Results:
Hang on java process

Expected Results:
Process normally exits

Additional info:
Reproduced on ubuntu@14.04, ubuntu@15.10 and centos@latest.

@stremlenye stremlenye changed the title from Java process hang on inside container to Java process hangs on inside container Dec 8, 2015

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 8, 2015

Member

This may be related to this issue; #18180

If you run docker top <yourcontainer>, do you also see a <defunct> process?

Member

thaJeztah commented Dec 8, 2015

This may be related to this issue; #18180

If you run docker top <yourcontainer>, do you also see a <defunct> process?

@andrewgdavis

This comment has been minimized.

Show comment
Hide comment
@andrewgdavis

andrewgdavis Dec 8, 2015

I don't know enough about java threading internals and how things are supposed to be shutdown cleanly... But I find it is very interesting that an explicit System.exit(0) call alleviates the problem as opposed to just leaving main. It would be interesting to implement a similar sequence of shutdown events in another language (modeled after java's) to see if it is java specific or not.

As a side note, this java + linux issue was recently post on hacker news: http://blog.omega-prime.co.uk/?p=161

I don't know enough about java threading internals and how things are supposed to be shutdown cleanly... But I find it is very interesting that an explicit System.exit(0) call alleviates the problem as opposed to just leaving main. It would be interesting to implement a similar sequence of shutdown events in another language (modeled after java's) to see if it is java specific or not.

As a side note, this java + linux issue was recently post on hacker news: http://blog.omega-prime.co.uk/?p=161

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 8, 2015

Member

Interesting, thanks!

Member

thaJeztah commented Dec 8, 2015

Interesting, thanks!

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Dec 8, 2015

Member

Yeah, this is definitely the same symptoms as #18180.

Member

tianon commented Dec 8, 2015

Yeah, this is definitely the same symptoms as #18180.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Dec 8, 2015

Member

Thanks for looking @tianon; is the link to the JAVA issue useful for resolving?

@stremlenye I'll close this issue in favor of #18180 so that we have the discussion on a single location. Please subscribe to that issue using the "subscribe" button to keep up to date on progress.

Thanks for reporting!

Member

thaJeztah commented Dec 8, 2015

Thanks for looking @tianon; is the link to the JAVA issue useful for resolving?

@stremlenye I'll close this issue in favor of #18180 so that we have the discussion on a single location. Please subscribe to that issue using the "subscribe" button to keep up to date on progress.

Thanks for reporting!

@thaJeztah thaJeztah closed this Dec 8, 2015

@tianon

This comment has been minimized.

Show comment
Hide comment
@tianon

tianon Dec 8, 2015

Member
Member

tianon commented Dec 8, 2015

@stremlenye

This comment has been minimized.

Show comment
Hide comment

@thaJeztah Thank you!

@binarytemple

This comment has been minimized.

Show comment
Hide comment

+1

@binarytemple

This comment has been minimized.

Show comment
Hide comment
@binarytemple

binarytemple Jan 7, 2016

Nothing to do with the console, just ran java > /dev/null 2>&1 ... container hangs.

Nothing to do with the console, just ran java > /dev/null 2>&1 ... container hangs.

@bhagwat

This comment has been minimized.

Show comment
Hide comment

bhagwat commented Jan 8, 2016

+1

@kgx

This comment has been minimized.

Show comment
Hide comment

kgx commented Jan 12, 2016

+1

@binarytemple

This comment has been minimized.

Show comment
Hide comment
@binarytemple

binarytemple Jan 12, 2016

This issue is closed, @bhagwat @kgx better to add your +1 to #18180.

Incidentally I worked around the problem by using 'devicemapper' rather than default AUFS storage driver - when using docker-machine you can set it to be the default by using the flag --engine-storage-driver=devicemapper when creating the boot2docker VM.

Also everything I read/everyone I speak to seems to be of the opinion that AUFS is a mess and not to use it, so this may be a chance to change early on.

This issue is closed, @bhagwat @kgx better to add your +1 to #18180.

Incidentally I worked around the problem by using 'devicemapper' rather than default AUFS storage driver - when using docker-machine you can set it to be the default by using the flag --engine-storage-driver=devicemapper when creating the boot2docker VM.

Also everything I read/everyone I speak to seems to be of the opinion that AUFS is a mess and not to use it, so this may be a chance to change early on.

@jgsqware

This comment has been minimized.

Show comment
Hide comment
@jgsqware

jgsqware Jan 29, 2016

thanks @binarytemple I got exactly the same problem with android tools and the switch to devicemapper did the trick!

thanks @binarytemple I got exactly the same problem with android tools and the switch to devicemapper did the trick!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment