New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

jmap not happy on alpine #76

Closed
devinrsmith opened this Issue May 25, 2016 · 12 comments

Comments

Projects
None yet
10 participants
@devinrsmith

devinrsmith commented May 25, 2016

We ran into an issue today where we needed to take a heap dump (jmap). But it wasn't working.

Image was java:openjdk-8u92-jdk-alpine.

$ docker exec server1_v3_1 ps aux
PID   USER     TIME   COMMAND
    1 root       1:54 java -server -verbose:gc ...
  121 root       0:00 ps aux

$ docker exec server1_v3_1 jmap -dump:live,format=b,file=/data/dump.map 1
1: Unable to get pid of LinuxThreads manager thread

Here's the source of the error: http://code.metager.de/source/xref/openjdk/jdk8/jdk/src/solaris/native/sun/tools/attach/LinuxVirtualMachine.c#255

We suspect it might be because alpine is using musl instead of glibc.

jmap works in java:openjdk-8u91-jdk.

@niloc132

This comment has been minimized.

niloc132 commented May 25, 2016

Other tools that don't work: jstack, jinfo, jhat.

@paranoid945

This comment has been minimized.

paranoid945 commented Aug 30, 2016

Not working in openjdk 7 neither.
does any openjdk engineer watching this issue?

@yosifkit

This comment has been minimized.

Member

yosifkit commented Sep 1, 2016

This seems related: gliderlabs/docker-alpine#11.

@michaelzg

This comment has been minimized.

michaelzg commented Jan 15, 2017

Tried what @yosifkit linked.

Installed glibc as suggested by sgerrand in sgerrand/alpine-pkg-glibc#1 and still no luck:

/opt # apk info | grep glibc
glibc
glibc-bin

/opt # java -version
openjdk version "1.8.0_111-internal"
OpenJDK Runtime Environment (build 1.8.0_111-internal-alpine-r0-b14)
OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)

/opt # ps aux
PID   USER     TIME   COMMAND
    1 root       8:38 java ...-jar ...
  303 root       0:00 /bin/sh
  421 root       0:00 ps aux

/opt # jmap -dump:format=b,file=heap.bin 1
1: Unable to get pid of LinuxThreads manager thread

Docker image: 8u111-jdk-alpine

@tianon

This comment has been minimized.

Member

tianon commented Jan 27, 2017

Indeed -- I don't think there's much we can do about this. 😞

@dkolog

This comment has been minimized.

dkolog commented Jan 31, 2017

I noticed that too. This happens because your java process PID is 1.
I use the same setup (alpine + glibc + jmap) and it works well if the PID is not 1 (when bash scripts forks the java process)

@yosifkit

This comment has been minimized.

Member

yosifkit commented Jan 31, 2017

If you keep bash as PID 1, you'd no longer get signals from docker stop and docker kill (without having to write traps). With docker 1.13 you can use --init to have docker put in tini as PID 1; it'll forward signals and reap zombies.

@dkolog

This comment has been minimized.

dkolog commented Feb 1, 2017

@yosifkit you are absolutely right.
@devinrsmith I was able to get heap/thread dump from PID 1 using Oracle's JDK.

@elyscape

This comment has been minimized.

elyscape commented Feb 20, 2017

This may be a result of the debug symbols not being installed in the image. See comment 9 on this issue or comment 15 on this issue.

@sgerrand

This comment has been minimized.

sgerrand commented Jun 20, 2017

Installed glibc as suggested by @sgerrand

For those arriving late to the party, like me, this issue appears to be unrelated to the underlying libc implementation.

@sitnik

This comment has been minimized.

sitnik commented Jul 5, 2017

I have to use docker 1.12 and don't know if '--init' will be used in Kubernetes, so I solved this issue by wrapping java process in shell script and adding trap for TERM signal. At least jmap now works for me.

@tianon

This comment has been minimized.

Member

tianon commented Jan 3, 2018

Closing, since this appears to be solved by either using Docker's --init flag, or adding something like tini / dumb-init to images which suffer from this problem. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment