Attach docker 里 pid 1的进程报错

[sunshibiao]

• 我已经在 issues 里搜索，没有重复的issue。

环境信息

是在docker容器里执行的 java -jar arthas-boot.jar
今天刚下的git最新版本
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (IcedTea 3.6.0) (Alpine 8.151.12-r0)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)

重现问题的步骤

/home # java -jar arthas-boot.jar

• [1]: 1 jar

[INFO] arthas home: /root/.arthas/lib/3.0.5/arthas
[INFO] Try to attach process 1

/home # ps xf
PID USER TIME COMMAND
1 root 489:16 java -Djava.security.egd=file:/dev/./urandom -jar /app.jar
16416 root 0:00 ./bin/sh
17936 root 0:00 ps xf

期望的结果

服务启动正常

实际运行的结果

[ERROR] Start arthas failed, exception stack trace:
com.sun.tools.attach.AttachNotSupportedException: Unable to get pid of LinuxThreads manager thread
at sun.tools.attach.LinuxVirtualMachine.(LinuxVirtualMachine.java:86)
at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:78)
at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:250)
at com.taobao.arthas.core.Arthas.attachAgent(Arthas.java:72)
at com.taobao.arthas.core.Arthas.(Arthas.java:25)
at com.taobao.arthas.core.Arthas.main(Arthas.java:99)
[ERROR] attach fail, targetPid: 1

把异常信息贴到这里

[ERROR] Start arthas failed, exception stack trace:
com.sun.tools.attach.AttachNotSupportedException: Unable to get pid of LinuxThreads manager thread
at sun.tools.attach.LinuxVirtualMachine.(LinuxVirtualMachine.java:86)
at sun.tools.attach.LinuxAttachProvider.attachVirtualMachine(LinuxAttachProvider.java:78)
at com.sun.tools.attach.VirtualMachine.attach(VirtualMachine.java:250)
at com.taobao.arthas.core.Arthas.attachAgent(Arthas.java:72)
at com.taobao.arthas.core.Arthas.(Arthas.java:25)
at com.taobao.arthas.core.Arthas.main(Arthas.java:99)
[ERROR] attach fail, targetPid: 1

[hengyunabc]

具体用的是什么 docker 镜像，详细的 url 。是否java进程都跑在docker里的。

[hengyunabc]

这个貌似是docker的问题： docker-library/openjdk#76

有两种解法：

• 执行 docker run时加上 --init 参数
• 在docker镜像里增加 init功能，比如 https://github.com/krallin/tini

[hengyunabc]

参考这个docker 镜像，测试没有问题： https://cloud.docker.com/repository/docker/hengyunabc/arthas

[hengyunabc]

增加 docker文档： https://alibaba.github.io/arthas/docker.html

[outh0527]

java -jar arthas-boot.jar 1

[robberphex]

这个问题的主要原因是没法通过信号启动jvm的Attach Listener，解决方案很简单：

pid=1 ;\
touch /proc/${pid}/cwd/.attach_pid${pid} && \
  kill -SIGQUIT ${pid} && \
  sleep 2 &&
  ls /proc/${pid}/root/tmp/.java_pid${pid}
# 接下来就可以正常 java -jar arthas-boot.jar 挂arthas了

具体的分析见https://robberphex.com/attach-jvm-in-container-at-arthas/。

[Delete-S]

@hengyunabc @robberphex
我试用了 jattach 1 properties 或者shell脚本的两种模式，报错还是一样的，并不能启动。

下面是log：
/opt/hawkbit # jattach 1 properties
Connected to remote JVM
JVM response code = 0
#Mon Oct 16 05:38:02 GMT 2023
java.specification.version=17
sun.jnu.encoding=UTF-8
java.class.path=hawkbit-update-server.jar
java.vm.vendor=Eclipse Adoptium
sun.arch.data.model=64
catalina.useNaming=false
java.vendor.url=https://adoptium.net/
user.timezone=GMT
os.name=Linux
java.vm.specification.version=17
sun.java.launcher=SUN_STANDARD
user.country=US
sun.boot.library.path=/opt/java/openjdk/lib
sun.java.command=hawkbit-update-server.jar --spring.profiles.active=mysql -Xms1024m -Xmx1024m -XX:MaxMetaspaceSize=250m -XX:MetaspaceSize=250m -Xss1024K -XX:+UseG1GC -XX:+UseStringDeduplication -XX:+UseCompressedOops -XX:+HeapDumpOnOutOfMemoryError
jdk.debug=release
sun.cpu.endian=little
user.home=/root
user.language=en
java.specification.vendor=Oracle Corporation
jdk.module.path=mariadb-java-client.jar
java.version.date=2023-04-18
java.home=/opt/java/openjdk
file.separator=/
java.vm.compressedOopsMode=32-bit
line.separator=\n
java.vm.specification.vendor=Oracle Corporation
java.specification.name=Java Platform API Specification
FILE_LOG_CHARSET=UTF-8
java.awt.headless=true
java.protocol.handler.pkgs=org.springframework.boot.loader
sun.management.compiler=HotSpot 64-Bit Tiered Compilers
java.runtime.version=17.0.7+7
user.name=root
path.separator=:
os.version=5.15.49-linuxkit-pr
java.runtime.name=OpenJDK Runtime Environment
file.encoding=UTF-8
spring.beaninfo.ignore=true
java.vm.name=OpenJDK 64-Bit Server VM
java.vendor.version=Temurin-17.0.7+7
java.vendor.url.bug=https://github.com/adoptium/adoptium-support/issues
org.apache.catalina.STRICT_SERVLET_COMPLIANCE=false
java.io.tmpdir=/tmp
catalina.home=/tmp/tomcat.8080.11568375747237983512
com.zaxxer.hikari.pool_number=1
java.version=17.0.7
user.dir=/opt/hawkbit
os.arch=amd64
java.vm.specification.name=Java Virtual Machine Specification
PID=1
CONSOLE_LOG_CHARSET=UTF-8
catalina.base=/tmp/tomcat.8080.11568375747237983512
native.encoding=UTF-8
java.library.path=/opt/java/openjdk/lib/server:/opt/java/openjdk/lib:/opt/java/openjdk/../lib:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib
java.vm.info=mixed mode, sharing
java.vendor=Eclipse Adoptium
java.vm.version=17.0.7+7
sun.io.unicode.encoding=UnicodeLittle
java.class.version=61.0

/opt/hawkbit # java -jar arthas-boot.jar
[INFO] JAVA_HOME: /opt/java/openjdk
[INFO] arthas-boot version: 3.7.1
[INFO] Found existing java process, please choose one and input the serial number of the process, eg : 1. Then hit ENTER.

• [1]: 1 hawkbit-update-server.jar
  1
  [INFO] arthas home: /root/.arthas/lib/3.7.1/arthas
  [INFO] Try to attach process 1
  Picked up JAVA_TOOL_OPTIONS:
  Error: Unable to initialize main class com.taobao.arthas.core.Arthas
  Caused by: java.lang.NoClassDefFoundError: com/sun/tools/attach/AgentLoadException
  [INFO] Attach process 1 success.
  [INFO] arthas-client connect 127.0.0.1 3658
  Connect to telnet server error: 127.0.0.1 3658
  java.net.ConnectException: Connection refused
  at java.base/sun.nio.ch.Net.pollConnect(Native Method)
  at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source)
  at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source)
  at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source)
  at java.base/java.net.SocksSocketImpl.connect(Unknown Source)
  at java.base/java.net.Socket.connect(Unknown Source)
  at org.apache.commons.net.SocketClient.connect(SocketClient.java:188)
  at org.apache.commons.net.SocketClient.connect(SocketClient.java:209)
  at com.taobao.arthas.client.TelnetConsole.process(TelnetConsole.java:306)
  at com.taobao.arthas.client.TelnetConsole.main(TelnetConsole.java:166)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
  at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
  at java.base/java.lang.reflect.Method.invoke(Unknown Source)
  at com.taobao.arthas.boot.Bootstrap.main(Bootstrap.java:633)
  Usage: arthas-client [-h ] [-t ] [-c ] [-f ]
  [--help] [-w ] [target-ip] [port]

Arthas Telnet Client

EXAMPLES:
java -jar arthas-client.jar 127.0.0.1 3658
java -jar arthas-client.jar -c 'dashboard -n 1'
java -jar arthas-client.jar -f batch.as 127.0.0.1

Options and Arguments:
-h,--height The terminal height
-t,--execution-timeout The timeout (ms) of execute commands or batch
file
-c,--command Command to execute, multiple commands
separated by ;
-f,--batch-file The batch file to execute
--help Print usage
-w,--width The terminal width
Target ip
The remote server port