Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Print informational message when kernel.core_pattern is piped #6838

Merged
merged 1 commit into from
Jan 24, 2023

Conversation

kgibm
Copy link
Contributor

@kgibm kgibm commented Dec 6, 2022

On Linux, when a core dump is requested, if kernel.core_pattern is
a piped pattern, then print a new informational message JVMPORT049I
to clarify the subsequent JVMDUMP012E message that although the JVM
cannot find the core dump, the user should review the manual for
the program specified in core_pattern to potentially find the core.

Fixes #6837

CC @pshipton @keithc-ca

@kgibm
Copy link
Contributor Author

kgibm commented Dec 6, 2022

Local test results:

When kernel.core_pattern=|/usr/lib/systemd/systemd-coredump [...]

root@ee4cd086c862:/openj9-openjdk-jdk8# sysctl kernel.core_pattern
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
root@ee4cd086c862:/openj9-openjdk-jdk8# ( cd build/linux-x86_64-normal-server-release/images/j2sdk-image; printf 'public class Test { public static void main(String... args) { System.out.println("Hello World"); } }' > Test.java && ./bin/javac Test.java && ./bin/java -Xshareclasses:name=test -Xdump:system:events=vmstop,request=exclusive+prepwalk Test )
Hello World
JVMDUMP039I Processing dump event "vmstop", detail "#0000000000000000" at 2022/12/06 21:42:32 - please wait.
JVMDUMP032I JVM requested System dump using '/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214232.76306.0001.dmp' in response to an event
JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.76329.  Review the manual for the external program to find where the core dump is written and ensure the program does not truncate it.

JVMPORT049I The core file created by child process with pid = 76329 was not found. Review the documentation for the /proc/sys/kernel/core_pattern program "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" to find where the core file is written and ensure that program does not truncate it.

JVMDUMP012E Error in System dump: /openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214232.76306.0001.dmp
JVMDUMP013I Processed dump event "vmstop", detail "#0000000000000000".
jdmpview output looks good
[root@localhost ~]# coredumpctl 
TIME                           PID UID GID SIG     COREFILE EXE                                                                                        SIZE
Tue 2022-12-06 15:42:36 CST 103652   0   0 SIGSEGV present  /openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/bin/java 14.6M
[root@localhost ~]# coredumpctl info 103652 | grep Storage
       Storage: /var/lib/systemd/coredump/core.DestroyJavaVM\x20h.0.c08b57045c9045bfa7b719b1c4cd7504.103652.1670362952000000.zst (present)
[root@localhost ~]# cp "/var/lib/systemd/coredump/core.DestroyJavaVM\x20h.0.c08b57045c9045bfa7b719b1c4cd7504.103652.1670362952000000.zst" /Users/kevin/jdkbuilds/

Kevins-MBP:jdkbuilds kevin$ mv core* core.dmp.zst
Kevins-MBP:jdkbuilds kevin$ unzstd core.dmp.zst 
core.dmp.zst        : 542658560 bytes                                          

root@ee4cd086c862:/openj9-openjdk-jdk8# cp /root/hostdir/core.dmp /
root@ee4cd086c862:/openj9-openjdk-jdk8# ( cd build/linux-x86_64-normal-server-release/images/j2sdk-image; ./bin/jdmpview -core /core.dmp )
DTFJView version 4.29.5, using DTFJ version 1.12.29003
Loading image from DTFJ...

For a list of commands, type "help"; for how to use "help", type "help help"
Available contexts (* = currently selected context) : 

Source : file:///core.dmp
	*0 : PID: 76329 : JRE 1.8.0 Linux amd64-64 (build 1.8.0_362-internal-_2022_12_06_20_25-b00)

> !shrc stats
!j9sharedclassconfig 0x00007F77980B8200

!j9sharedcacheheader 0x00007F77600000F0    (layer 0)
cache size       : 66744320
free bytes       : 58833764
softmx bytes     : -1
read write area  : 0x00007F7760000260 - 0x00007F7760000260 size 364304 used 0
segment area     : 0x00007F7760059000 - 0x00007F77601C9A48 size 1509960
metadata area    : 0x00007F77639E55AC - 0x00007F7763AE9000 size 1063508
class debug area : 0x00007F7763AE9000 - 0x00007F7764000000 size 5337088 used 145438 (% used 2)
Meta data region to be used: 0x00007F77639E55AC..0x00007F7763AE8FFC
No entry found in the cache

Cache contains 522 classes, 1 orphans, 2 classpaths, 0 URLs, 0 tokens
24 AOT, 68 SCOPES, 51 BYTE data, 0 UNINDEXED DATA, 0 CHARARRAY, 0 stale
stale bytes 0
4 JITPROFILE, 11 JITHINT
AOT data length 39408 code length 37952 metadata 672 total 77360
JITPROFILE data length 84 metadata 128 
JITHINT data length 44 metadata 352 
ROMClass data 1509960 metadata 22988
SCOPE data 2437 metadata 816 total 3253
BYTE data 955136 metadata 1836 rwarea 0
UNINDEXEDBYTE data 0 metadata 0
CHARARRAY data 0 metadata 0
BYTEDATA Summary
	UNKNOWN 0  HELPER 0  POOL 0  AOTHEADER 264
	JCL 0  VM 0  ROMSTRING 0  ZIPCACHE 953152  STARTUPHINTS 0
	JITHINT 0  AOTCLASSCHAIN 1520 AOTTHUNK 200
DEBUG Area Summary
	LineNumberTable bytes    : 145438
	LocalVariableTable bytes : 0
> nfo thread *
Unrecognised command: nfo
> info thread *
native threads for address space
 process id: 76306

  thread id: 76307
   registers:
   native stack sections:
   native stack frames:
   properties:
   associated Java thread: 
    name:          DestroyJavaVM helper thread
    Thread object: java/lang/Thread @ 0xfff7d198
    Native info:   !j9vmthread 0x0  !stack 0x0
    Daemon:        false
    ID:            19 (0x13)
    Priority:      5
    Thread.State:  RUNNABLE 
    JVMTI state:   ALIVE RUNNABLE 
    Java stack frames: <no frames to print>


  thread id: 76329
   registers:
    cs     = 0x0000000000000033   ds     = 0x0000000000000000   eflags = 0x0000000000000282   es     = 0x0000000000000000
    fs     = 0x0000000000000000   fs_base = 0x00007f779f990700   gs     = 0x0000000000000000   gs_base = 0x0000000000000000
    r8     = 0x0000000000000000   r9     = 0x0000000000000000   r10    = 0x0000000000000008   r11    = 0x0000000000000282
    r12    = 0x00007f779eb603a0   r13    = 0x00007f779f98f2f0   r14    = 0x00007f77980246d0   r15    = 0x00007f779f98f770
    rax    = 0x0000000000000000   rbp    = 0x0000000000000000   rbx    = 0x00007f779f98f2a0   rcx    = 0x00007f779ed8259f
    rdi    = 0x0000000000012a29   rdx    = 0x000000000000000b   rip    = 0x00007f779ed8259f   rsi    = 0x0000000000012a29
    rsp    = 0x00007f779f98eda8   ss     = 0x000000000000002b
   native stack sections:
    0x7f779f951000 to 0x7f779f991000 (length 0x40000)
   native stack frames:
    bp: 0x00007f779f98edb0 pc: 0x00007f779ed8259f libpthread.so.0::pthread_kill+0x2f
    bp: 0x00007f779f98edf0 pc: 0x00007f779d9aa41d libj9prt29.so::j9port_allocate_library+0x1becd
    bp: 0x00007f779f98f060 pc: 0x00007f779d53c2b2 libj9dmp29.so+0x82b2
    bp: 0x00007f779f98f070 pc: 0x00007f779d5380c5 libj9dmp29.so+0x40c5
    bp: 0x00007f779f98f1d0 pc: 0x00007f779d9abf03 libj9prt29.so::j9port_allocate_library+0x1d9b3
    bp: 0x00007f779f98f230 pc: 0x00007f779d53b71b libj9dmp29.so+0x771b
    bp: 0x00007f779f98f6f0 pc: 0x00007f779d53b8af libj9dmp29.so+0x78af
    bp: 0x00007f779f98fb40 pc: 0x00007f779d554366 libj9dmp29.so::J9VMDllMain+0x16056
    bp: 0x00007f779f98fbb0 pc: 0x00007f779d5522a6 libj9dmp29.so::J9VMDllMain+0x13f96
    bp: 0x00007f779f98fc50 pc: 0x00007f779df0339e libj9hookable29.so+0x139e
    bp: 0x00007f779f98fcc0 pc: 0x00007f779e36e6fb libj9vm29.so::J9_GetCreatedJavaVMs+0x119b
    bp: 0x00007f779f98fe20 pc: 0x00007f779d9abf03 libj9prt29.so::j9port_allocate_library+0x1d9b3
    bp: 0x00007f779f98fea0 pc: 0x00007f779e36daf9 libj9vm29.so::J9_GetCreatedJavaVMs+0x599
    bp: 0x00007f779f98feb0 pc: 0x00007f779e936cfa libjvm.so::JVM_IsUseContainerSupport+0x2da
    bp: 0x00007f779f98ff60 pc: 0x00007f779f567708 libjli.so+0x8708
    bp: 0x00007f779f990000 pc: 0x00007f779ed7b6ba libpthread.so.0::__pthread_get_minstack+0x164a
    bp: 0x00007f779f990008 pc: 0x00007f779f29c51d libc.so.6::clone+0x6d
   properties:
    Thread sys time secs=0    Thread sys time usecs=1000
    Thread user time secs=0   Thread user time usecs=1000
   associated Java thread: <no associated Java thread>

Java threads not associated with known native threads:

    name:          IProfiler
    id:            76320
    Thread object: java/lang/Thread @ 0xfff585f0
    Native info:   !j9vmthread 0x3cb00  !stack 0x3cb00
    Daemon:        true
    ID:            11 (0xb)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-iprofilerMonitor" with ID 0x7f7770001e28 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-001 Suspended
    id:            76310
    Thread object: java/lang/Thread @ 0xfff570e0
    Native info:   !j9vmthread 0x1e800  !stack 0x1e800
    Daemon:        true
    ID:            3 (0x3)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f779805c258 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          VM Runtime State Listener
    id:            76321
    Thread object: java/lang/Thread @ 0xfff61198
    Native info:   !j9vmthread 0x120100  !stack 0x120100
    Daemon:        true
    ID:            12 (0xc)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "VM state notification mutex" with ID 0x7f7798059f48 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-002 Suspended
    id:            76311
    Thread object: java/lang/Thread @ 0xfff57350
    Native info:   !j9vmthread 0x22500  !stack 0x22500
    Daemon:        true
    ID:            4 (0x4)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f779805c468 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-003 Suspended
    id:            76312
    Thread object: java/lang/Thread @ 0xfff575c0
    Native info:   !j9vmthread 0x26100  !stack 0x26100
    Daemon:        true
    ID:            5 (0x5)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f779805c678 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          Concurrent Mark Helper
    id:            76323
    Thread object: java/lang/Thread @ 0xfff62190
    Native info:   !j9vmthread 0x12bd00  !stack 0x12bd00
    Daemon:        true
    ID:            13 (0xd)
    Priority:      1
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ConcurrentGC::conHelpersActivation" with ID 0x7f7798005e58 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-004 Suspended
    id:            76313
    Thread object: java/lang/Thread @ 0xfff57830
    Native info:   !j9vmthread 0x29d00  !stack 0x29d00
    Daemon:        true
    ID:            6 (0x6)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f779805c888 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76324
    Thread object: java/lang/Thread @ 0xfff63178
    Native info:   !j9vmthread 0x12fa00  !stack 0x12fa00
    Daemon:        true
    ID:            14 (0xe)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f77980058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-005 Suspended
    id:            76314
    Thread object: java/lang/Thread @ 0xfff57aa0
    Native info:   !j9vmthread 0x2da00  !stack 0x2da00
    Daemon:        true
    ID:            7 (0x7)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f7770000a38 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76325
    Thread object: java/lang/Thread @ 0xfff64178
    Native info:   !j9vmthread 0x133600  !stack 0x133600
    Daemon:        true
    ID:            15 (0xf)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f77980058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-006 Suspended
    id:            76315
    Thread object: java/lang/Thread @ 0xfff57d48
    Native info:   !j9vmthread 0x31600  !stack 0x31600
    Daemon:        true
    ID:            8 (0x8)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f7770000c48 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76326
    Thread object: java/lang/Thread @ 0xfff65178
    Native info:   !j9vmthread 0x137200  !stack 0x137200
    Daemon:        true
    ID:            16 (0x10)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f77980058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Diagnostic Compilation Thread-007 Suspended
    id:            76316
    Thread object: java/lang/Thread @ 0xfff57fd0
    Native info:   !j9vmthread 0x35200  !stack 0x35200
    Daemon:        true
    ID:            9 (0x9)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f7770000e58 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT-SamplerThread
    id:            76319
    Thread object: java/lang/Thread @ 0xfff58220
    Native info:   !j9vmthread 0x38f00  !stack 0x38f00
    Daemon:        true
    ID:            10 (0xa)
    Priority:      10
    Thread.State:  <no matching state> 
    JVMTI state:   ALIVE WAITING SLEEPING 
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-000
    id:            76309
    Thread object: java/lang/Thread @ 0xfff56e70
    Native info:   !j9vmthread 0x1ac00  !stack 0x1ac00
    Daemon:        true
    ID:            2 (0x2)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "Thread public flags mutex" with ID 0x7f779805c0f8 owner name: <unowned>
    Java stack frames: <no frames to print>

> !classloaderssummary

[Classloaders Summary]
<num, Classloader name                      , # Classloaders, # Loaded Classes, Total Segments, Segments memory>
  0)  *System*                              , 1             , 573             , 35            , 1146880
  1)  java/lang/InternalAnonymousClassLoader, 1             , 0               , 0             , 0
  2)  sun/misc/Launcher$ExtClassLoader      , 1             , 0               , 0             , 0
  3)  sun/misc/Launcher$AppClassLoader      , 1             , 1               , 1             , 32768

<# Loaded Classes, # Classloaders, Classloader Names>
573              , 1             , *System*
1                , 1             , sun/misc/Launcher$AppClassLoader
0                , 2             , java/lang/InternalAnonymousClassLoader, sun/misc/Launcher$ExtClassLoader

Number of Classloaders: 4

Segment Totals: 

<# RAM Segments, RAM Segments memory, # ROM Segments, ROM Segments memory, Total Segments, Segments memory>
 36            , 1179648            , 0             , 0                  , 36            , 1179648
> !dumpallregions
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
|    region      |     start      |      end       |    subspace    | flags  |      size      |      region type     
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
 00007f7798084820 00000000648e0000 0000000064ee0000 00007f7798072130 00000009           600000 ADDRESS_ORDERED
 00007f77980843d0 00000000ffe00000 00000000fff00000 00007f779807e1f0 0000000a           100000 ADDRESS_ORDERED
 00007f7798083f80 00000000fff00000 0000000100000000 00007f7798078750 0000000a           100000 ADDRESS_ORDERED
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
> !dumpallsegments
memorySegments - !j9memorysegmentlist 0x7f779805e130
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f779805e570 00007f7781568030 00007f77815bb980 00007f7781668030 00800040   100000
 00007f779805e4a8 00007f77639e55ac 00007f7763ae8ffc 00007f7763ae8ffc 08000000  3a8fffc
 00007f779805e3e0 00007f7782600000 00007f77829b0000 00007f7783600000 01000440  1000000
 00007f779805e318 00007f7793941030 00007f7793a41020 00007f7793a41030 00800040   100000
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000080281596 (0000000004c8fffc)
Total memory in use:    0000000006321040 (0000000000607390)
Total memory free:      0000000073960556 (0000000004688c6c)

classMemorySegments - !j9memorysegmentlist 0x7f779805eb30
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f7798334b00 000000000017ad28 0000000000182d28 0000000000182d28 00010040     8000
 00007f7798334a38 0000000000172cd8 000000000017acd8 000000000017acd8 00010040     8000
 00007f7798334970 000000000016ac88 0000000000172c88 0000000000172c88 00010040     8000
 00007f77983348a8 0000000000162c38 000000000016ac38 000000000016ac38 00010040     8000
 00007f77983347e0 00007f77601ad050 00007f77601c9a48 00007f7763fffff8 00020104  3e52fa8
 00007f7798334718 000000000015abe8 0000000000162be8 0000000000162be8 00010040     8000
 00007f7798334650 000000000014bae0 0000000000153ae0 0000000000153ae0 00010040     8000
 00007f7798334588 0000000000143a90 000000000014ba90 000000000014ba90 00010040     8000
 00007f77983344c0 00007f776018f5d0 00007f77601ad050 00007f77601ad050 00020104    1da80
 00007f77983343f8 0000000000137de0 000000000013fde0 000000000013fde0 00010040     8000
 00007f77982ad8a0 0000000000120c10 0000000000128c10 0000000000128c10 00010040     8000
 00007f77982ad7d8 0000000000114f60 000000000011cf60 000000000011cf60 00010040     8000
 00007f77982ad710 00007f776016f708 00007f776018f5d0 00007f776018f5d0 00020104    1fec8
 00007f77982ad648 000000000010cf10 0000000000114f10 0000000000114f10 00010040     8000
 00007f77982ad580 0000000000104ec0 000000000010cec0 000000000010cec0 00010040     8000
 00007f77982ad4b8 00000000000fce70 0000000000104e70 0000000000104e70 00010040     8000
 00007f77982ad3f0 00000000000f4e20 00000000000fce20 00000000000fce20 00010040     8000
 00007f77982ad328 00007f77601505e0 00007f776016f708 00007f776016f708 00020104    1f128
 00007f77982ad260 00000000000e5d58 00000000000edd58 00000000000edd58 00010040     8000
 00007f77982ad198 00000000000ddd08 00000000000e5d08 00000000000e5d08 00010040     8000
 00007f77982a56b0 00007f7760130620 00007f77601505e0 00007f77601505e0 00020104    1ffc0
 00007f77982a55e8 00000000000d5cb8 00000000000ddcb8 00000000000ddcb8 00010040     8000
 00007f77982a5520 00000000000cdc68 00000000000d5c68 00000000000d5c68 00010040     8000
 00007f77982a5458 00000000000c5c18 00000000000cdc18 00000000000cdc18 00010040     8000
 00007f77982a5390 00007f7760110c88 00007f7760130620 00007f7760130620 00020104    1f998
 00007f77982a52c8 00000000000bdbc8 00000000000c5bc8 00000000000c5bc8 00010040     8000
 00007f77982a5200 00000000000b5b78 00000000000bdb78 00000000000bdb78 00010040     8000
 00007f77982a5138 00000000000adb28 00000000000b5b28 00000000000b5b28 00010040     8000
 00007f77982a5070 00000000000a5ad8 00000000000adad8 00000000000adad8 00010040     8000
 00007f77982a4fa8 00007f77600f0de0 00007f7760110c88 00007f7760110c88 00020104    1fea8
 00007f779824e1f0 000000000009da88 00000000000a5a88 00000000000a5a88 00010040     8000
 00007f779824e128 0000000000095a38 000000000009da38 000000000009da38 00010040     8000
 00007f779824e060 00007f77600d1168 00007f77600f0de0 00007f77600f0de0 00020104    1fc78
 00007f779824df98 000000000008d9e8 00000000000959e8 00000000000959e8 00010040     8000
 00007f779824ded0 0000000000085998 000000000008d998 000000000008d998 00010040     8000
 00007f779824de08 000000000007d948 0000000000085948 0000000000085948 00010040     8000
 00007f779824dd40 00007f77600b42f0 00007f77600d1168 00007f77600d1168 00020104    1ce78
 00007f779824dc78 00000000000758f8 000000000007d8f8 000000000007d8f8 00010040     8000
 00007f779824dbb0 000000000006d8a8 00000000000758a8 00000000000758a8 00010040     8000
 00007f779824dae8 0000000000065858 000000000006d858 000000000006d858 00010040     8000
 00007f779805f420 00007f7760095748 00007f77600b42f0 00007f77600b42f0 00020104    1eba8
 00007f779805f358 000000000005d808 0000000000065808 0000000000065808 00010040     8000
 00007f779805f290 00000000000557b8 000000000005d7b8 000000000005d7b8 00010040     8000
 00007f779805f1c8 000000000004d768 0000000000055768 0000000000055768 00010040     8000
 00007f779805f100 00007f7760078bd0 00007f7760095748 00007f7760095748 00020104    1cb78
 00007f779805f038 0000000000045718 000000000004d718 000000000004d718 00010040     8000
 00007f779805ef70 00007f779e71ce10 00007f779e71d468 00007f779e71d468 04020104      688
 00007f779805eea8 00007f779e71c7b0 00007f779e71cdd8 00007f779e71cdd8 04200104      658
 00007f779805ede0 000000000003d6c8 00000000000456c8 00000000000456c8 00010040     8000
 00007f779805ed18 00007f7760059000 00007f7760078bd0 00007f7760078bd0 00020104    1fbd0
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000067927256 (00000000040c7cd8)
Total memory in use:    0000000002692808 (00000000002916c8)
Total memory free:      0000000065234448 (0000000003e36610)

jit code segments - !j9memorysegmentlist 0x7f77980a4160
+----------------+----------------+----------------+----------------+----------------+--------+
|    segment     |     start      |    warmAlloc   |    coldAlloc   |      end       |  size  |
+----------------+----------------+----------------+----------------+----------------+--------+
 00007f77980a4348 00007f7783600000 0000000000000000 000000000fff9da0 00007f7793600000 10000000
+----------------+----------------+----------------+----------------+----------------+--------+
Total memory:           0000000268435456 (0000000010000000)
Total memory in use:    0000000000025184 (0000000000006260)
Total memory free:      0000000268410272 (000000000fff9da0)

jit data segments - !j9memorysegmentlist 0x7f77980a45f0
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f77980a48a0 00007f7781669030 00007f7781669090 00007f7781869030 00000048   200000
 00007f77980a47d8 00007f778186a030 00007f7781a6a030 00007f7781a6a030 00000048   200000
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000004194304 (0000000000400000)
Total memory in use:    0000000002097248 (0000000000200060)
Total memory free:      0000000002097056 (00000000001fffa0)


> !findallmodules
This command only works with core file created by VM with Java version 9 or higher
The current VM Java version is: 8
> !gccheck
Starting GC Check
Checking HEAP...done (288 ms).
Checking CLASS HEAP...done (194 ms).
Checking REMEMBERED SET...done (32 ms).
Checking UNFINALIZED...done (1 ms).
Checking FINALIZABLE...done (5 ms).
Checking OWNABLE_SYNCHRONIZER...done (3 ms).
Checking STRING TABLE...done (586 ms).
Checking CLASS LOADERS...done (0 ms).
Checking JNI GLOBAL REFS...done (2 ms).
Checking JNI WEAK GLOBAL REFS...done (2 ms).
Checking JVMTI OBJECT TAG TABLES...done (6 ms).
Checking VM CLASS SLOTS...done (1 ms).
Checking MONITOR TABLE...done (4 ms).
Checking VM THREAD SLOTS...done (284 ms).
Checking THREAD STACKS...done (5 ms).
Done (1419ms)
> !nativememinfo

JRE: 3,244,070,192 bytes / 1910 allocations
|  
+--VM: 2,952,297,072 bytes / 1654 allocations
|  |  
|  +--Classes: 71,656,184 bytes / 182 allocations
|  |  |  
|  |  +--Shared Class Cache: 67,108,960 bytes / 2 allocations
|  |  |  
|  |  +--Other: 4,547,224 bytes / 180 allocations
|  |  
|  +--Memory Manager (GC): 2,660,113,768 bytes / 439 allocations
|  |  |  
|  |  +--Java Heap: 2,608,001,024 bytes / 1 allocation
|  |  |  
|  |  +--Other: 52,112,744 bytes / 438 allocations
|  |  
|  +--Threads: 10,987,360 bytes / 135 allocations
|  |  |  
|  |  +--Java Stack: 197,760 bytes / 16 allocations
|  |  |  
|  |  +--Native Stack: 10,616,832 bytes / 17 allocations
|  |  |  
|  |  +--Other: 172,768 bytes / 102 allocations
|  |  
|  +--Trace: 179,736 bytes / 263 allocations
|  |  
|  +--JVMTI: 17,776 bytes / 13 allocations
|  |  
|  +--JNI: 35,800 bytes / 57 allocations
|  |  
|  +--Port Library: 208,275,040 bytes / 72 allocations
|  |  |  
|  |  +--Unused <32bit allocation regions: 208,264,448 bytes / 1 allocation
|  |  |  
|  |  +--Other: 10,592 bytes / 71 allocations
|  |  
|  +--Other: 1,031,408 bytes / 493 allocations
|  
+--JIT: 291,636,704 bytes / 215 allocations
|  |  
|  +--JIT Code Cache: 268,435,456 bytes / 1 allocation
|  |  
|  +--JIT Data Cache: 4,194,432 bytes / 2 allocations
|  |  
|  +--Other: 19,006,816 bytes / 212 allocations
|  
+--Class Libraries: 136,416 bytes / 41 allocations
|  |  
|  +--VM Class Libraries: 136,416 bytes / 41 allocations
|  |  |  
|  |  +--sun.misc.Unsafe: 528 bytes / 6 allocations
|  |  |  |  
|  |  |  +--Direct Byte Buffers: 528 bytes / 6 allocations
|  |  |  
|  |  +--Other: 135,888 bytes / 35 allocations
> !ramclasssummary
574 classes, using: 1085756 bytes or 1060 kB
Section                                 Byte     kB       %     
  jitVTables                            85176    83.18    7.84  
    Padding                             9184     8.97     0.85  
  ramHeader                             211232   206.28   19.45 
  vTable                                88360    86.29    8.14  
    Padding                             7776     7.59     0.72  
  Extended method block                 8236     8.04     0.76  
  RAM methods                           263552   257.38   24.27 
    J9Method                            263552   257.38   24.27 
  Constant Pool                         394592   385.34   36.34 
    J9CPTYPE_ANNOTATION_UTF8            1024     1.00     0.09  
    J9CPTYPE_CLASS                      115904   113.19   10.67 
    J9CPTYPE_FIELD                      37840    36.95    3.49  
    J9CPTYPE_FLOAT                      448      0.44     0.04  
    J9CPTYPE_HANDLE_METHOD              160      0.16     0.01  
    J9CPTYPE_INSTANCE_METHOD            124576   121.66   11.47 
    J9CPTYPE_INT                        2752     2.69     0.25  
    J9CPTYPE_INTERFACE_METHOD           12096    11.81    1.11  
    J9CPTYPE_METHODHANDLE               576      0.56     0.05  
    J9CPTYPE_METHOD_TYPE                352      0.34     0.03  
    J9CPTYPE_STATIC_METHOD              46960    45.86    4.33  
    J9CPTYPE_STRING                     43184    42.17    3.98  
    J9CPTYPE_UNUSED                     8352     8.16     0.77  
  Ram static                            11704    11.43    1.08  
  Superclasses                          6992     6.83     0.64  
  iTable                                15912    15.54    1.47  
> !romclasssummary
531 classes, using: 1657022 bytes or 1618 kB
Section                                 Byte     kB       %     
  romHeader                             89208    87.12    5.38  
  constantPool                          198360   193.71   11.97 
  fields                                33000    32.23    1.99  
    field                               33000    32.23    1.99  
      fieldAnnotation                   36       0.04     0.00  
  interfacesSRPs                        964      0.94     0.06  
  innerClassesSRPs                      1276     1.25     0.08  
  enclosedInnerClassesSRPs              1452     1.42     0.09  
  cpNamesAndSignaturesSRPs              99032    96.71    5.98  
    Padding                             184      0.18     0.01  
  methods                               681960   665.98   41.16 
    method                              681960   665.98   41.16 
      methodAnnotation                  2400     2.34     0.14  
      methodBytecodes                   354556   346.25   21.40 
      methodDebugInfo out of line       28100    27.44    1.70  
      stackMap                          89668    87.57    5.41  
        Padding                         4788     4.68     0.29  
  cpShapeDescription                    13408    13.09    0.81  
  classAnnotations                      84       0.08     0.01  
  enclosingObject                       224      0.22     0.01  
  optionalInfo                          3200     3.13     0.19  
  UTF8                                  373558   364.80   22.54 
    Padding                             820      0.80     0.05  
  Padding                               15858    15.49    0.96  
  methodDebugInfos                      145438   142.03   8.78  
    methodDebugInfo                     145438   142.03   8.78  
> !vmcheck
<vm check: Checking threads>
<vm check: Checking 16 threads done>
<vm check: Checking classes>
<vm check: Checking 574 classes (0 obsolete) done>
<vm check: Checking ROM classes>
<vm check: Checking 574 ROM classes done>
<vm check: Checking methods>
<vm check: Checking 8236 methods done>
<vm check: Checking ROM intern string nodes>
<vm check: Checking 0 ROM intern string nodes done>

When kernel.core_pattern=core

root@ee4cd086c862:/openj9-openjdk-jdk8# sysctl kernel.core_pattern
kernel.core_pattern = core
root@ee4cd086c862:/openj9-openjdk-jdk8# ( cd build/linux-x86_64-normal-server-release/images/j2sdk-image; printf 'public class Test { public static void main(String... args) { System.out.println("Hello World"); } }' > Test.java && ./bin/javac Test.java && ./bin/java -Xshareclasses:name=test -Xdump:system:events=vmstop,request=exclusive+prepwalk Test )
Hello World
JVMDUMP039I Processing dump event "vmstop", detail "#0000000000000000" at 2022/12/06 21:46:19 - please wait.
JVMDUMP032I JVM requested System dump using '/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214619.76384.0001.dmp' in response to an event
JVMDUMP010I System dump written to /openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214619.76384.0001.dmp
JVMDUMP013I Processed dump event "vmstop", detail "#0000000000000000".
jdmpview output looks good
root@ee4cd086c862:/openj9-openjdk-jdk8# ( cd build/linux-x86_64-normal-server-release/images/j2sdk-image; ./bin/jdmpview -core core*dmp )
DTFJView version 4.29.5, using DTFJ version 1.12.29003
Loading image from DTFJ...

For a list of commands, type "help"; for how to use "help", type "help help"
Available contexts (* = currently selected context) : 

Source : file:///openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214619.76384.0001.dmp
	*0 : PID: 76405 : JRE 1.8.0 Linux amd64-64 (build 1.8.0_362-internal-_2022_12_06_20_25-b00)

> !shrc stats
!j9sharedclassconfig 0x00007F38CC0B81E0

!j9sharedcacheheader 0x00007F38900000F0    (layer 0)
cache size       : 66744320
free bytes       : 58818680
softmx bytes     : -1
read write area  : 0x00007F3890000260 - 0x00007F3890000260 size 364304 used 0
segment area     : 0x00007F3890059000 - 0x00007F38901C9A48 size 1509960
metadata area    : 0x00007F38939E1AC0 - 0x00007F3893AE9000 size 1078592
class debug area : 0x00007F3893AE9000 - 0x00007F3894000000 size 5337088 used 145438 (% used 2)
Meta data region to be used: 0x00007F38939E1AC0..0x00007F3893AE8FFC
No entry found in the cache

Cache contains 523 classes, 1 orphans, 2 classpaths, 0 URLs, 0 tokens
28 AOT, 71 SCOPES, 54 BYTE data, 0 UNINDEXED DATA, 0 CHARARRAY, 1 stale
stale bytes 44
27 JITPROFILE, 13 JITHINT
AOT data length 45864 code length 43392 metadata 784 total 89256
JITPROFILE data length 1976 metadata 864 
JITHINT data length 52 metadata 416 
ROMClass data 1509960 metadata 23032
SCOPE data 2467 metadata 852 total 3319
BYTE data 955240 metadata 1944 rwarea 0
UNINDEXEDBYTE data 0 metadata 0
CHARARRAY data 0 metadata 0
BYTEDATA Summary
	UNKNOWN 0  HELPER 0  POOL 0  AOTHEADER 264
	JCL 0  VM 0  ROMSTRING 0  ZIPCACHE 953152  STARTUPHINTS 0
	JITHINT 0  AOTCLASSCHAIN 1624 AOTTHUNK 200
DEBUG Area Summary
	LineNumberTable bytes    : 145438
	LocalVariableTable bytes : 0
> info thread *
native threads for address space
 process id: 76384

  thread id: 76385
   registers:
   native stack sections:
   native stack frames:
   properties:
   associated Java thread: 
    name:          DestroyJavaVM helper thread
    Thread object: java/lang/Thread @ 0xfff7d198
    Native info:   !j9vmthread 0x0  !stack 0x0
    Daemon:        false
    ID:            19 (0x13)
    Priority:      5
    Thread.State:  RUNNABLE 
    JVMTI state:   ALIVE RUNNABLE 
    Java stack frames: <no frames to print>


  thread id: 76405
   registers:
    cs     = 0x0000000000000033   ds     = 0x0000000000000000   eflags = 0x0000000000000282   es     = 0x0000000000000000
    fs     = 0x0000000000000000   fs_base = 0x00007f38d14a0700   gs     = 0x0000000000000000   gs_base = 0x0000000000000000
    r8     = 0x0000000000000000   r9     = 0x0000000000000000   r10    = 0x0000000000000008   r11    = 0x0000000000000282
    r12    = 0x00007f38d06703a0   r13    = 0x00007f38d149f2f0   r14    = 0x00007f38cc0246a0   r15    = 0x00007f38d149f770
    rax    = 0x0000000000000000   rbp    = 0x0000000000000000   rbx    = 0x00007f38d149f2a0   rcx    = 0x00007f38d089259f
    rdi    = 0x0000000000012a75   rdx    = 0x000000000000000b   rip    = 0x00007f38d089259f   rsi    = 0x0000000000012a75
    rsp    = 0x00007f38d149eda8   ss     = 0x000000000000002b
   native stack sections:
    0x7f38d1461000 to 0x7f38d14a1000 (length 0x40000)
   native stack frames:
    bp: 0x00007f38d149edb0 pc: 0x00007f38d089259f libpthread.so.0::pthread_kill+0x2f
    bp: 0x00007f38d149edf0 pc: 0x00007f38cb4a741d libj9prt29.so::j9port_allocate_library+0x1becd
    bp: 0x00007f38d149f060 pc: 0x00007f38cb0392b2 libj9dmp29.so+0x82b2
    bp: 0x00007f38d149f070 pc: 0x00007f38cb0350c5 libj9dmp29.so+0x40c5
    bp: 0x00007f38d149f1d0 pc: 0x00007f38cb4a8f03 libj9prt29.so::j9port_allocate_library+0x1d9b3
    bp: 0x00007f38d149f230 pc: 0x00007f38cb03871b libj9dmp29.so+0x771b
    bp: 0x00007f38d149f6f0 pc: 0x00007f38cb0388af libj9dmp29.so+0x78af
    bp: 0x00007f38d149fb40 pc: 0x00007f38cb051366 libj9dmp29.so::J9VMDllMain+0x16056
    bp: 0x00007f38d149fbb0 pc: 0x00007f38cb04f2a6 libj9dmp29.so::J9VMDllMain+0x13f96
    bp: 0x00007f38d149fc50 pc: 0x00007f38cba0039e libj9hookable29.so+0x139e
    bp: 0x00007f38d149fcc0 pc: 0x00007f38cbc506fb libj9vm29.so::J9_GetCreatedJavaVMs+0x119b
    bp: 0x00007f38d149fe20 pc: 0x00007f38cb4a8f03 libj9prt29.so::j9port_allocate_library+0x1d9b3
    bp: 0x00007f38d149fea0 pc: 0x00007f38cbc4faf9 libj9vm29.so::J9_GetCreatedJavaVMs+0x599
    bp: 0x00007f38d149feb0 pc: 0x00007f38d0446cfa libjvm.so::JVM_IsUseContainerSupport+0x2da
    bp: 0x00007f38d149ff60 pc: 0x00007f38d1077708 libjli.so+0x8708
    bp: 0x00007f38d14a0000 pc: 0x00007f38d088b6ba libpthread.so.0::__pthread_get_minstack+0x164a
    bp: 0x00007f38d14a0008 pc: 0x00007f38d0dac51d libc.so.6::clone+0x6d
   properties:
    Thread sys time secs=0    Thread sys time usecs=0
    Thread user time secs=0   Thread user time usecs=2000
   associated Java thread: <no associated Java thread>

Java threads not associated with known native threads:

    name:          JIT Compilation Thread-003 Suspended
    id:            76390
    Thread object: java/lang/Thread @ 0xfff575c0
    Native info:   !j9vmthread 0x26100  !stack 0x26100
    Daemon:        true
    ID:            5 (0x5)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f38cc05c648 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-004 Suspended
    id:            76391
    Thread object: java/lang/Thread @ 0xfff57830
    Native info:   !j9vmthread 0x29d00  !stack 0x29d00
    Daemon:        true
    ID:            6 (0x6)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f38cc05c858 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-005 Suspended
    id:            76392
    Thread object: java/lang/Thread @ 0xfff57aa0
    Native info:   !j9vmthread 0x2da00  !stack 0x2da00
    Daemon:        true
    ID:            7 (0x7)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f389c000a38 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-006 Suspended
    id:            76393
    Thread object: java/lang/Thread @ 0xfff57d48
    Native info:   !j9vmthread 0x31600  !stack 0x31600
    Daemon:        true
    ID:            8 (0x8)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f389c000c48 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Diagnostic Compilation Thread-007 Suspended
    id:            76394
    Thread object: java/lang/Thread @ 0xfff57fd0
    Native info:   !j9vmthread 0x35200  !stack 0x35200
    Daemon:        true
    ID:            9 (0x9)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f389c000e58 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT-SamplerThread
    id:            76395
    Thread object: java/lang/Thread @ 0xfff58220
    Native info:   !j9vmthread 0x38f00  !stack 0x38f00
    Daemon:        true
    ID:            10 (0xa)
    Priority:      10
    Thread.State:  <no matching state> 
    JVMTI state:   ALIVE WAITING SLEEPING 
    Java stack frames: <no frames to print>

    name:          IProfiler
    id:            76396
    Thread object: java/lang/Thread @ 0xfff585f0
    Native info:   !j9vmthread 0x3cb00  !stack 0x3cb00
    Daemon:        true
    ID:            11 (0xb)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-iprofilerMonitor" with ID 0x7f389c002198 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          VM Runtime State Listener
    id:            76397
    Thread object: java/lang/Thread @ 0xfff61198
    Native info:   !j9vmthread 0x120100  !stack 0x120100
    Daemon:        true
    ID:            12 (0xc)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "VM state notification mutex" with ID 0x7f38cc059f18 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-000
    id:            76387
    Thread object: java/lang/Thread @ 0xfff56e70
    Native info:   !j9vmthread 0x1ac00  !stack 0x1ac00
    Daemon:        true
    ID:            2 (0x2)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "Thread public flags mutex" with ID 0x7f38cc05c0c8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-001 Suspended
    id:            76388
    Thread object: java/lang/Thread @ 0xfff570e0
    Native info:   !j9vmthread 0x1e800  !stack 0x1e800
    Daemon:        true
    ID:            3 (0x3)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f38cc05c228 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          Concurrent Mark Helper
    id:            76399
    Thread object: java/lang/Thread @ 0xfff62190
    Native info:   !j9vmthread 0x12bd00  !stack 0x12bd00
    Daemon:        true
    ID:            13 (0xd)
    Priority:      1
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ConcurrentGC::conHelpersActivation" with ID 0x7f38cc005e58 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          JIT Compilation Thread-002 Suspended
    id:            76389
    Thread object: java/lang/Thread @ 0xfff57350
    Native info:   !j9vmthread 0x22500  !stack 0x22500
    Daemon:        true
    ID:            4 (0x4)
    Priority:      10
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "JIT-CompThreadMonitor-??" with ID 0x7f38cc05c438 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76400
    Thread object: java/lang/Thread @ 0xfff63178
    Native info:   !j9vmthread 0x12fa00  !stack 0x12fa00
    Daemon:        true
    ID:            14 (0xe)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f38cc0058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76401
    Thread object: java/lang/Thread @ 0xfff64178
    Native info:   !j9vmthread 0x133600  !stack 0x133600
    Daemon:        true
    ID:            15 (0xf)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f38cc0058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

    name:          GC Worker
    id:            76402
    Thread object: java/lang/Thread @ 0xfff65178
    Native info:   !j9vmthread 0x137200  !stack 0x137200
    Daemon:        true
    ID:            16 (0x10)
    Priority:      5
    Thread.State:  WAITING 
    JVMTI state:   ALIVE WAITING WAITING_INDEFINITELY IN_OBJECT_WAIT 
      waiting to be notified on: "MM_ParallelDispatcher::workerThread" with ID 0x7f38cc0058d8 owner name: <unowned>
    Java stack frames: <no frames to print>

> !classloaderssummary

[Classloaders Summary]
<num, Classloader name                      , # Classloaders, # Loaded Classes, Total Segments, Segments memory>
  0)  *System*                              , 1             , 573             , 35            , 1146880
  1)  java/lang/InternalAnonymousClassLoader, 1             , 0               , 0             , 0
  2)  sun/misc/Launcher$ExtClassLoader      , 1             , 0               , 0             , 0
  3)  sun/misc/Launcher$AppClassLoader      , 1             , 1               , 1             , 32768

<# Loaded Classes, # Classloaders, Classloader Names>
573              , 1             , *System*
1                , 1             , sun/misc/Launcher$AppClassLoader
0                , 2             , java/lang/InternalAnonymousClassLoader, sun/misc/Launcher$ExtClassLoader

Number of Classloaders: 4

Segment Totals: 

<# RAM Segments, RAM Segments memory, # ROM Segments, ROM Segments memory, Total Segments, Segments memory>
 36            , 1179648            , 0             , 0                  , 36            , 1179648
> !dumpallregions
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
|    region      |     start      |      end       |    subspace    | flags  |      size      |      region type     
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
 00007f38cc0847f0 00000000648e0000 0000000064ee0000 00007f38cc072100 00000009           600000 ADDRESS_ORDERED
 00007f38cc0843a0 00000000ffe00000 00000000fff00000 00007f38cc07e1c0 0000000a           100000 ADDRESS_ORDERED
 00007f38cc083f50 00000000fff00000 0000000100000000 00007f38cc078720 0000000a           100000 ADDRESS_ORDERED
+----------------+----------------+----------------+----------------+--------+----------------+----------------------
> !dumpallsegments
memorySegments - !j9memorysegmentlist 0x7f38cc05e100
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f38cc05e540 00007f38a6c61030 00007f38a6cb82d0 00007f38a6d61030 00800040   100000
 00007f38cc05e478 00007f38939e1ac0 00007f3893ae8ffc 00007f3893ae8ffc 08000000  3a8fffc
 00007f38cc05e3b0 00007f38b4355000 00007f38b43d5000 00007f38b5355000 01000440  1000000
 00007f38cc05e2e8 00007f38c5456030 00007f38c5556020 00007f38c5556030 00800040   100000
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000080281596 (0000000004c8fffc)
Total memory in use:    0000000003008460 (00000000002de7cc)
Total memory free:      0000000077273136 (00000000049b1830)

classMemorySegments - !j9memorysegmentlist 0x7f38cc05eb00
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f38cc130190 000000000017ad28 0000000000182d28 0000000000182d28 00010040     8000
 00007f38cc1300c8 0000000000172cd8 000000000017acd8 000000000017acd8 00010040     8000
 00007f38cc130000 000000000016ac88 0000000000172c88 0000000000172c88 00010040     8000
 00007f38cc12ff38 0000000000162c38 000000000016ac38 000000000016ac38 00010040     8000
 00007f38cc12fe70 000000000015abe8 0000000000162be8 0000000000162be8 00010040     8000
 00007f38cc12fda8 000000000014bae0 0000000000153ae0 0000000000153ae0 00010040     8000
 00007f38cc12fce0 0000000000143a90 000000000014ba90 000000000014ba90 00010040     8000
 00007f38cc12fc18 0000000000137de0 000000000013fde0 000000000013fde0 00010040     8000
 00007f38cc12fb50 0000000000120c10 0000000000128c10 0000000000128c10 00010040     8000
 00007f38cc12fa88 0000000000114f60 000000000011cf60 000000000011cf60 00010040     8000
 00007f38cc103c80 000000000010cf10 0000000000114f10 0000000000114f10 00010040     8000
 00007f38cc103bb8 0000000000104ec0 000000000010cec0 000000000010cec0 00010040     8000
 00007f38cc103af0 00000000000fce70 0000000000104e70 0000000000104e70 00010040     8000
 00007f38cc103a28 00000000000f4e20 00000000000fce20 00000000000fce20 00010040     8000
 00007f38cc103960 00000000000e5d58 00000000000edd58 00000000000edd58 00010040     8000
 00007f38cc103898 00000000000ddd08 00000000000e5d08 00000000000e5d08 00010040     8000
 00007f38cc1037d0 00000000000d5cb8 00000000000ddcb8 00000000000ddcb8 00010040     8000
 00007f38cc103708 00000000000cdc68 00000000000d5c68 00000000000d5c68 00010040     8000
 00007f38cc103640 00000000000c5c18 00000000000cdc18 00000000000cdc18 00010040     8000
 00007f38cc103578 00000000000bdbc8 00000000000c5bc8 00000000000c5bc8 00010040     8000
 00007f38cc0fa0b0 00000000000b5b78 00000000000bdb78 00000000000bdb78 00010040     8000
 00007f38cc0f9fe8 00000000000adb28 00000000000b5b28 00000000000b5b28 00010040     8000
 00007f38cc0f9f20 00000000000a5ad8 00000000000adad8 00000000000adad8 00010040     8000
 00007f38cc0f9e58 000000000009da88 00000000000a5a88 00000000000a5a88 00010040     8000
 00007f38cc0f9d90 0000000000095a38 000000000009da38 000000000009da38 00010040     8000
 00007f38cc0f9cc8 000000000008d9e8 00000000000959e8 00000000000959e8 00010040     8000
 00007f38cc0f9c00 0000000000085998 000000000008d998 000000000008d998 00010040     8000
 00007f38cc0f9b38 000000000007d948 0000000000085948 0000000000085948 00010040     8000
 00007f38cc0f9a70 00000000000758f8 000000000007d8f8 000000000007d8f8 00010040     8000
 00007f38cc0f99a8 000000000006d8a8 00000000000758a8 00000000000758a8 00010040     8000
 00007f38cc0e4f50 0000000000065858 000000000006d858 000000000006d858 00010040     8000
 00007f38cc0e4e88 000000000005d808 0000000000065808 0000000000065808 00010040     8000
 00007f38cc0e4dc0 00000000000557b8 000000000005d7b8 000000000005d7b8 00010040     8000
 00007f38cc0e4cf8 000000000004d768 0000000000055768 0000000000055768 00010040     8000
 00007f38cc0e4c30 0000000000045718 000000000004d718 000000000004d718 00010040     8000
 00007f38cc0e4b68 00007f38cbffee10 00007f38cbfff468 00007f38cbfff468 04020104      688
 00007f38cc0e4aa0 00007f38cbffe7b0 00007f38cbffedd8 00007f38cbffedd8 04200104      658
 00007f38cc0e49d8 000000000003d6c8 00000000000456c8 00000000000456c8 00010040     8000
 00007f38cc0e4910 00007f38901ad050 00007f38901c9a48 00007f3893fffff8 00020104  3e52fa8
 00007f38cc0e4848 00007f389018f5d0 00007f38901ad050 00007f38901ad050 00020104    1da80
 00007f38cc05f3f0 00007f389016f708 00007f389018f5d0 00007f389018f5d0 00020104    1fec8
 00007f38cc05f328 00007f38901505e0 00007f389016f708 00007f389016f708 00020104    1f128
 00007f38cc05f260 00007f3890130620 00007f38901505e0 00007f38901505e0 00020104    1ffc0
 00007f38cc05f198 00007f3890110c88 00007f3890130620 00007f3890130620 00020104    1f998
 00007f38cc05f0d0 00007f38900f0de0 00007f3890110c88 00007f3890110c88 00020104    1fea8
 00007f38cc05f008 00007f38900d1168 00007f38900f0de0 00007f38900f0de0 00020104    1fc78
 00007f38cc05ef40 00007f38900b42f0 00007f38900d1168 00007f38900d1168 00020104    1ce78
 00007f38cc05ee78 00007f3890095748 00007f38900b42f0 00007f38900b42f0 00020104    1eba8
 00007f38cc05edb0 00007f3890078bd0 00007f3890095748 00007f3890095748 00020104    1cb78
 00007f38cc05ece8 00007f3890059000 00007f3890078bd0 00007f3890078bd0 00020104    1fbd0
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000067927256 (00000000040c7cd8)
Total memory in use:    0000000002692808 (00000000002916c8)
Total memory free:      0000000065234448 (0000000003e36610)

jit code segments - !j9memorysegmentlist 0x7f38cc0a4140
+----------------+----------------+----------------+----------------+----------------+--------+
|    segment     |     start      |    warmAlloc   |    coldAlloc   |      end       |  size  |
+----------------+----------------+----------------+----------------+----------------+--------+
 00007f38cc0a4328 00007f38b5456000 0000000000000000 000000000fff0d60 00007f38c5456000 10000000
+----------------+----------------+----------------+----------------+----------------+--------+
Total memory:           0000000268435456 (0000000010000000)
Total memory in use:    0000000000062112 (000000000000f2a0)
Total memory free:      0000000268373344 (000000000fff0d60)

jit data segments - !j9memorysegmentlist 0x7f38cc0a45d0
+----------------+----------------+----------------+----------------+--------+--------+
|    segment     |     start      |     alloc      |      end       |  type  |  size  |
+----------------+----------------+----------------+----------------+--------+--------+
 00007f38cc0a4880 00007f38a6f9b030 00007f38a6f9b030 00007f38a719b030 00000048   200000
 00007f38cc0a47b8 00007f38a719c030 00007f38a739c030 00007f38a739c030 00000048   200000
+----------------+----------------+----------------+----------------+--------+--------+
Total memory:           0000000004194304 (0000000000400000)
Total memory in use:    0000000002097152 (0000000000200000)
Total memory free:      0000000002097152 (0000000000200000)


> !findallmodules
This command only works with core file created by VM with Java version 9 or higher
The current VM Java version is: 8
> !gccheck
Starting GC Check
Checking HEAP...done (347 ms).
Checking CLASS HEAP...done (218 ms).
Checking REMEMBERED SET...done (61 ms).
Checking UNFINALIZED...done (2 ms).
Checking FINALIZABLE...done (6 ms).
Checking OWNABLE_SYNCHRONIZER...done (4 ms).
Checking STRING TABLE...done (690 ms).
Checking CLASS LOADERS...done (0 ms).
Checking JNI GLOBAL REFS...done (2 ms).
Checking JNI WEAK GLOBAL REFS...done (1 ms).
Checking JVMTI OBJECT TAG TABLES...done (5 ms).
Checking VM CLASS SLOTS...done (2 ms).
Checking MONITOR TABLE...done (4 ms).
Checking VM THREAD SLOTS...done (285 ms).
Checking THREAD STACKS...done (8 ms).
Done (1645ms)
> !nativememinfo

JRE: 3,243,680,568 bytes / 1842 allocations
|  
+--VM: 2,951,980,480 bytes / 1652 allocations
|  |  
|  +--Classes: 71,340,728 bytes / 181 allocations
|  |  |  
|  |  +--Shared Class Cache: 67,108,960 bytes / 2 allocations
|  |  |  
|  |  +--Other: 4,231,768 bytes / 179 allocations
|  |  
|  +--Memory Manager (GC): 2,660,113,768 bytes / 439 allocations
|  |  |  
|  |  +--Java Heap: 2,608,001,024 bytes / 1 allocation
|  |  |  
|  |  +--Other: 52,112,744 bytes / 438 allocations
|  |  
|  +--Threads: 10,987,360 bytes / 135 allocations
|  |  |  
|  |  +--Java Stack: 197,760 bytes / 16 allocations
|  |  |  
|  |  +--Native Stack: 10,616,832 bytes / 17 allocations
|  |  |  
|  |  +--Other: 172,768 bytes / 102 allocations
|  |  
|  +--Trace: 179,736 bytes / 263 allocations
|  |  
|  +--JVMTI: 17,776 bytes / 13 allocations
|  |  
|  +--JNI: 35,800 bytes / 57 allocations
|  |  
|  +--Port Library: 208,275,040 bytes / 72 allocations
|  |  |  
|  |  +--Unused <32bit allocation regions: 208,264,448 bytes / 1 allocation
|  |  |  
|  |  +--Other: 10,592 bytes / 71 allocations
|  |  
|  +--Other: 1,030,272 bytes / 492 allocations
|  
+--JIT: 291,627,736 bytes / 150 allocations
|  |  
|  +--JIT Code Cache: 268,435,456 bytes / 1 allocation
|  |  
|  +--JIT Data Cache: 4,194,432 bytes / 2 allocations
|  |  
|  +--Other: 18,997,848 bytes / 147 allocations
|  
+--Class Libraries: 72,352 bytes / 40 allocations
|  |  
|  +--VM Class Libraries: 72,352 bytes / 40 allocations
|  |  |  
|  |  +--sun.misc.Unsafe: 528 bytes / 6 allocations
|  |  |  |  
|  |  |  +--Direct Byte Buffers: 528 bytes / 6 allocations
|  |  |  
|  |  +--Other: 71,824 bytes / 34 allocations
> !ramclasssummary
574 classes, using: 1085756 bytes or 1060 kB
Section                                 Byte     kB       %     
  jitVTables                            85176    83.18    7.84  
    Padding                             9184     8.97     0.85  
  ramHeader                             211232   206.28   19.45 
  vTable                                88360    86.29    8.14  
    Padding                             7776     7.59     0.72  
  Extended method block                 8236     8.04     0.76  
  RAM methods                           263552   257.38   24.27 
    J9Method                            263552   257.38   24.27 
  Constant Pool                         394592   385.34   36.34 
    J9CPTYPE_ANNOTATION_UTF8            1024     1.00     0.09  
    J9CPTYPE_CLASS                      115904   113.19   10.67 
    J9CPTYPE_FIELD                      37840    36.95    3.49  
    J9CPTYPE_FLOAT                      448      0.44     0.04  
    J9CPTYPE_HANDLE_METHOD              160      0.16     0.01  
    J9CPTYPE_INSTANCE_METHOD            124576   121.66   11.47 
    J9CPTYPE_INT                        2752     2.69     0.25  
    J9CPTYPE_INTERFACE_METHOD           12096    11.81    1.11  
    J9CPTYPE_METHODHANDLE               576      0.56     0.05  
    J9CPTYPE_METHOD_TYPE                352      0.34     0.03  
    J9CPTYPE_STATIC_METHOD              46960    45.86    4.33  
    J9CPTYPE_STRING                     43184    42.17    3.98  
    J9CPTYPE_UNUSED                     8352     8.16     0.77  
  Ram static                            11704    11.43    1.08  
  Superclasses                          6992     6.83     0.64  
  iTable                                15912    15.54    1.47  
> !romclasssummary
531 classes, using: 1657022 bytes or 1618 kB
Section                                 Byte     kB       %     
  romHeader                             89208    87.12    5.38  
  constantPool                          198360   193.71   11.97 
  fields                                33000    32.23    1.99  
    field                               33000    32.23    1.99  
      fieldAnnotation                   36       0.04     0.00  
  interfacesSRPs                        964      0.94     0.06  
  innerClassesSRPs                      1276     1.25     0.08  
  enclosedInnerClassesSRPs              1452     1.42     0.09  
  cpNamesAndSignaturesSRPs              99032    96.71    5.98  
    Padding                             184      0.18     0.01  
  methods                               681960   665.98   41.16 
    method                              681960   665.98   41.16 
      methodAnnotation                  2400     2.34     0.14  
      methodBytecodes                   354556   346.25   21.40 
      methodDebugInfo out of line       28100    27.44    1.70  
      stackMap                          89668    87.57    5.41  
        Padding                         4788     4.68     0.29  
  cpShapeDescription                    13408    13.09    0.81  
  classAnnotations                      84       0.08     0.01  
  enclosingObject                       224      0.22     0.01  
  optionalInfo                          3200     3.13     0.19  
  UTF8                                  373558   364.80   22.54 
    Padding                             820      0.80     0.05  
  Padding                               15858    15.49    0.96  
  methodDebugInfos                      145438   142.03   8.78  
    methodDebugInfo                     145438   142.03   8.78  
> !vmcheck
<vm check: Checking threads>
<vm check: Checking 16 threads done>
<vm check: Checking classes>
<vm check: Checking 574 classes (0 obsolete) done>
<vm check: Checking ROM classes>
<vm check: Checking 574 ROM classes done>
<vm check: Checking methods>
<vm check: Checking 8236 methods done>
<vm check: Checking ROM intern string nodes>
<vm check: Checking 0 ROM intern string nodes done>

@kgibm
Copy link
Contributor Author

kgibm commented Jan 11, 2023

Not sure who to follow-up on this with. Perhaps @babsingh, can you please review?

Copy link
Contributor

@babsingh babsingh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Highlighted places in the modified code where the coding standard is not satisfied, and minor nitpicks.

port/linux/omrosdump_helpers.c Outdated Show resolved Hide resolved
port/linux/omrosdump_helpers.c Outdated Show resolved Hide resolved
port/linux/omrosdump_helpers.c Outdated Show resolved Hide resolved
@babsingh
Copy link
Contributor

The commit message and PR description do not follow the Commit Guidelines: https://github.com/eclipse/omr/blob/master/CONTRIBUTING.md#commit-guidelines.

The first line should be less than 70 characters ...

The body should be wrapped at 72 characters ...

@babsingh
Copy link
Contributor

babsingh commented Jan 11, 2023

re #6838 (comment):

JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.76329.  Review the manual for the external program to find where the core dump is written and ensure the program does not truncate it.

JVMPORT049I The core file created by child process with pid = 76329 was not found. Review the documentation for the /proc/sys/kernel/core_pattern program "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" to find where the core file is written and ensure that program does not truncate it.

JVMDUMP012E Error in System dump: /openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214232.76306.0001.dmp
  • Are the above three lines the new messages?
  • In the first line, there is an extra space after the full-stop/period. e.g. an external program. Attempting to and core.76329. Review. Can the extra space be removed?
  • Before and after the second message, there is a new line. Existing behaviour doesn't have new lines between messages. Can the new lines be removed?
  • There is redundant content in the first and second message: Review the ... does not truncate it. Is it possible to remove the redundancy?

@babsingh
Copy link
Contributor

babsingh commented Jan 11, 2023

Preliminary testing to verify the functionality of the changes.

jenkins build all

@babsingh
Copy link
Contributor

I also recommend to run an OpenJ9 personal build since some OpenJ9 tests look at the output of core file generation. The following TEST_TARGETS should suffice: sanity.functional, extended.functional, sanity.system and sanity.openjdk.

@keithc-ca
Copy link
Member

  • there is an extra space after the full-stop/period. e.g. an external program. Attempting to and core.76329. Review. Can the extra space be removed?

It's convention to use two spaces between sentences, so I think the "extra space" should stay.

@babsingh
Copy link
Contributor

It's convention to use two spaces between sentences, so I think the "extra space" should stay.

Then, we should add two spaces in the below message: found. Review.

JVMPORT049I The core file created by child process with pid = 76329 was not found. Review the documentation for the /proc/sys/kernel/core_pattern program "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" to find where the core file is written and ensure that program does not truncate it.

@keithc-ca
Copy link
Member

Then, we should add two spaces in the below message: found. Review.

That is a matter for the OpenJ9 project (the text you quote is not in this repository).

Did you mean to suggest an extra space on line 240? If so, I agree.

@kgibm
Copy link
Contributor Author

kgibm commented Jan 17, 2023

@babsingh

re #6838 (comment):

JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.76329.  Review the manual for the external program to find where the core dump is written and ensure the program does not truncate it.

JVMPORT049I The core file created by child process with pid = 76329 was not found. Review the documentation for the /proc/sys/kernel/core_pattern program "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" to find where the core file is written and ensure that program does not truncate it.

JVMDUMP012E Error in System dump: /openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20221206.214232.76306.0001.dmp
* Are the above three lines the new messages?

The first message, JVMPORT030W, is not new, but its last sentence is new. The second message, JVMPORT049I, is the new message. The third message, JVMDUMP012E, is not new and is not modified.

However, I think JVMDUMP012E is not correct as the JVM does not know whether or not there was an error in the system dump, and I don't think it should assume there was an error. I see that openj9 can handle a 0 return code for dump_create and a NULL file name and it will print a JVMDUMP010I System dump written to message with "{unable to determine dump name}". However, this also might be deceptive because the JVM doesn't know that the dump was written successfully. Personally, I think it's better to err on the side of assuming success as so many systems move towards core-dump processing programs which do often work. What do you think? CC @keithc-ca @pshipton if you have any thoughts. In summary, I would like to change the behavior as follows:

If core_pattern is a piped core-dump processing program, then, in addition to printing JVMPORT049I, print JVMDUMP010I with "{unable to determine dump name}" instead of JVMDUMP012E.

* In the first line, there is an extra space after the full-stop/period. e.g. `an external program.  Attempting to` and `core.76329.  Review`. Can the extra space be removed?

As per the other comments, I'll update to double-spaces.

* Before and after the second message, there is a new line. Existing behaviour doesn't have new lines between messages. Can the new lines be removed?

These messages are in the openj9 project. I agree that it would be better without the newlines, although I do worry they were there for some reason because no other messages in that messages file have newlines, so I'd rather just avoid changing them in case it might break something.

* There is redundant content in the first and second message: `Review the ... does not truncate it`. Is it possible to remove the redundancy?

I agree in general, although the underlying issue is that end users are misunderstanding these messages and thinking core dumps aren't being produced in these cases, so I'd rather lean towards some redundancy. For example, I see support cases where just one of the messages related to producing the dump is extracted out of the logs and put in the case instead of all the messages. So I'd rather keep the redundancy, but if others feel strongly otherwise, I can remove it (although these messages are in the openj9 project so I'll need a separate PR).

@kgibm
Copy link
Contributor Author

kgibm commented Jan 17, 2023

@babsingh

I also recommend to run an OpenJ9 personal build since some OpenJ9 tests look at the output of core file generation. The following TEST_TARGETS should suffice: sanity.functional, extended.functional, sanity.system and sanity.openjdk.

Thanks, it looks like you ran this test. I see one test failed (threadtest): https://ci.eclipse.org/omr/job/PullRequest-linux_riscv64_cross/960/consoleText

[2023-01-11T22:35:28.100Z] 21: /home/jenkins/workspace/Build/fvtest/threadtest/rwMutexTest.cpp:708: Failure
[2023-01-11T22:35:28.100Z] 21: Value of: 1 == infoReader2->readCounter
[2023-01-11T22:35:28.100Z] 21:   Actual: false
[2023-01-11T22:35:28.100Z] 21: Expected: true
[2023-01-11T22:35:28.100Z] 21: [  FAILED  ] RWMutex.AllReadersProceedTest (3015 ms)

Looks like this is a known issue: #6704

@kgibm
Copy link
Contributor Author

kgibm commented Jan 17, 2023

I did a preliminary test with the additional proposed change and it works as I guessed:

root@2a950dfabdff:/openj9-openjdk-jdk8# sysctl kernel.core_pattern
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h
root@2a950dfabdff:/openj9-openjdk-jdk8# ( cd build/linux-x86_64-normal-server-release/images/j2sdk-image; printf 'public class Test { public static void main(String... args) { System.out.println("Hello World"); } }' > Test.java && ./bin/javac Test.java && ./bin/java -Xshareclasses:name=test -Xdump:system:events=vmstop,request=exclusive+prepwalk Test )
Hello World
JVMDUMP039I Processing dump event "vmstop", detail "#0000000000000000" at 2023/01/17 16:54:55 - please wait.
JVMDUMP032I JVM requested System dump using '/openj9-openjdk-jdk8/build/linux-x86_64-normal-server-release/images/j2sdk-image/core.20230117.165455.74345.0001.dmp' in response to an event
JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.74368.  Review the manual for the external program to find where the core dump is written and ensure the program does not truncate it.

JVMPORT049I The core file created by child process with pid = 74368 was not found. Review the documentation for the /proc/sys/kernel/core_pattern program "|/usr/lib/systemd/systemd-coredump %P %u %g %s %t %c %h" to find where the core file is written and ensure that program does not truncate it.

JVMDUMP010I System dump written to {unable to determine dump name}
JVMDUMP013I Processed dump event "vmstop", detail "#0000000000000000".

@babsingh
Copy link
Contributor

LGTM, all of my suggested changes have been addressed. @keithc-ca Does this PR look good to you?

re #6838 (comment) and #6838 (comment):

  • The reasons to change the message(s) are valid.
  • We can survey the OpenJ9 community (via a Github issue + community call) on the suggested approach since this topic will be subjective and opinions will vary from person to person.
  • Fixing the new lines is not critical and can be ignored (IMO).
  • Also, I agree with the reason for keeping redundancy.

re #6838 (comment):

  • I didn't run the OpenJ9 personal builds. These changes will go through the OpenJ9 acceptance builds. Those builds will verify these changes for us. So, running the OpenJ9 personal builds is optional.
  • The riscv64 failure is known, intermittent and unrelated to these changes. It will probably not be seen in the next set of PR builds.

@babsingh babsingh changed the title Use new informational message when kernel.core_pattern starts with a pipe Print informational message when kernel.core_pattern is piped Jan 18, 2023
@kgibm kgibm requested review from keithc-ca and removed request for babsingh January 23, 2023 17:07
@babsingh
Copy link
Contributor

babsingh commented Jan 23, 2023

@kgibm Thanks for addressing the feedback. Please squash the commits once @keithc-ca approves the PR. Afterwards, I will launch the PR builds and merge the PR.

port/linux/omrosdump_helpers.c Outdated Show resolved Hide resolved
port/linux/omrosdump_helpers.c Outdated Show resolved Hide resolved
@kgibm kgibm requested a review from keithc-ca January 23, 2023 18:39
@keithc-ca
Copy link
Member

I think this change is good, but the description and the (squashed) commit message should be updated, in particular the phrase "no longer treated as an error" which is now inaccurate.

On Linux, when a core dump is requested, if kernel.core_pattern is
a piped pattern, then print a new informational message JVMPORT049I
to clarify the subsequent JVMDUMP012E message that although the JVM
cannot find the core dump, the user should review the manual for
the program specified in core_pattern to potentially find the core.

Signed-off-by: Kevin Grigorenko <kevin.grigorenko@us.ibm.com>
@kgibm
Copy link
Contributor Author

kgibm commented Jan 24, 2023

Squashed and updated commit message and issue description

@babsingh
Copy link
Contributor

jenkins build all

@babsingh
Copy link
Contributor

@babsingh babsingh merged commit d9f0697 into eclipse:master Jan 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Use new informational message when kernel.core_pattern starts with a pipe
3 participants