ParallelStreamsLoadTest_special_J9_15 ASSERTION FAILED codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1)))))) #12696

pshipton · 2021-05-15T16:34:06Z

https://ci.eclipse.org/openj9/job/Test_openjdk16_j9_special.system_s390x_linux_Personal_testList_2/13
ub18-390-1
ParallelStreamsLoadTest_special_J9_15
variation: Mode555
JVM_OPTIONS: -XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Test/Test_openjdk16_j9_special.system_s390x_linux_Personal_testList_2/13/system_test_output.tar.gz
https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Build_JDK16_s390x_linux_Personal/92/OpenJ9-JDK16-s390x_linux-20210515-063703.tar.gz
https://140-211-168-230-openstack.osuosl.org/artifactory/ci-eclipse-openj9/Build_JDK16_s390x_linux_Personal/92//debug-image.tar.gz

LT  14:29:16.178 - Completed 100.0%. Number of tests started=2 (+0)
LT  stderr 14:29:35.149 0x16d8900j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK16_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
LT  stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2021/05/15 14:29:35 - please wait.
LT  stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Personal_testList_2/openjdk-tests/TKG/output_16210796471323/ParallelStreamsLoadTest_special_J9_15/20210515-142812-ParallelStreamsLoadTest/results/core.20210515.142935.9275.0001.dmp' in response to an event
LT  stderr JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P %E" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.9371.
LT  stderr 
LT  stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Personal_testList_2/openjdk-tests/TKG/output_16210796471323/ParallelStreamsLoadTest_special_J9_15/20210515-142812-ParallelStreamsLoadTest/results/core.20210515.142935.9275.0001.dmp
LT  stderr Unhandled exception
LT  stderr Type=Segmentation error vmState=0x0002000f
LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
LT  stderr Handler1=000003FFB54C7410 Handler2=000003FFB53B2190 InaccessibleAddress=01414EFFFFFFF000
LT  stderr gpr0=00000000016E09F8 gpr1=0000000000000060 gpr2=01414F0000000000 gpr3=000000002CE976D8
LT  stderr gpr4=FEBEB10010BD8270 gpr5=0000000000000060 gpr6=000003FFB56B1949 gpr7=00000000016E0578
LT  stderr gpr8=0000000004600006 gpr9=0000000000000000 gpr10=0000000001344600 gpr11=000003FFB46FE170
LT  stderr gpr12=000003FFB68A6000 gpr13=0000000000000000 gpr14=2CE976E800000000 gpr15=000003FFB46FDFE0
LT  stderr psw=000003FFB550F9D4 mask=0705100180000000 fpc=00c80000 bea=000003FFB550F942
LT  stderr fpr0 4194a00000000000 (f: 0.000000, d: 8.650752e+07)
LT  stderr fpr1 4194a00000000000 (f: 0.000000, d: 8.650752e+07)
LT  stderr fpr2 41d4a00000000000 (f: 0.000000, d: 1.384120e+09)
LT  stderr fpr3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr4 4030000000000000 (f: 0.000000, d: 1.600000e+01)
LT  stderr fpr5 0000000000001000 (f: 4096.000000, d: 2.023693e-320)
LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr7 000002aa157d3730 (f: 360527680.000000, d: 1.447379e-311)
LT  stderr fpr8 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
LT  stderr fpr9 000003ffdfafbbe0 (f: 3752836096.000000, d: 2.172656e-311)
LT  stderr fpr10 000003ffb46bf000 (f: 3026972672.000000, d: 2.172297e-311)
LT  stderr fpr11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr12 0005c25f2bb11bbc (f: 733027264.000000, d: 8.009243e-309)
LT  stderr fpr13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr Module=/home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Personal_testList_2/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
LT  stderr Module_base_address=000003FFB5480000
LT  stderr Target=2_90_20210515_92 (Linux 4.15.0-135-generic)
LT  stderr CPU=s390x (4 logical CPUs) (0x1f710c000 RAM)
LT  stderr ----------- Stack Backtrace -----------
LT  stderr (0x000003FFB550F9D4 [libj9vm29.so+0x8f9d4])
LT  stderr (0x000003FFB43488E4 [libj9gc29.so+0x488e4])
LT  stderr (0x000003FFB433F34A [libj9gc29.so+0x3f34a])
LT  stderr (0x000003FFB433DC6A [libj9gc29.so+0x3dc6a])
LT  stderr (0x000003FFB43FF3C4 [libj9gc29.so+0xff3c4])
LT  stderr (0x000003FFB4400990 [libj9gc29.so+0x100990])
LT  stderr (0x000003FFB444B788 [libj9gc29.so+0x14b788])
LT  stderr (0x000003FFB444ADF8 [libj9gc29.so+0x14adf8])
LT  stderr (0x000003FFB53B32FE [libj9prt29.so+0x332fe])
LT  stderr (0x000003FFB444A7B8 [libj9gc29.so+0x14a7b8])
LT  stderr (0x000003FFB5305A14 [libj9thr29.so+0x5a14])
LT  stderr (0x000003FFB6687AA8 [libpthread.so.0+0x7aa8])
LT  stderr (0x000003FFB64FA896 [libc.so.6+0xfa896])
LT  stderr (0x0000000000000000 [<unknown>+0x0])
LT  stderr ---------------------------------------

The text was updated successfully, but these errors were encountered:

pshipton · 2021-05-15T18:57:59Z

20 x 4 grinder https://ci.eclipse.org/openj9/job/Grinder/1816/ - passed

pshipton · 2021-05-28T13:17:44Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/8 - ub20-390-2
ParallelStreamsLoadTest_special_J9_15

https://openj9-artifactory.osuosl.org/artifactory/ci-eclipse-openj9/Test/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/8/system_test_output.tar.gz

LT  stderr 05:09:00.043 0x156a900j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK16_s390x_linux_Nightly/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
LT  stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2021/05/28 05:09:00 - please wait.
LT  stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/openjdk-tests/TKG/output_16221691956574/ParallelStreamsLoadTest_special_J9_15/20210528-050737-ParallelStreamsLoadTest/results/core.20210528.050900.1127308.0001.dmp' in response to an event
LT  stderr JVMPORT030W /proc/sys/kernel/core_pattern setting "|/usr/share/apport/apport %p %s %c %d %P %E" specifies that the core dump is to be piped to an external program.  Attempting to rename either core or core.1127399.
LT  stderr 
LT  stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/openjdk-tests/TKG/output_16221691956574/ParallelStreamsLoadTest_special_J9_15/20210528-050737-ParallelStreamsLoadTest/results/core.20210528.050900.1127308.0001.dmp
LT  stderr Unhandled exception
LT  stderr Type=Segmentation error vmState=0x0002000f
LT  stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
LT  stderr Handler1=000003FF8E2C7458 Handler2=000003FF8E1B2190 InaccessibleAddress=012A67FFFFFFF000
LT  stderr gpr0=0000000001572A18 gpr1=000000000DA78DA8 gpr2=012A680000000000 gpr3=0000000031446848
LT  stderr gpr4=FED5980031446978 gpr5=000000000DA78DA8 gpr6=000003FF8E4B1F39 gpr7=00000000015725A8
LT  stderr gpr8=0000000004600006 gpr9=0000000000000000 gpr10=00000000011D4000 gpr11=000003FF8D2FE170
LT  stderr gpr12=000003FF8F6A6F88 gpr13=0000000000000000 gpr14=3144685800000000 gpr15=000003FF8D2FDFE0
LT  stderr psw=000003FF8E30FB04 mask=0705100180000000 fpc=00c80000 bea=000003FF8E30FA72
LT  stderr fpr0 4197c00000000000 (f: 0.000000, d: 9.961472e+07)
LT  stderr fpr1 4197c00000000000 (f: 0.000000, d: 9.961472e+07)
LT  stderr fpr2 41d7c00000000000 (f: 0.000000, d: 1.593836e+09)
LT  stderr fpr3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr4 4030000000000000 (f: 0.000000, d: 1.600000e+01)
LT  stderr fpr5 000000642ccfe2a0 (f: 751821440.000000, d: 2.125710e-312)
LT  stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr7 0000000080008000 (f: 2147516416.000000, d: 1.061014e-314)
LT  stderr fpr8 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
LT  stderr fpr9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr10 000003ff8d2bf000 (f: 2368466944.000000, d: 2.171972e-311)
LT  stderr fpr11 000003ffebc7c1f0 (f: 3955737088.000000, d: 2.172756e-311)
LT  stderr fpr12 0005c35cdadddcf5 (f: 3671973120.000000, d: 8.014626e-309)
LT  stderr fpr13 000002aa37705290 (f: 930108032.000000, d: 1.447661e-311)
LT  stderr fpr14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
LT  stderr Module=/home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
LT  stderr Module_base_address=000003FF8E280000
LT  stderr Target=2_90_20210527_9 (Linux 5.4.0-73-generic)
LT  stderr CPU=s390x (4 logical CPUs) (0x1f648a000 RAM)
LT  stderr ----------- Stack Backtrace -----------
LT  stderr (0x000003FF8E30FB04 [libj9vm29.so+0x8fb04])
LT  stderr (0x000003FF8C1C8BB4 [libj9gc29.so+0x48bb4])
LT  stderr (0x000003FF8C1BF61A [libj9gc29.so+0x3f61a])
LT  stderr (0x000003FF8C1BDF3A [libj9gc29.so+0x3df3a])
LT  stderr (0x000003FF8C27FD9C [libj9gc29.so+0xffd9c])
LT  stderr (0x000003FF8C281368 [libj9gc29.so+0x101368])
LT  stderr (0x000003FF8C2CC190 [libj9gc29.so+0x14c190])
LT  stderr (0x000003FF8C2CB800 [libj9gc29.so+0x14b800])
LT  stderr (0x000003FF8E1B32FE [libj9prt29.so+0x332fe])
LT  stderr (0x000003FF8C2CB1C0 [libj9gc29.so+0x14b1c0])
LT  stderr (0x000003FF8E105A14 [libj9thr29.so+0x5a14])
LT  stderr (0x000003FF8F489986 [libpthread.so.0+0x9986])
LT  stderr (0x000003FF8F303CC6 [libc.so.6+0x103cc6])
LT  stderr (0x0000000000000000 [<unknown>+0x0])

dmitripivkine · 2021-05-28T14:07:01Z

I will find out reason for crash in Copy Forward

dmitripivkine · 2021-05-28T15:04:44Z

This crash occur an attempt to walk java thread stack:

LT  stderr (0x000003FF8E30FB04 [libj9vm29.so+0x8fb04])	<walkStackFrames+692>:
LT  stderr (0x000003FF8C1C8BB4 [libj9gc29.so+0x48bb4])	<_ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+132>:
LT  stderr (0x000003FF8C1BF61A [libj9gc29.so+0x3f61a])	<_ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+330>:
LT  stderr (0x000003FF8C1BDF3A [libj9gc29.so+0x3df3a])	<_ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+226>
LT  stderr (0x000003FF8C27FD9C [libj9gc29.so+0xffd9c])	<_ZN20MM_CopyForwardScheme9scanRootsEP19MM_EnvironmentVLHGC+308>
LT  stderr (0x000003FF8C281368 [libj9gc29.so+0x101368])	<_ZN20MM_CopyForwardScheme24workThreadGarbageCollectEP19MM_EnvironmentVLHGC+240
LT  stderr (0x000003FF8C2CC190 [libj9gc29.so+0x14c190])	<_ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+608>

I guess there is a thread stack jdmpview failed to walk too:

	!stack 0x0156a900	!j9vmthread 0x0156a900	!j9thread 0x3ff882f5500	tid 0x1133a2 (1127330) // load-0

<156a900> *** BEGIN STACK WALK, flags = 00400001 walkThread = 0x000000000156A900 ***
<156a900> 	ITERATE_O_SLOTS
<156a900> 	RECORD_BYTECODE_PC_OFFSET
<156a900> Initial values: walkSP = 0x0000000001572610, PC = 0x0000000000000005, literals = 0x0000000000000000, A0 = 0x00000000015725A0, j2iFrame = 0x0000000001572A18, ELS = 0x000003FF87EFE2C8, decomp = 0x0000000000000000
<156a900> JIT resolve frame: bp = 0x0000000001572630, sp = 0x0000000001572610, pc = 0x0000000000000005, cp = 0x0000000000000000, arg0EA = 0x00000000015725A0, flags = 0x0000000000000074
<156a900> 	Object push (savedJITException)
<156a900> 		O-Slot[0x0000000001572610] = 0x0000000036D80000
May 28, 2021 10:53:21 AM com.ibm.j9ddr.vm29.events.DefaultEventListener corruptData
WARNING: CorruptDataException thrown walking stack. walkThread = 0x000000000156A900
com.ibm.j9ddr.corereaders.memory.MemoryFault: Memory Fault reading 0x3144685800000000
	at openj9.dtfj/com.ibm.j9ddr.corereaders.memory.AbstractMemory.getBytesAt(AbstractMemory.java:185)

Values in the registers corresponds with this stack also:

> !gpinfo
Failing Thread: !j9vmthread 0x11d4000
Failing Thread ID: 0x11339d (1127325)
gpInfo:
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=000003FF8E2C7458 Handler2=000003FF8E1B2190 InaccessibleAddress=012A67FFFFFFF000
gpr0=0000000001572A18 gpr1=000000000DA78DA8 gpr2=012A680000000000 gpr3=0000000031446848
gpr4=FED5980031446978 gpr5=000000000DA78DA8 gpr6=000003FF8E4B1F39 gpr7=00000000015725A8
gpr8=0000000004600006 gpr9=0000000000000000 gpr10=00000000011D4000 gpr11=000003FF8D2FE170
gpr12=000003FF8F6A6F88 gpr13=0000000000000000 gpr14=3144685800000000 gpr15=000003FF8D2FDFE0
psw=000003FF8E30FB04 mask=0705100180000000 fpc=00c80000 bea=000003FF8E30FA72
Module=/home/jenkins/workspace/Test_openjdk16_j9_special.system_s390x_linux_Nightly_testList_2/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
Module_base_address=000003FF8E280000

dmitripivkine · 2021-05-28T18:24:40Z

I can not decode java stack using tool. I believe somebody who can do it manually should take a look.
I have results/jdk downloaded
@tajila FYI

pshipton · 2022-01-21T14:45:10Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_3/21 - rh7-390-2
MathLoadTest_bigdecimal_special_5m_16
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_3/21/system_test_output.tar.gz

MLT stderr 08:58:55.597 0x5b0b00j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK17_s390x_linux_Release/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
MLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2022/01/21 03:58:55 - please wait.
MLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_3/aqa-tests/TKG/output_16427476637940/MathLoadTest_bigdecimal_special_5m_16/20220121-035807-MathLoadTest/results/core.20220121.035855.37988.0001.dmp' in response to an event
MLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_3/aqa-tests/TKG/output_16427476637940/MathLoadTest_bigdecimal_special_5m_16/20220121-035807-MathLoadTest/results/core.20220121.035855.37988.0001.dmp
MLT stderr 00000000005B0B00: Object neither in heap nor stack-allocated in thread load-2
MLT stderr 00000000005B0B00:	O-Slot=00000000005DC400
MLT stderr 00000000005B0B00:	O-Slot value=0000000000000073
MLT stderr 00000000005B0B00:	PC=0000000000000005
MLT stderr 00000000005B0B00:	framesWalked=0
MLT stderr 00000000005B0B00:	arg0EA=00000000005DC390
MLT stderr 00000000005B0B00:	walkSP=00000000005DC400
MLT stderr 00000000005B0B00:	literals=0000000000000000
MLT stderr 00000000005B0B00:	jitInfo=0000000000000000
MLT stderr 00000000005B0B00:	stack=00000000005D8118-00000000005DD950
MLT stderr Unhandled exception
MLT stderr Type=Segmentation error vmState=0x0002000f
MLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=565e5670 Signal_Code=00000002
MLT stderr Handler1=000003FF84148FD8 Handler2=000003FF7FF32260 InaccessibleAddress=0000000099669000
MLT stderr gpr0=000003FF786DFB44 gpr1=0000000099669966 gpr2=0000000000000024 gpr3=0000000000000001
MLT stderr gpr4=0000000000900000 gpr5=0000000020000000 gpr6=0000000000000004 gpr7=000000000000001C
MLT stderr gpr8=000003FF84370490 gpr9=0000000000000003 gpr10=0000000000900000 gpr11=000003FF840FE198
MLT stderr gpr12=000003FF85219000 gpr13=0000000000000000 gpr14=000003FF84192D52 gpr15=000003FF840FDF18
MLT stderr psw=000003FF79172C34 mask=0705000180000000 fpc=0048fe00 bea=000003FF7917247A
MLT stderr fpr0 000003ff840fe788 (f: 2215634944.000000, d: 2.171896e-311)
MLT stderr fpr1 40bbf00000000000 (f: 0.000000, d: 7.152000e+03)
MLT stderr fpr2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr4 40bbf00000000000 (f: 0.000000, d: 7.152000e+03)
MLT stderr fpr5 3e924925d10d4986 (f: 3507309056.000000, d: 2.724787e-07)
MLT stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr7 3e3a332500000000 (f: 0.000000, d: 6.100112e-09)
MLT stderr fpr8 000000003ffb56e0 (f: 1073436416.000000, d: 5.303480e-315)
MLT stderr fpr9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr10 000000003ffb5d40 (f: 1073438016.000000, d: 5.303488e-315)
MLT stderr fpr11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr12 0005d613d1696e86 (f: 3513347840.000000, d: 8.116291e-309)
MLT stderr fpr13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr fpr14 000003ffffef81f0 (f: 4293886464.000000, d: 2.172923e-311)
MLT stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
MLT stderr Module=/home/jenkins/workspace/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_3/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so
MLT stderr Module_base_address=000003FF78780000
MLT stderr Target=2_90_20220120_21 (Linux 3.10.0-1160.25.1.el7.s390x)
MLT stderr CPU=s390x (4 logical CPUs) (0x1ec1b5000 RAM)
MLT stderr ----------- Stack Backtrace -----------
MLT stderr (0x000003FF79172C34 [libj9jit29.so+0x9f2c34])
MLT stderr (0x000003FF84192D52 [libj9vm29.so+0x92d52])
MLT stderr (0x000003FF784C98D4 [libj9gc29.so+0x498d4])
MLT stderr (0x000003FF784C020A [libj9gc29.so+0x4020a])
MLT stderr (0x000003FF784BEB2A [libj9gc29.so+0x3eb2a])
MLT stderr (0x000003FF7857EFEC [libj9gc29.so+0xfefec])
MLT stderr (0x000003FF785810A8 [libj9gc29.so+0x1010a8])
MLT stderr (0x000003FF785CA9A0 [libj9gc29.so+0x14a9a0])
MLT stderr (0x000003FF785CA010 [libj9gc29.so+0x14a010])
MLT stderr (0x000003FF7FF333CE [libj9prt29.so+0x333ce])
MLT stderr (0x000003FF785C99D0 [libj9gc29.so+0x1499d0])
MLT stderr (0x000003FF84085A14 [libj9thr29.so+0x5a14])
MLT stderr (0x000003FF85208312 [libpthread.so.0+0x8312])
MLT stderr (0x000003FF8508E232 [libc.so.6+0x10e232])
MLT stderr ---------------------------------------

pshipton · 2022-04-11T13:57:00Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_4/22 - rh7-390-3
ParallelStreamsLoadTest_special_J9_15
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_4/22/system_test_output.tar.gz

LT  00:04:26.440 - Completed 100.0%. Number of tests started=2 (+0)
LT  stderr 04:04:38.855 0x5a4100j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK17_s390x_linux_Release/openj9/runtime/codert_vm/cnathelp.cpp:129: (!((0 != (((iTableOffset)) & ((1))))))

pshipton · 2023-03-17T14:15:10Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_s390x_linux_Personal_testList_4/64
MathLoadTest_all_special_5m_16
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_s390x_linux_Personal_testList_4/64/system_test_output.tar.gz

MLT stderr 01:46:53.769 0x42f200j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK17_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

pshipton · 2023-07-31T19:36:19Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/4/
DaaLoadTest_daa3_special_5m_16
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/4/system_test_output.tar.gz

DLT stderr 22:16:24.109 0x184d300j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/07/29 22:16:24 - please wait.
DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/core.20230729.221624.3681177.0001.dmp' in response to an event
DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/core.20230729.221624.3681177.0001.dmp
DLT stderr JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/javacore.20230729.221624.3681177.0002.txt' in response to an event
STF 22:16:57.487 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/javacore.20230729.221624.3681177.0002.txt
STF 22:16:57.487 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/core.20230729.221624.3681177.0001.dmp
DLT 22:16:57.794 - Completed 11.1%. Number of tests started=2
DLT stderr javacore file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/javacore.20230729.221624.3681177.0002.txt
DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/core.20230729.221624.3681177.0001.dmp
DLT stderr JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16906618201410/DaaLoadTest_daa3_special_5m_16/20230729-221614-DaaLoadTest/results/javacore.20230729.221624.3681177.0002.txt
DLT stderr 000000000184D300: Object neither in heap nor stack-allocated in thread load-0
DLT stderr 000000000184D300:	O-Slot=0000000001A35BA0
DLT stderr 000000000184D300:	O-Slot value=000003FF61189698
DLT stderr 000000000184D300:	PC=0000000000000005
DLT stderr 000000000184D300:	framesWalked=0
DLT stderr 000000000184D300:	arg0EA=0000000001A35B30
DLT stderr 000000000184D300:	walkSP=0000000001A35BA0
DLT stderr 000000000184D300:	literals=0000000000000000
DLT stderr 000000000184D300:	jitInfo=0000000000000000
DLT stderr 000000000184D300:	stack=0000000001A30AA0-0000000001A362E0
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=000003FF867C9B88 Handler2=000003FF866B1998 InaccessibleAddress=014DCBFFFFFFF000
DLT stderr gpr0=000003FF00000000 gpr1=FFFFFFFF00000001 gpr2=000003FF00000000 gpr3=000000000F22E018
DLT stderr gpr4=FEB234000F21D990 gpr5=014DCC0000000000 gpr6=0000000001A35B38 gpr7=0000000004600006
DLT stderr gpr8=000003FF86A784B0 gpr9=000000000F21D990 gpr10=0000000001597A00 gpr11=000003FF86B7E3E8
DLT stderr gpr12=000003FF874A6F88 gpr13=0000000000000000 gpr14=0000000000000000 gpr15=000003FF86B7E230
DLT stderr psw=000003FF86816D9C mask=0705100180000000 fpc=00080000 bea=000003FF86816D3E
DLT stderr fpr0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr1 40fb098000000000 (f: 0.000000, d: 1.107440e+05)
DLT stderr fpr2 000003ff802f0888 (f: 2150566144.000000, d: 2.171864e-311)
DLT stderr fpr3 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr4 40fb098000000000 (f: 0.000000, d: 1.107440e+05)
DLT stderr fpr5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr8 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
DLT stderr fpr9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr10 000003ff86b3f000 (f: 2259939328.000000, d: 2.171918e-311)
DLT stderr fpr11 000003ffd73faf70 (f: 3611275008.000000, d: 2.172586e-311)
DLT stderr fpr12 000601a78c05ede2 (f: 2349198848.000000, d: 8.353015e-309)
DLT stderr fpr13 000002aa3b1a5690 (f: 991581824.000000, d: 1.447691e-311)
DLT stderr fpr14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/openjdkbinary/j2sdk-image/lib/default/libj9vm29.so
DLT stderr Module_base_address=000003FF86780000
DLT stderr Target=2_90_20230729_14 (Linux 5.4.0-137-generic)
DLT stderr CPU=s390x (4 logical CPUs) (0x1f58f7000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr walkStackFrames+0x504 (0x000003FF86816D9C [libj9vm29.so+0x96d9c])
DLT stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x4e (0x000003FF845CAACE [libj9gc29.so+0x4aace])
DLT stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0x142 (0x000003FF845C0F72 [libj9gc29.so+0x40f72])
DLT stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xda (0x000003FF845BF48A [libj9gc29.so+0x3f48a])
DLT stderr _ZN22MM_GlobalMarkingScheme20markLiveObjectsRootsEP19MM_EnvironmentVLHGC+0x186 (0x000003FF84694E06 [libj9gc29.so+0x114e06])
DLT stderr _ZN25MM_ParallelGlobalMarkTask3runEP18MM_EnvironmentBase+0x1d8 (0x000003FF84696C18 [libj9gc29.so+0x116c18])
DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x22e (0x000003FF846D2196 [libj9gc29.so+0x152196])
DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x138 (0x000003FF846D0C98 [libj9gc29.so+0x150c98])
DLT stderr omrsig_protect+0x3d8 (0x000003FF866B2AE8 [libj9prt29.so+0x32ae8])
DLT stderr dispatcher_thread_proc+0x5c (0x000003FF846D0934 [libj9gc29.so+0x150934])
DLT stderr thread_wrapper+0xf6 (0x000003FF86608A46 [libj9thr29.so+0x8a46])
DLT stderr start_thread+0xd6 (0x000003FF87287E66 [libpthread.so.0+0x7e66])
DLT stderr  (0x000003FF870FCBE6 [libc.so.6+0xfcbe6])
DLT stderr  (0x0000000000000000 [<unknown>+0x0])
DLT stderr ---------------------------------------

pshipton · 2023-08-01T18:01:18Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_0/64
ParallelStreamsLoadTest_special_J9_15
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk17_j9_special.system_s390x_linux_Release_testList_0/64/system_test_output.tar.gz

LT  stderr 10:00:55.853 0x20a1300j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK17_s390x_linux_Release/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

pshipton · 2023-11-13T15:00:46Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/18/
DaaLoadTest_daa3_special_5m_28 -Xcompressedrefs -Xgcpolicy:gencon -Xshareclasses -Xjit

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/18/system_test_output.tar.gz

DLT stderr 20:56:17.578 0x6f8f00j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/11/11 20:56:17 - please wait.
DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16997275408574/DaaLoadTest_daa3_special_5m_28/20231111-205536-DaaLoadTest/results/core.20231111.205617.1080332.0001.dmp' in response to an event
DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_16997275408574/DaaLoadTest_daa3_special_5m_28/20231111-205536-DaaLoadTest/results/core.20231111.205617.1080332.0001.dmp
DLT stderr Unhandled exception
DLT stderr Type=Segmentation error vmState=0x00020002
DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
DLT stderr Handler1=000003FFAA549CD0 Handler2=000003FFAA431998 InaccessibleAddress=001EBDFFFFFFF000
DLT stderr gpr0=00000000897DE838 gpr1=FFFFFFFF00000000 gpr2=00000000897DE838 gpr3=000000000C9A08A8
DLT stderr gpr4=FFE142000CAF4A68 gpr5=001EBE0000000000 gpr6=000000000070C398 gpr7=0000000004600006
DLT stderr gpr8=000003FFAA7F84B8 gpr9=000000000CAF4A68 gpr10=0000000000306A00 gpr11=000003FFAA3FE340
DLT stderr gpr12=000003FFAB226F88 gpr13=000000000070C9B0 gpr14=0000000000000000 gpr15=000003FFAA3FE188
DLT stderr psw=000003FFAA59742C mask=0705100180000000 fpc=00080000 bea=000003FFAA5973CE
DLT stderr fpr0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr1 40a8300000000000 (f: 0.000000, d: 3.096000e+03)
DLT stderr fpr2 000003ffa4262c08 (f: 2753965056.000000, d: 2.172162e-311)
DLT stderr fpr3 3f336e21dbeabbfe (f: 3689593856.000000, d: 2.964814e-04)
DLT stderr fpr4 40a8300000000000 (f: 0.000000, d: 3.096000e+03)
DLT stderr fpr5 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr fpr8 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
DLT stderr fpr9 000002aa0dcc0b10 (f: 231475984.000000, d: 1.447315e-311)
DLT stderr fpr10 000003ffaa3bf000 (f: 2856054784.000000, d: 2.172213e-311)
DLT stderr fpr11 000003ffc87fa250 (f: 3363807744.000000, d: 2.172464e-311)
DLT stderr fpr12 000609e6a9087804 (f: 2835904512.000000, d: 8.397812e-309)
DLT stderr fpr13 000002aa0c722938 (f: 208808256.000000, d: 1.447304e-311)
DLT stderr fpr14 000003ffeb877cc4 (f: 3951525120.000000, d: 2.172754e-311)
DLT stderr fpr15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
DLT stderr Module=/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/jdkbinary/j2sdk-image/lib/default/libj9vm29.so
DLT stderr Module_base_address=000003FFAA500000
DLT stderr Target=2_90_20231111_107 (Linux 5.4.0-137-generic)
DLT stderr CPU=s390x (4 logical CPUs) (0x1f58f7000 RAM)
DLT stderr ----------- Stack Backtrace -----------
DLT stderr walkStackFrames+0x504 (0x000003FFAA59742C [libj9vm29.so+0x9742c])
DLT stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x4e (0x000003FFA83CCCA6 [libj9gc29.so+0x4cca6])
DLT stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0x142 (0x000003FFA83C0742 [libj9gc29.so+0x40742])
DLT stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xda (0x000003FFA83BEC5A [libj9gc29.so+0x3ec5a])
DLT stderr _ZN14MM_RootScanner9scanRootsEP18MM_EnvironmentBase+0x5e (0x000003FFA83C21B6 [libj9gc29.so+0x421b6])
DLT stderr _ZN18MM_MarkingDelegate9scanRootsEP18MM_EnvironmentBaseb+0x14c (0x000003FFA854C724 [libj9gc29.so+0x1cc724])
DLT stderr _ZN19MM_ParallelMarkTask3runEP18MM_EnvironmentBase+0x94 (0x000003FFA857941C [libj9gc29.so+0x1f941c])
DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x22e (0x000003FFA84D41B6 [libj9gc29.so+0x1541b6])
DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x138 (0x000003FFA84D2CB8 [libj9gc29.so+0x152cb8])
DLT stderr omrsig_protect+0x3d8 (0x000003FFAA432AE8 [libj9prt29.so+0x32ae8])
DLT stderr dispatcher_thread_proc+0x5c (0x000003FFA84D2954 [libj9gc29.so+0x152954])
DLT stderr thread_wrapper+0xf6 (0x000003FFAA388A46 [libj9thr29.so+0x8a46])
DLT stderr start_thread+0xd6 (0x000003FFAB007E66 [libpthread.so.0+0x7e66])
DLT stderr  (0x000003FFAAE7CBE6 [libc.so.6+0xfcbe6])
DLT stderr  (0x0000000000000000 [<unknown>+0x0])

JasonFengJ9 · 2023-11-13T16:22:35Z

JDK21 s390x_linux(rhel8s390x-svl-rt2-1)

java version "21.0.1-beta" 2023-10-17
IBM Semeru Runtime Certified Edition 21.0.1+12-202311111639 (build 21.0.1-beta+12-202311111639)
Eclipse OpenJ9 VM 21.0.1+12-202311111639 (build master-f1d8ad75f, JRE 21 Linux s390x-64-Bit Compressed References 20231111_34 (JIT enabled, AOT enabled)
OpenJ9   - f1d8ad75f
OMR      - 9bc6e08c9
JCL      - bd7ef8f78 based on jdk-21.0.1+12)

[2023-11-11T20:29:20.121Z] TEST: sun/security/ssl/CipherSuite/NamedGroupsWithCipherSuite.java

[2023-11-11T20:29:20.126Z] STDOUT:
[2023-11-11T20:29:20.126Z] NamedGroup: secP384r1
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.3, cipher suite: TLS_AES_128_GCM_SHA256
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.3, cipher suite: TLS_AES_256_GCM_SHA384
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.3, cipher suite: TLS_CHACHA20_POLY1305_SHA256
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.2, cipher suite: TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256
[2023-11-11T20:29:20.126Z] Protocol: TLSv1.1, cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
[2023-11-11T20:29:20.126Z] STDERR:
[2023-11-11T20:29:20.126Z] 20:29:11.863 0x3ff040dae00j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk21u/jdk21u-linux-s390x-openj9-IBM/workspace/build/src/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
[2023-11-11T20:29:20.126Z] JVMDUMP039I Processing dump event "traceassert", detail "" at 2023/11/11 12:29:11 - please wait.

[2023-11-11T20:54:58.089Z] jdk_security3_1_FAILED

50x grinder - one failure reproduced at sles12s390x-svl-rt3-1

dmitripivkine · 2023-11-13T20:35:49Z

DDR is crashing at attempt to walk java stack, so I can not narrow down to the reason for crash in GC. I guess DDR suffers from the same problem.

pshipton · 2023-11-14T14:45:06Z

In the grinder there was 1 assertion failure
javax/net/ssl/SSLSession/SessionCacheSizeTests.java
0x3ff1030fe00j9codertvm(j9ji.105 * ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk21u/jdk21u-linux-s390x-openj9-IBM/workspace/build/src/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

vij-singh · 2023-11-14T15:27:01Z

@r30shah Just FYI... this one has started showing up again

r30shah · 2023-11-20T14:41:18Z

Started looking into this last week so picked up failure in https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/18/ and ran a grinder internally [1] to get the reproducibility, I couldn't get it to fail in 250 runs. Checking out the other set of failures and see if some new changes went in which caused this to pop up again.

[1]. Test_grinder/job/Grinder/36106/

r30shah · 2023-11-20T15:05:32Z

Launched a 100x grinder for the failure reported in #12696 (comment) : Test_grinder/job/Grinder/36244.

r30shah · 2023-11-21T16:20:17Z

I inspected the system dump from https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/18/system_test_output.tar.gz and the top compiled method is from DAA (Not necessarily a DAA issue) but @VermaSh Can you take a look at this one.

I am unable to reproduce so far (The grinder launched in #12696 (comment) failed with different failure - ClassNotFoundException exception which is different. Peter has moved it out from 0.42 milestone but we should keep an eye for this popping up again and jump to investigate.

JasonFengJ9 · 2024-03-14T22:20:41Z

JDK22 s390x_linux (rhel8s390x-svl-rt10-1)

[2024-03-14T03:04:30.427Z] variation: Mode650
[2024-03-14T03:04:30.427Z] JVM_OPTIONS:  -XX:-UseCompressedOops -Xverbosegclog 

[2024-03-14T03:21:17.379Z] TEST: javax/net/ssl/TLSCommon/ConcurrentClientAccessTest.java

[2024-03-14T03:21:17.379Z] STDERR:
[2024-03-14T03:21:17.379Z] 03:21:05.008 0x3fef81ca700j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/build-scripts/jobs/jdk22/jdk22-linux-s390x-openj9/workspace/build/src/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
[2024-03-14T03:21:17.379Z] JVMDUMP039I Processing dump event "traceassert", detail "" at 2024/03/13 20:21:05 - please wait.
[2024-03-14T03:21:17.379Z] JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/core.20240313.202105.835028.0001.dmp' in response to an event
[2024-03-14T03:21:17.379Z] JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/core.20240313.202105.835028.0001.dmp
[2024-03-14T03:21:17.379Z] 000003FEF81CA700: Object neither in heap nor stack-allocated in thread pool-42-thread-1
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	O-Slot=000003FF0802D808
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	O-Slot value=000003FF5C2E4352
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	PC=0000000000000005
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	framesWalked=0
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	arg0EA=000003FF0802D798
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	walkSP=000003FF0802D808
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	literals=0000000000000000
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	jitInfo=0000000000000000
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	stack=000003FF080284E8-000003FF0802DCE0
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] *** Invalid JIT return address 000003FF95E62B18 in 000003FF9C1FE300
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/javacore.20240313.202105.835028.0002.txt' in response to an event
[2024-03-14T03:21:17.379Z] JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/javacore.20240313.202105.835028.0002.txt
[2024-03-14T03:21:17.379Z] 000003FEF81CA700: Object neither in heap nor stack-allocated in thread pool-42-thread-1
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	O-Slot=000003FF0802D808
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	O-Slot value=000003FF5C2E4352
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	PC=0000000000000005
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	framesWalked=0
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	arg0EA=000003FF0802D798
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	walkSP=000003FF0802D808
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	literals=0000000000000000
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	jitInfo=0000000000000000
[2024-03-14T03:21:17.379Z] 000003FEF81CA700:	stack=000003FF080284E8-000003FF0802DCE0
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] *** Invalid JIT return address 000003FF95E62B18 in 000003FF9C0FCF90
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.379Z] *** Invalid JIT return address 000003FF95E62B18 in 000003FF9C27E3A0
[2024-03-14T03:21:17.379Z] 
[2024-03-14T03:21:17.380Z] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/Snap.20240313.202105.835028.0003.trc' in response to an event
[2024-03-14T03:21:17.380Z] JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work/scratch/0/Snap.20240313.202105.835028.0003.trc
[2024-03-14T03:21:17.380Z] JVMDUMP013I Processed dump event "traceassert", detail "".

[2024-03-14T03:21:17.380Z] TEST RESULT: Failed. Unexpected exit from test [exit code: 255]
[2024-03-14T03:21:17.380Z] --------------------------------------------------
[2024-03-14T04:15:53.370Z] Test results: passed: 934; failed: 1
[2024-03-14T04:15:53.370Z] Report written to /home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/report/html/report.html
[2024-03-14T04:15:53.370Z] Results written to /home/jenkins/workspace/Test_openjdk22_j9_extended.openjdk_s390x_linux_testList_1/aqa-tests/TKG/output_17103827418969/jdk_security3_1/work
[2024-03-14T04:15:53.370Z] Error: Some tests failed or other problems occurred.
[2024-03-14T04:15:53.370Z] -----------------------------------
[2024-03-14T04:15:53.370Z] jdk_security3_1_FAILED

50x grinder - 49 passed.

One failure of
sun/security/ssl/SSLEngineImpl/SSLEngineBadBufferArrayAccess.java on rhel9s390xrt-1 due to

05:29:35  Warning: java.net.SocketTimeoutException: Read timed out
05:29:35  server unwrap: BUFFER_UNDERFLOW/NOT_HANDSHAKING, 0/0 bytes
05:29:35  ----
05:29:35  server wrap: OK/NOT_HANDSHAKING, 29/85 bytes
05:29:35  Server died ...
05:29:35  java.lang.Exception: Client: Data length error - IF THIS FAILS, PLEASE REPORT THIS TO THE SECURITY TEAM.  WE HAVE BEEN UNABLE TO RELIABLY DUPLICATE.
05:29:35  	at SSLEngineBadBufferArrayAccess.runServerApplication(SSLEngineBadBufferArrayAccess.java:369)
05:29:35  	at SSLEngineBadBufferArrayAccess.runTest(SSLEngineBadBufferArrayAccess.java:247)
05:29:35  	at SSLEngineBadBufferArrayAccess.main(SSLEngineBadBufferArrayAccess.java:165)

pshipton · 2024-04-16T19:04:39Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/40 - rh8-390-1
DaaLoadTest_daa3_special_5m_16
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/40/system_test_output.tar.gz
21:34:37.212 0x60d500j9codertvm(j9ji.105 * ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

r30shah · 2024-04-16T19:06:45Z

FYI @VermaSh

pshipton · 2024-05-02T12:52:02Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk22_j9_sanity.system_s390x_linux_Nightly_testList_1/56 - rh8-390-1
TestIBMJlmRemoteClassAuth_1 -Xcompressedrefs -Xjit -Xgcpolicy:gencon
07:39:49.298 0x7d3900j9codertvm(j9ji.105 * ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK22_s390x_linux_Nightly/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk22_j9_sanity.system_s390x_linux_Nightly_testList_1/56/system_test_output.tar.gz

VermaSh · 2024-05-09T15:40:40Z

From core dump of #12696 (comment):
Test seems to fail trying to de-reference a null pointer in R1[3]. LLGF (32->64) loads J9Class pointer from R11 into R1[3]. But for some reason the memory address in R11 was off by 4 bytes[5], so we end up loading null in R1 instead of the J9Class object. The actual J9Class object seems fine. I haven't been able to reproduce the failure locally or in a grinder so will come back to this after my zOS jdk21 work.

[1] Console output

17:34:51  DLT stderr Unhandled exception
17:34:51  DLT stderr Type=Segmentation error vmState=0x00020002
17:34:51  DLT stderr J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
17:34:51  DLT stderr Handler1=000003FFB39CA580 Handler2=000003FFB38B1AB0 InaccessibleAddress=0000000000000000
17:34:51  DLT stderr gpr0=0000000000618980 gpr1=0000000000000000 gpr2=0000000099669966 gpr3=0000000000618980
17:34:51  DLT stderr gpr4=000003FFB397E3D8 gpr5=0000000000618980 gpr6=0000000000618980 gpr7=0000000004600006
17:34:51  DLT stderr gpr8=000003FFB3C7B4D8 gpr9=000003FFB1B7D890 gpr10=000003FF48001D68 gpr11=00000000006189B8
17:34:51  DLT stderr gpr12=000003FFB86AAF70 gpr13=000003FFB42ABDF8 gpr14=000003FFB18CCE98 gpr15=000003FFB397E0A0
17:34:51  DLT stderr psw=000003FFB199B592 mask=0705000180000000 fpc=00080000 bea=000003FFB199B2EA
17:34:51  DLT stderr fpr0 000003ffb401c428 (f: 3020014592.000000, d: 2.172294e-311)
17:34:51  DLT stderr fpr1 000003ffb839f968 (f: 3090807040.000000, d: 2.172329e-311)
17:34:51  DLT stderr fpr2 0000000000000000 (f: 0.000000, d: 0.000000e+00)
17:34:51  DLT stderr fpr3 0000000000000007 (f: 7.000000, d: 3.458460e-323)
17:34:51  DLT stderr fpr4 000003ff480009ec (f: 1207962112.000000, d: 2.171399e-311)
17:34:51  DLT stderr fpr5 000003ff48000d40 (f: 1207962880.000000, d: 2.171399e-311)
17:34:51  DLT stderr fpr6 0000000000000000 (f: 0.000000, d: 0.000000e+00)
17:34:51  DLT stderr fpr7 da5b2c0f8a46dcc2 (f: 2319899904.000000, d: -1.839338e+127)
17:34:51  DLT stderr fpr8 000003ffb3808950 (f: 3011545344.000000, d: 2.172290e-311)
17:34:51  DLT stderr fpr9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
17:34:51  DLT stderr fpr10 000003ffb430a110 (f: 3023085824.000000, d: 2.172295e-311)
17:34:51  DLT stderr fpr11 000003fff36f9c78 (f: 4084177920.000000, d: 2.172820e-311)
17:34:51  DLT stderr fpr12 000003ffb397f910 (f: 3013081344.000000, d: 2.172290e-311)
17:34:51  DLT stderr fpr13 0000000000000001 (f: 1.000000, d: 4.940656e-324)
17:34:51  DLT stderr fpr14 0000000000041000 (f: 266240.000000, d: 1.315400e-318)
17:34:51  DLT stderr fpr15 0000000000cc3358 (f: 13382488.000000, d: 6.611828e-317)
17:34:51  DLT stderr Module=/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/jdkbinary/j2sdk-image/lib/default/libj9gc29.so
17:34:51  DLT stderr Module_base_address=000003FFB1880000
17:34:51  DLT stderr Target=2_90_20240413_209 (Linux 4.18.0-513.11.1.el8_9.s390x)
17:34:51  DLT stderr CPU=s390x (4 logical CPUs) (0x1d5ba6000 RAM)
17:34:51  DLT stderr ----------- Stack Backtrace -----------
17:34:51  DLT stderr _ZN32MM_GlobalMarkingSchemeRootMarker11doStackSlotEPP8J9ObjectPvPKv+0x44a (0x000003FFB199B592 [libj9gc29.so+0x11b592])
17:34:51  DLT stderr vmThreadStackDoOSlotIterator+0x30 (0x000003FFB18CCE98 [libj9gc29.so+0x4ce98])
17:34:51  DLT stderr walkStackFrames+0xde6 (0x000003FFB3A18696 [libj9vm29.so+0x98696])
17:34:51  DLT stderr _ZN28GC_VMThreadStackSlotIterator9scanSlotsEP10J9VMThreadS1_PvPFvP8J9JavaVMPP8J9ObjectS2_P16J9StackWalkStatePKvEbb+0x4e (0x000003FFB18CCFB6 [libj9gc29.so+0x4cfb6])
17:34:51  DLT stderr _ZN14MM_RootScanner13scanOneThreadEP18MM_EnvironmentBaseP10J9VMThreadPv+0x142 (0x000003FFB18C086A [libj9gc29.so+0x4086a])
17:34:51  DLT stderr _ZN14MM_RootScanner11scanThreadsEP18MM_EnvironmentBase+0xda (0x000003FFB18BED42 [libj9gc29.so+0x3ed42])
17:34:51  DLT stderr _ZN22MM_GlobalMarkingScheme20markLiveObjectsRootsEP19MM_EnvironmentVLHGC+0x194 (0x000003FFB1997CAC [libj9gc29.so+0x117cac])
17:34:51  DLT stderr _ZN25MM_ParallelGlobalMarkTask3runEP18MM_EnvironmentBase+0x1d8 (0x000003FFB1999AE8 [libj9gc29.so+0x119ae8])
17:34:51  DLT stderr _ZN21MM_ParallelDispatcher16workerEntryPointEP18MM_EnvironmentBase+0x22e (0x000003FFB19D5566 [libj9gc29.so+0x155566])
17:34:51  DLT stderr _Z23dispatcher_thread_proc2P14OMRPortLibraryPv+0x138 (0x000003FFB19D4068 [libj9gc29.so+0x154068])
17:34:51  DLT stderr omrsig_protect+0x3d8 (0x000003FFB38B2C00 [libj9prt29.so+0x32c00])
17:34:51  DLT stderr dispatcher_thread_proc+0x5c (0x000003FFB19D3D04 [libj9gc29.so+0x153d04])
17:34:51  DLT stderr thread_wrapper+0xf6 (0x000003FFB3808A46 [libj9thr29.so+0x8a46])
17:34:51  DLT stderr start_thread+0xd6 (0x000003FFB84880DE [libpthread.so.0+0x80de])
17:34:51  DLT stderr  (0x000003FFB8228B42 [libc.so.6+0x28b42])
17:34:51  DLT stderr  (0x0000000000000000 [<unknown>+0x0])
17:34:51  DLT stderr ---------------------------------------
17:34:51  DLT stderr JVMDUMP039I Processing dump event "gpf", detail "" at 2024/04/13 17:34:50 - please wait.
17:34:51  DLT stderr JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173450.17496.0003.dmp' in response to an event
17:34:51  STF 17:34:50.917 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173437.17496.0001.dmp
17:34:51  DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173437.17496.0001.dmp
17:35:14  DLT stderr JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173450.17496.0003.dmp
17:35:14  DLT stderr JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/javacore.20240413.173450.17496.0004.txt' in response to an event
17:35:14  DLT stderr JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/javacore.20240413.173450.17496.0004.txt
17:35:14  DLT stderr JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/Snap.20240413.173450.17496.0005.trc' in response to an event
17:35:14  DLT stderr JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/Snap.20240413.173450.17496.0005.trc
17:35:14  DLT stderr JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/jitdump.20240413.173450.17496.0006.dmp' in response to an event
17:35:14  DLT stderr JVMDUMP051I JIT dump occurred in 'GC Worker' thread 0x00000000001AEA00
17:35:14  DLT stderr JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/jitdump.20240413.173450.17496.0006.dmp
17:35:14  DLT stderr JVMDUMP013I Processed dump event "gpf", detail "".
17:35:14  DLT stderr 000000000060D500: Unhandled exception while validating object in stack frame in thread load-1
17:35:14  DLT stderr 000000000060D500:	O-Slot=0000000000618980
17:35:14  DLT stderr 000000000060D500:	O-Slot value=00000000006189B8
17:35:14  DLT stderr 000000000060D500:	PC=0000000000000005
17:35:14  DLT stderr 000000000060D500:	framesWalked=0
17:35:14  DLT stderr 000000000060D500:	arg0EA=00000000006189D8
17:35:14  DLT stderr 000000000060D500:	walkSP=0000000000618980
17:35:14  DLT stderr 000000000060D500:	literals=0000000000000000
17:35:14  DLT stderr 000000000060D500:	jitInfo=0000000000000000
17:35:14  DLT stderr 000000000060D500:	stack=0000000000613B30-0000000000619350
17:35:14  STF 17:35:12.866 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/javacore.20240413.173450.17496.0004.txt
17:35:14  STF 17:35:12.866 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173450.17496.0003.dmp
17:35:14  STF 17:35:12.866 - Found dump at: /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/Snap.20240413.173450.17496.0005.trc
17:35:14  STF 17:35:12.866 - **FAILED** Process DLT ended with exit code (255) and not the expected exit code/s (0)
17:35:14  DLT stderr javacore file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/javacore.20240413.173450.17496.0004.txt
17:35:14  DLT stderr core file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/core.20240413.173450.17496.0003.dmp
17:35:14  DLT stderr Snap file generated - /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/results/Snap.20240413.173450.17496.0005.trc
17:35:14  STF 17:35:12.866 - Monitoring Report Summary:
17:35:14  STF 17:35:12.866 -   o Process DLT has crashed unexpectedly
17:35:14  STF 17:35:12.867 - Killing processes: DLT
17:35:14  STF 17:35:12.867 -   o Process DLT pid 17496 is not running
17:35:14  **FAILED** at step 1 (Run daa load test). Expected return value=0 Actual=1 at /home/jenkins/workspace/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_1/aqa-tests/TKG/../TKG/output_17130316601848/DaaLoadTest_daa3_special_5m_16/20240413-173409-DaaLoadTest/execute.pl line 117.

[2] Register context

(kca) whatis regs
$r0   = 0x0000000000618980 on-java-stack(0x60d500) Ptr on-java-stack(0x60d500) (6392192)
$r1   = 0x0000000000000000 Null
$r2   = 0x0000000099669966 Unknown
$r3   = 0x0000000000618980 on-java-stack(0x60d500) Ptr on-java-stack(0x60d500) (6392192)
$r4   = 0x000003ffb397e3d8 Ptr MemBlk - /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/omr/gc/base/PacketList.cpp:41 +8
$r5   = 0x0000000000618980 on-java-stack(0x60d500) Ptr on-java-stack(0x60d500) (6392192)
$r6   = 0x0000000000618980 on-java-stack(0x60d500) Ptr on-java-stack(0x60d500) (6392192)
$r7   = 0x0000000004600006 (73400326)
$r8   = 0x000003ffb3c7b4d8 Ptr Unknown
$r9   = 0x000003ffb1b7d890 Ptr Unknown
$r10  = 0x000003ff48001d68 MemBlk - /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/gc_vlhgc/ConfigurationIncrementalGenerational.cpp:174 +80
$r11  = 0x00000000006189b8 on-java-stack(0x60d500) Ptr J9Class - {java/util/LinkedHashMap}
$r12  = 0x000003ffb86aaf70 Ptr Unknown
$r13  = 0x000003ffb42abdf8 MemBlk - /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/vm/jnicsup.cpp:2167 +32
$r14  = 0x000003ffb18cce98 Ptr Unknown
$r15  = 0x000003ffb397e0a0 Ptr J9Method - {java/lang/Thread.run}
$addr = 0x000003ffb199b592 Ptr Unknown
$bea  = 0x000003ffb199b2ea Ptr Unknown

[3] Failing instruction

(kca) x/20i 0x3ffb199b57a
0x3ffb199b57a      a7b10007     tmll      %r11, 7
0x3ffb199b57e      a77400a3     jne       0x3ffb199b6c4
0x3ffb199b582      e310b0000016 llgf      %r1, 0(%r11)
0x3ffb199b588      c02f99669966 llilf     %r2, 0x99669966  CONST 0x99669966
0x3ffb199b58e      a517ff00     nill      %r1, 0xff00
0x3ffb199b592      e32010000020 cg        %r2, 0(%r1) < crashing instruction
0x3ffb199b598      a78400a2     je        0x3ffb199b6dc

[4] Last jump inst

(kca) x/20i 0x000003ffb199b2ea-32
0x3ffb199b2ca      40080004     sth       %r0, 4
0x3ffb199b2ce      e31011380004 lg        %r1, 0x138(%r1)
0x3ffb199b2d4      a7840170     je        0x3ffb199b5b4
0x3ffb199b2d8      e3b010000021 clg       %r11, 0(%r1)
0x3ffb199b2de *    a7a40008     jhe       0x3ffb199b2ee
0x3ffb199b2e2 |    41101028     la        %r1, 0x28(%r1)
0x3ffb199b2e6 |    b92100b1     clgr      %r11, %r1
0x3ffb199b2ea |    a7a40148     jhe       0x3ffb199b57a  < last jump inst
0x3ffb199b2ee >    c040000a90db larl      %r4, 0x3ffb1aed4a4 Ptr

[5] Memory location pointed to by R11

(kca) x/20b 0x00000000006189b8
0x00000000006189b8: 00 00 00 00
0x00000000006189bc: 00 24 55 00
0x00000000006189c0: 00 00 00 00
0x00000000006189c4: 10 7d e2 80
0x00000000006189c8: 00 00 03 ff

[6] Potential J9Class to be loaded in R1

> !j9class 0x00245500
J9Class at 0x245500 {
  Fields for J9Class:
	0x0: UDATA eyecatcher = 0x0000000099669966 (2573637990)
	0x8: struct J9ROMClass* romClass = !j9romclass 0x000003FF59658120
	0x10: void** superclasses = !j9x 0x0000000000244E30
	0x18: UDATA classDepthAndFlags = 0x00000000820E0003 (2181955587)
	0x20: U32 classDepthWithFlags = 0x00000000 (0)
	0x24: U32 classFlags = 0x00000000 (0)
	0x28: struct J9ClassLoader* classLoader = !j9classloader 0x000003FFB4112438
	0x30: struct J9Object* classObject = !j9object 0x0000000010B98570 // java/lang/Class
	0x38: volatile UDATA initializeStatus = 0x0000000000000001 (1)
	0x40: struct J9Method* ramMethods = !j9method 0x0000000000245DD8 // java/util/LinkedHashMap.linkNodeAtEnd(Ljava/util/LinkedHashMap$Entry;)V
	0x48: UDATA* ramStatics = !j9x 0x0000000000244DB0
	0x50: struct J9Class* arrayClass = !j9class 0x0000000000000000
	0x58: UDATA totalInstanceSize = 0x0000000000000034 (52)
	0x60: struct J9ITable* lastITable = !j9itable 0x00000000000A82A8
	0x68: UDATA* instanceDescription = !j9x 0x0000000000000C3D
	0x70: UDATA* instanceLeafDescription = !j9x 0x0000000000000001
	0x78: UDATA instanceHotFieldDescription = 0x0000000000000000 (0)
	0x80: UDATA selfReferencingField1 = 0x0000000000000000 (0)
	0x88: UDATA selfReferencingField2 = 0x0000000000000000 (0)
	0x90: struct J9Method* initializerCache = !j9method 0x0000000000000000
	0x98: UDATA romableAotITable = 0x000003FFB28208F0 (4396746410224)
	0xa0: UDATA packageID = 0x000003FF59489D69 (4395249474921)
	0xa8: struct J9Module* module = !j9module 0x000003FFB4113578
	0xb0: struct J9Class* subclassTraversalLink = !j9class 0x00000000000C7800 // java/util/concurrent/ConcurrentHashMap
	0xb8: struct J9Class* subclassTraversalReverseLink = !j9class 0x00000000000EEF00 // java/util/HashMap
	0xc0: void** iTable = !j9x 0x0000000000246278
	0xc8: UDATA castClassCache = 0x00000000000A7800 (686080)
	0xd0: void** jniIDs = !j9x 0x0000000000000000
	0xd8: UDATA lockOffset = 0x0000000000000004 (4)
	0xe0: U32 paddingForGLRCounters = 0x00000000 (0)
	0xe4: U16 reservedCounter = 0x0000 (0)
	0xe6: U16 cancelCounter = 0x0000 (0)
	0xe8: UDATA newInstanceCount = 0x0000000000000001 (1)
	0xf0: IDATA backfillOffset = 0x0000000000000038 (56)
	0xf8: struct J9Class* replacedClass = !j9class 0x0000000000000000
	0x100: UDATA finalizeLinkOffset = 0x0000000000000000 (0)
	0x108: struct J9Class* nextClassInSegment = !j9class 0x0000000000245000 // java/util/jar/Attributes
	0x110: UDATA* ramConstantPool = !j9x 0x00000000002458D0
	0x118: struct J9Object** callSites = !j9x 0x0000000000000000
	0x120: struct J9Object** invokeCache = !j9x 0x0000000000000000
	0x128: struct J9Object** varHandleMethodTypes = !j9x 0x0000000000000000
	0x130: struct J9VMCustomSpinOptions* customSpinOption = !j9vmcustomspinoptions 0x0000000000000000
	0x138: void** staticSplitMethodTable = !j9x 0x0000000000000000
	0x140: void** specialSplitMethodTable = !j9x 0x0000000000000000
	0x148: struct J9JITExceptionTable* jitMetaDataList = !j9jitexceptiontable 0x0000000000000000
	0x150: struct J9Class* gcLink = !j9class 0x0000000000000000
	0x158: struct J9Class* hostClass = !j9class 0x0000000000245500 // java/util/LinkedHashMap
	0x160: struct J9Class* nestHost = !j9class 0x0000000000000000
	0x168: struct J9FlattenedClassCache* flattenedClassCache = !j9flattenedclasscache 0x0000000000000000
	0x170: struct J9ClassHotFieldsInfo* hotFieldsInfo = !j9classhotfieldsinfo 0x0000000000000000
	0x178: struct J9MemberNameListNode* memberNames = !j9membernamelistnode 0x0000000000000000
}

pshipton · 2024-06-10T16:52:45Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_0/48/
DaaLoadTest_daa1_special_5m_1 -Xgcpolicy:optthruput -Xjit:count=0 -Xnocompressedrefs

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Personal_testList_0/48/system_test_output.tar.gz

15:34:33  DLT stderr 19:34:32.751 0x3ff98b12c00j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK21_s390x_linux_Personal/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
15:34:33  DLT stderr JVMDUMP039I Processing dump event "traceassert", detail "" at 2024/06/08 15:34:32 - please wait.

pshipton · 2024-06-12T17:48:44Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk23_j9_sanity.system_s390x_linux_Nightly_testList_2/2 - rh7-390-2
TestIBMJlmRemoteMemoryAuth_1 -Xcompressedrefs -Xjit -Xgcpolicy:gencon

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk23_j9_sanity.system_s390x_linux_Nightly_testList_2/2/system_test_output.tar.gz

03:58:26.169 0x82ae00j9codertvm(j9ji.105    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK23_s390x_linux_Nightly/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))
JVMDUMP039I Processing dump event "traceassert", detail "" at 2024/06/11 23:58:26 - please wait.

pshipton · 2024-06-27T00:35:06Z

https://openj9-jenkins.osuosl.org/job/Test_openjdk21_j9_special.system_s390x_linux_Release_testList_4/22 - rh8-390-2
DaaLoadTest_daa2_special_5m_16
-XX:+UseCompressedOops -Xgcpolicy:balanced -Xjit:counts=- - - - - - 1 1 1 1000 250 250 - - - 10000 100000 10000,gcOnResolve,rtResolve,sampleInterval=2,scorchingSampleThreshold=10000,quickProfile -Xcheck:gc:vmthreads:all:quiet

https://openj9-artifactory.osuosl.org/artifactory/ci-openj9/Test/Test_openjdk21_j9_special.system_s390x_linux_Release_testList_4/22/system_test_output.tar.gz
12:36:05 DLT stderr 16:36:03.563 0x611600j9codertvm(j9ji.105 * ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK21_s390x_linux_Release/openj9/runtime/codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1))))))

tajila · 2024-07-03T13:27:04Z

@VermaSh Any update on this? This failure is being seen in Liberty testing.

VermaSh · 2024-07-04T14:36:06Z

@tajila I was occupied with a zLinux blocker so wasn't able to look into this. But now that my investigation into the blocker is almost done so I'll look into this next.

tajila · 2024-07-04T14:52:43Z

Thanks @VermaSh

VermaSh · 2024-07-10T18:14:10Z

Was able to reproduce the failure in a grinder with 1 out of 42 failing rate. Looking into the generated core file.

pshipton added the test failure label May 15, 2021

pshipton added this to the Backlog milestone Apr 11, 2022

pshipton added the comp:jit label Apr 11, 2022

pshipton modified the milestones: Backlog, Java 21 (0.42) Nov 13, 2023

pshipton removed this from the Java 21 (0.42) milestone Nov 20, 2023

cjjdespres mentioned this issue Mar 22, 2024

Internal JITServer test failures #14538

Open

JasonFengJ9 mentioned this issue Apr 26, 2024

jdk_security3_0_FAILED javax/net/ssl/DTLS/PacketLossRetransmission.java Segmentation error vmState=0x0005ff04 #19386

Open

JasonFengJ9 mentioned this issue Jun 11, 2024

jdk_security3_0_FAILED sun/security/ssl/SSLEngineImpl/SSLEngineBadBufferArrayAccess.java Exception: Client: Data length error #19682

Closed

pshipton mentioned this issue Jul 3, 2024

Assertion failure in interface lookup jit helper #19793

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ParallelStreamsLoadTest_special_J9_15 ASSERTION FAILED codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1)))))) #12696

ParallelStreamsLoadTest_special_J9_15 ASSERTION FAILED codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1)))))) #12696

pshipton commented May 15, 2021 •

edited

Loading

pshipton commented May 15, 2021 •

edited

Loading

pshipton commented May 28, 2021

dmitripivkine commented May 28, 2021

dmitripivkine commented May 28, 2021

dmitripivkine commented May 28, 2021

pshipton commented Jan 21, 2022

pshipton commented Apr 11, 2022

pshipton commented Mar 17, 2023

pshipton commented Jul 31, 2023

pshipton commented Aug 1, 2023

pshipton commented Nov 13, 2023

JasonFengJ9 commented Nov 13, 2023 •

edited

Loading

dmitripivkine commented Nov 13, 2023

pshipton commented Nov 14, 2023

vij-singh commented Nov 14, 2023

r30shah commented Nov 20, 2023

r30shah commented Nov 20, 2023

r30shah commented Nov 21, 2023

JasonFengJ9 commented Mar 14, 2024 •

edited by pshipton

Loading

pshipton commented Apr 16, 2024

r30shah commented Apr 16, 2024

pshipton commented May 2, 2024

VermaSh commented May 9, 2024 •

edited

Loading

pshipton commented Jun 10, 2024 •

edited

Loading

pshipton commented Jun 12, 2024

pshipton commented Jun 27, 2024

tajila commented Jul 3, 2024

VermaSh commented Jul 4, 2024

tajila commented Jul 4, 2024

VermaSh commented Jul 10, 2024

ParallelStreamsLoadTest_special_J9_15 ASSERTION FAILED codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1)))))) #12696

ParallelStreamsLoadTest_special_J9_15 ASSERTION FAILED codert_vm/cnathelp.cpp:125: (!((0 != (((iTableOffset)) & ((1)))))) #12696

Comments

pshipton commented May 15, 2021 • edited Loading

pshipton commented May 15, 2021 • edited Loading

pshipton commented May 28, 2021

dmitripivkine commented May 28, 2021

dmitripivkine commented May 28, 2021

dmitripivkine commented May 28, 2021

pshipton commented Jan 21, 2022

pshipton commented Apr 11, 2022

pshipton commented Mar 17, 2023

pshipton commented Jul 31, 2023

pshipton commented Aug 1, 2023

pshipton commented Nov 13, 2023

JasonFengJ9 commented Nov 13, 2023 • edited Loading

dmitripivkine commented Nov 13, 2023

pshipton commented Nov 14, 2023

vij-singh commented Nov 14, 2023

r30shah commented Nov 20, 2023

r30shah commented Nov 20, 2023

r30shah commented Nov 21, 2023

JasonFengJ9 commented Mar 14, 2024 • edited by pshipton Loading

pshipton commented Apr 16, 2024

r30shah commented Apr 16, 2024

pshipton commented May 2, 2024

VermaSh commented May 9, 2024 • edited Loading

pshipton commented Jun 10, 2024 • edited Loading

pshipton commented Jun 12, 2024

pshipton commented Jun 27, 2024

tajila commented Jul 3, 2024

VermaSh commented Jul 4, 2024

tajila commented Jul 4, 2024

VermaSh commented Jul 10, 2024

pshipton commented May 15, 2021 •

edited

Loading

pshipton commented May 15, 2021 •

edited

Loading

JasonFengJ9 commented Nov 13, 2023 •

edited

Loading

JasonFengJ9 commented Mar 14, 2024 •

edited by pshipton

Loading

VermaSh commented May 9, 2024 •

edited

Loading

pshipton commented Jun 10, 2024 •

edited

Loading