Mixed builds, gccheck test failing, DDR problem? #11801

pshipton · 2021-01-27T18:43:36Z

https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_s390x_linux_mixed_Nightly_testList_0/1/
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_s390x_linux_mixed_Nightly_testList_1/1/
cmdLineTester_GCCheck (all modes)

 [OUT] Checking HEAP...  <gc check (1): from debugger: HEAP: Object 3ff0d590008: corrupt data exception>
 [OUT]   <gc check (1): Corrupt 3ff0d590008 header: 00000000 00000000>
 [OUT]   <gc check (2): from debugger: HEAP: Object 3ff88400008: corrupt data exception>
 [OUT]   <gc check (2): Corrupt 3ff88400008 header: 00000000 00000000>
 [OUT]   <gc check (3): from debugger: HEAP: Object 3ff88500000: corrupt data exception>
 [OUT]   <gc check (3): Corrupt 3ff88500000 header: 00000000 00000001>
 [OUT] done (39 ms).
 [OUT] Checking CLASS HEAP...  <gc check (4): from debugger: CLASS HEAP: slots slot 3ff903b2d00(3ff903b2d30) -> 3ff0d5ec5b0: class pointer not in a class segment>
 [OUT]   <gc check (5): from debugger: CLASS HEAP: static slot 3ff903b2900(3ff903b2858) -> 3ff884a7500: class pointer not in a class segment>
 [OUT]   <gc check (6): from debugger: CLASS HEAP: slots slot 3ff903b2300(3ff903b2330) -> 3ff0d5ec2c0: class pointer not in a class segment>
 [OUT]   <gc check (7): from debugger: CLASS HEAP: static slot 3ff903b2000(3ff903b21b8) -> 3ff884a74d8: class pointer not in a class segment>
 [OUT]   <gc check (8): from debugger: CLASS HEAP: static slot 3ff903b1e00(3ff903a5178) -> 3ff884a74c0: class pointer not in a class segment>
...

This is a newly ported test, in case you are wondering why we didn't see this fail previously.

@sharon-wang @dmitripivkine @keithc-ca fyi

The text was updated successfully, but these errors were encountered:

pshipton · 2021-01-27T20:25:54Z

~~May be~~ all platforms, ~~the build is still running~~. I'll update here as they fail.

https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_ppc64le_linux_mixed_Nightly/1/
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_x86-64_windows_mixed_Nightly/1
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_ppc64_aix_mixed_Nightly/1
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_x86-64_linux_mixed_Nightly/1
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_sanity.functional_x86-64_mac_mixed_Nightly/1

gacholio · 2021-01-27T20:51:40Z

@sharon-wang Is the GC check library split?

keithc-ca · 2021-01-27T20:56:32Z

Yes, see https://github.com/ibmruntimes/openj9-openjdk-jdk11/blob/openj9/closed/custom/copy/Copy-java.base.gmk#L65-L68.

keithc-ca · 2021-01-27T21:59:16Z

Trying to run the tests locally with BUILD_LIST=functional/cmdLineTests/gcCheck fails because it does not create cmdlinetester.jar; perhaps some dependency is missing? I tried again with BUILD_LIST=functional/cmdLineTests, but still no luck.

keithc-ca · 2021-01-27T22:03:50Z

I don't know if it's significant, but it's at least surprising to see the reference to 'Windows' in the ppc64le test output:

[2021-01-27T18:45:57.371Z] *** Starting test suite: Shared Classes URLClassLoaderTests Suite - Windows Configuration ***

smlambert · 2021-01-27T22:10:47Z

I believe cmdLineTests dependencies are spread across several directories (grr!), I always end up just setting BUILD_LIST=functional because waiting the extra minutes for everything to compile is somehow less annoying than trying to get the build list right.

Is it:
BUILD_LIST=functional/cmdline_options_testresources,functional/cmdLineTests/gcCheck,functional/cmdline_options_tester

I'd have to check...

keithc-ca · 2021-01-27T22:24:10Z

I now have a core file (on Linux x64) that exhibits !gccheck problems; will investigate.

pshipton · 2021-01-27T22:24:33Z

What Shelley mentioned works in general. I typically add functional/cmdline_options_tester in addition to the test directory.

sharon-wang · 2021-01-27T22:35:56Z

I don't know if it's significant, but it's at least surprising to see the reference to 'Windows' in the ppc64le test output:
[2021-01-27T18:45:57.371Z] *** Starting test suite: Shared Classes URLClassLoaderTests Suite - Windows Configuration ***

I see that some tests in test/functional/cmdLineTests/shareClassTests and test/functional/cmdLineTests/URLClassLoaderTests have "Windows Configuration" in the test suite id, so I think "Windows Configuration" will show up on all platforms that run those tests.

It looks like there are two versions of some tests in playlist files (e.g. one windows and one unix), but both versions use the same xml file which contains "Windows Configuration" in the test suite id.

I can put in a change to remove "Windows Configuration" from the test suite ids to remove that confusion.

JasonFengJ9 · 2021-01-28T12:47:00Z

cmdLineTester_GCCheck_1_FAILED at an internal build Test_openjdk11_j9_sanity.functional_ppc64_aix_Nightly/593:

23:37:40  openjdk version "11.0.10-internal" 2021-01-19
23:37:40  OpenJDK Runtime Environment (build 11.0.10-internal+0-adhoc.jenkins.BuildJDK11ppc64aixNightly)
23:37:40  Eclipse OpenJ9 VM (build master-00e156b6c95, JRE 11 AIX ppc64-64-Bit Compressed References 20210127_713 (JIT enabled, AOT enabled)
23:37:40  OpenJ9   - 00e156b6c95
23:37:40  OMR      - 94012c0f8a7
23:37:40  JCL      - f7475ea97ee based on jdk-11.0.10+9)

===============================================
Running test cmdLineTester_GCCheck_1 ...
===============================================
cmdLineTester_GCCheck_1 Start Time: Thu Jan 28 00:21:52 2021 Epoch Time (ms): 1611811312093
variation: Mode610
JVM_OPTIONS:  -Xcompressedrefs -Xjit -Xgcpolicy:gencon 
Parsing exclude list...
  add excludes for [all]
  add excludes for [aix_ppc-64_cr]
DONE in 0 ms.

Testing: Run gccheck
Test start time: 2021/01/28 00:22:23 Eastern Standard Time
Running command: "/home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64_aix_Nightly_testList_1/openjdkbinary/j2sdk-image/bin/java"  -Xcompressedrefs -Xjit -Xgcpolicy:gencon  --add-exports=java.base/jdk.internal.org.objectweb.asm=ALL-UNNAMED -cp /home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64_aix_Nightly_testList_1/openjdkbinary/j2sdk-image/lib/ddr/j9ddr.jar:/home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64_aix_Nightly_testList_1/openjdk-tests/TKG/../../jvmtest/functional/cmdline_options_testresources/cmdlinetestresources.jar:/home/jenkins/workspace/Test_openjdk11_j9_sanity.functional_ppc64_aix_Nightly_testList_1/openjdk-tests/TKG/../../jvmtest/functional/IBM_Internal/cmdLineTests_IBM_Internal/gcCheck/caffeine.jar com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive j9core.dmp
Time spent starting: 24 milliseconds
Time spent executing: 9454 milliseconds
Test result: FAILED
Output from test:
 [ERR] Exception in thread "main" java.lang.IllegalAccessError: com/ibm/j9ddr/IVMData is not visible
 [ERR] 	at java.base/java.lang.ClassLoader.defineClassInternal(ClassLoader.java:479)
 [ERR] 	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:440)
 [ERR] 	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:392)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.J9DDRClassLoader.findClass(J9DDRClassLoader.java:137)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.J9DDRClassLoader.loadClass(J9DDRClassLoader.java:206)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.J9DDRClassLoader.loadClassRelativeToStream(J9DDRClassLoader.java:195)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.J9DDRClassLoader.getIVMData(J9DDRClassLoader.java:259)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.VMDataFactory.getVMData(VMDataFactory.java:213)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.VMDataFactory.getAllVMData(VMDataFactory.java:143)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.VMDataFactory.getVMData(VMDataFactory.java:106)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.locateRuntimes(DDRInteractive.java:387)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.locateRuntimes(DDRInteractive.java:371)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.<init>(DDRInteractive.java:273)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.<init>(DDRInteractive.java:258)
 [ERR] 	at openj9.dtfj/com.ibm.j9ddr.tools.ddrinteractive.DDRInteractive.main(DDRInteractive.java:282)
>> Success condition was not found: [Output match: Checking HEAP...done]
>> Failure condition was not found: [Output match: gc check]
>> Failure condition was not found: [Output match: unable to read]
>> Failure condition was found: [Output match: Exception]

cmdLineTester_GCCheck_1_FAILED

keithc-ca · 2021-01-28T14:59:55Z

DTFJ is not properly recognizing that the core is from a mixed reference VM.

pshipton · 2021-01-28T16:40:58Z

Many other DDR related failures in extended testing, likely the same issue.
https://ci.eclipse.org/openj9/job/Test_openjdk11_j9_extended.functional_s390x_linux_mixed_Nightly/1/tapTestReport/

keithc-ca · 2021-01-28T17:20:18Z

I think we should have a separate issue for the failure mode mentioned above.

pshipton added the test failure label Jan 27, 2021

keithc-ca self-assigned this Jan 27, 2021

sharon-wang mentioned this issue Jan 27, 2021

Remove Windows Configuration text from test suite id #11804

Merged

pshipton added this to the Release 0.26 (Java 8, 11, 16) Apr refresh milestone Jan 28, 2021

keithc-ca mentioned this issue Jan 28, 2021

Fix algorithm version constant MIXED_REFERENCE_MODE for DDR #11810

Merged

gacholio closed this as completed in #11810 Jan 28, 2021

keithc-ca mentioned this issue Jan 28, 2021

cmdLineTester_GCCheck test failure #11811

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mixed builds, gccheck test failing, DDR problem? #11801

Mixed builds, gccheck test failing, DDR problem? #11801

pshipton commented Jan 27, 2021

pshipton commented Jan 27, 2021 •

edited

Loading

gacholio commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

smlambert commented Jan 27, 2021 •

edited by pshipton

Loading

keithc-ca commented Jan 27, 2021

pshipton commented Jan 27, 2021

sharon-wang commented Jan 27, 2021 •

edited

Loading

JasonFengJ9 commented Jan 28, 2021

keithc-ca commented Jan 28, 2021

pshipton commented Jan 28, 2021

keithc-ca commented Jan 28, 2021

Mixed builds, gccheck test failing, DDR problem? #11801

Mixed builds, gccheck test failing, DDR problem? #11801

Comments

pshipton commented Jan 27, 2021

pshipton commented Jan 27, 2021 • edited Loading

gacholio commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

keithc-ca commented Jan 27, 2021

smlambert commented Jan 27, 2021 • edited by pshipton Loading

keithc-ca commented Jan 27, 2021

pshipton commented Jan 27, 2021

sharon-wang commented Jan 27, 2021 • edited Loading

JasonFengJ9 commented Jan 28, 2021

keithc-ca commented Jan 28, 2021

pshipton commented Jan 28, 2021

keithc-ca commented Jan 28, 2021

pshipton commented Jan 27, 2021 •

edited

Loading

smlambert commented Jan 27, 2021 •

edited by pshipton

Loading

sharon-wang commented Jan 27, 2021 •

edited

Loading