Consolidate AOT Relocation Records #7601

dsouzai · 2019-10-25T23:36:23Z

dsouzai · 2019-10-25T23:41:13Z

jenkins test sanity.functional+aot all jdk8,jdk11

dsouzai · 2019-10-26T04:13:12Z

All the failures are known failures (see #6057) except for

[2019-10-26T02:11:36.167Z] Testing: abort  
[2019-10-26T02:11:36.167Z] Test start time: 2019/10/26 02:11:35 Coordinated Universal Time
[2019-10-26T02:11:36.167Z] Running command: "/home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdkbinary/j2sdk-image/bin/java" -Xshareclasses:name=cmdLineTest_gpTest_0 -Xscmx400M -Xscmaxaot256m -Xcompressedrefs  -Xdump:system:none -cp "/home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/../../jvmtest/functional/cmdline_options_testresources/cmdlinetestresources.jar" VMBench.GPTests.GPTest   abort
[2019-10-26T02:11:36.167Z] Time spent starting: 13 milliseconds
[2019-10-26T02:16:50.312Z] ***[TEST INFO 2019/10/26 02:16:35] ProcessKiller detected a timeout after 300000 milliseconds!***
[2019-10-26T02:16:50.312Z] INFO: getUnixPID() has failed indicating this is not a UNIX System.'Debug on timeout' is currently only supported on Linux.
[2019-10-26T02:16:50.312Z] Output from test:
[2019-10-26T02:16:50.312Z]  [ERR] JVMDUMP039I Processing dump event "abort", detail "" at 2019/10/26 02:11:36 - please wait.
[2019-10-26T02:16:50.312Z]  [ERR] JVMDUMP032I JVM requested Java dump using '/home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/javacore.20191026.021136.31813.0001.txt' in response to an event
[2019-10-26T02:18:02.217Z] Time spent executing: 380122 milliseconds
[2019-10-26T02:18:02.217Z] Test result: FAILED
[2019-10-26T02:18:02.217Z] Output from test:
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP010I Java dump written to /home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/javacore.20191026.021136.31813.0001.txt
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP032I JVM requested Snap dump using '/home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/Snap.20191026.021136.31813.0002.trc' in response to an event
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP010I Snap dump written to /home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/Snap.20191026.021136.31813.0002.trc
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP007I JVM Requesting JIT dump using '/home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/jitdump.20191026.021136.31813.0003.dmp'
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP010I JIT dump written to /home/jenkins/jenkins-agent/workspace/Test_openjdk11_j9_sanity.functional_ppc64le_linux_Personal/openjdk-tests/TestConfig/test_output_15720500014898/cmdLineTest_gpTest_0_ITER_1/jitdump.20191026.021136.31813.0003.dmp
[2019-10-26T02:18:02.217Z]  [ERR] JVMDUMP013I Processed dump event "abort", detail "".
[2019-10-26T02:18:02.217Z]

will take a look at some point

runtime/compiler/codegen/J9AheadOfTimeCompile.cpp

mstoodle · 2019-11-07T16:31:52Z

runtime/compiler/codegen/J9AheadOfTimeCompile.cpp

-uint8_t *
-J9::AheadOfTimeCompile::emitClassChainOffset(uint8_t* cursor, TR_OpaqueClassBlock* classToRemember)
+uintptrj_t
+J9::AheadOfTimeCompile::getClassChainOffset(TR_OpaqueClassBlock *classToRemember)


Please add a better commit message to explain why you're doing this refactoring. I expect that it's temporary, right? Since eventually all the calls to emitClassChainOffset() will be transitioned into calls to getClassChainOffset() right?

Yup, eventually emitClassChainOffset will be removed.

mstoodle

Mostly very straight-forward, but I think we should come to a common approach to dealing with different relocation records that have the same "shape": either we define an appropriate sounding base class representing that shape and use it for all the relocation records that share that shape, or we should replicate the code and have each relocation record be handled independently of other relocation records. I am mildly in favour of having "shape" record types and having one copy of the code to handle relocations with that shape, but I might go the other way if we can't come up with sensible names for those common "shape" types.

dsouzai · 2019-12-17T15:09:57Z

but I think we should come to a common approach to dealing with different relocation records that have the same "shape": either we define an appropriate sounding base class representing that shape and use it for all the relocation records that share that shape, or we should replicate the code and have each relocation record be handled independently of other relocation records.

@mstoodle Would it be ok for this to happen later, once all the records are consolidated? I'd prefer to just get the consolidation work done first, and then do a second pass to clean up the architectural decisions. The point of this consolidation work is to facilitate the more complex cleanup that has to happen.

Remove the duplicated code that writes the TR_ValidateStaticField relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_ValidateClass relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_ProfiledMethodGuardRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_ProfiledClassGuardRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_ProfiledInlinedMethodRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_MethodPointer relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Remove the duplicated code that writes the TR_ClassPointer relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

dsouzai · 2020-03-25T14:53:16Z

Here we go again :)

@mstoodle could you take a look? I didn't address the architectural concern for dealing with relocation records with the same "shape" because I'd rather get this consolidation work done first; it'll be a lot easier to make architectural changes when there's only copy of the code.

dsouzai · 2020-03-25T14:58:52Z

jenkins test sanity.functional+aot all jdk11

dsouzai · 2020-03-25T18:25:19Z

The tests failed because of

Started by upstream project "Pipeline_Build_Test_JDK11_x86-64_linux" build number 1732
originally caused by:
 Started by upstream project "PullRequest-OpenJ9" build number 3094
 originally caused by:
  GitHub pull request #7601 of commit 19345a17b011d07bd833a2e873de19db081a45ce, no merge conflicts.
java.lang.IllegalArgumentException: Null value not allowed as an environment variable: KEEP_REPORTDIR
	at hudson.EnvVars.put(EnvVars.java:379)
	at hudson.model.StringParameterValue.buildEnvironment(StringParameterValue.java:59)
	at hudson.model.ParametersAction.buildEnvironment(ParametersAction.java:145)
	at hudson.model.Run.getEnvironment(Run.java:2354)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.getEnvironment(WorkflowRun.java:471)
	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:111)
	at org.jenkinsci.plugins.workflow.cps.CpsScmFlowDefinition.create(CpsScmFlowDefinition.java:67)
	at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:299)
	at hudson.model.ResourceController.execute(ResourceController.java:97)
	at hudson.model.Executor.run(Executor.java:429)
Finished: FAILURE

@AdamBrousseau knows what the issue is, he should have a fix available in the next few days.

dsouzai · 2020-03-31T15:44:02Z

jenkins test sanity.functional+aot all jdk11

dsouzai · 2020-04-01T13:00:39Z

All test failures are due to known issues (#6057).

Over to you @mstoodle

mstoodle · 2020-04-24T21:26:49Z

jenkins test sanity.functional+aot all jdk11

dsouzai · 2020-04-26T17:40:07Z

@mstoodle all tests passed :)

mstoodle · 2020-04-27T14:54:59Z

Thanks @dsouzai merging now.

dsouzai added comp:jit comp:jit:aot labels Oct 25, 2019

dsouzai mentioned this pull request Oct 25, 2019

Consolidate writing of relocation record header #4803

Open

13 tasks

dsouzai requested a review from mstoodle October 25, 2019 23:39

dsouzai assigned mstoodle Oct 25, 2019

dsouzai changed the title ~~Consolidate AOT Relocation Records~~ WIP: Consolidate AOT Relocation Records Oct 25, 2019

mstoodle reviewed Nov 7, 2019

View reviewed changes

mstoodle requested changes Nov 7, 2019

View reviewed changes

dsouzai added 8 commits March 24, 2020 15:23

Consolidate TR_ValidateStaticField

157a756

Remove the duplicated code that writes the TR_ValidateStaticField relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_ValidateClass

0ca9292

Remove the duplicated code that writes the TR_ValidateClass relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Refactor emitClassChainOffset

980279f

Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_ProfiledMethodGuardRelocation

857be03

Remove the duplicated code that writes the TR_ProfiledMethodGuardRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_ProfiledClassGuardRelocation

f437f94

Remove the duplicated code that writes the TR_ProfiledClassGuardRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_ProfiledInlinedMethodRelocation

34a99f9

Remove the duplicated code that writes the TR_ProfiledInlinedMethodRelocation relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_MethodPointer

4f1f05f

Remove the duplicated code that writes the TR_MethodPointer relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

Consolidate TR_ClassPointer

19345a1

Remove the duplicated code that writes the TR_ClassPointer relocation header information, and consolidate it in one canonical location. Signed-off-by: Irwin D'Souza <dsouzai.gh@gmail.com>

dsouzai force-pushed the aotConsolidateRecords branch from 507cd9f to 19345a1 Compare March 25, 2020 14:50

dsouzai changed the title ~~WIP: Consolidate AOT Relocation Records~~ Consolidate AOT Relocation Records Mar 25, 2020

mstoodle approved these changes Apr 27, 2020

View reviewed changes

mstoodle merged commit a63afad into eclipse-openj9:master Apr 27, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Consolidate AOT Relocation Records #7601

Consolidate AOT Relocation Records #7601

dsouzai commented Oct 25, 2019

dsouzai commented Oct 25, 2019

dsouzai commented Oct 26, 2019

mstoodle Nov 7, 2019

dsouzai Dec 17, 2019

mstoodle left a comment

dsouzai commented Dec 17, 2019

dsouzai commented Mar 25, 2020

dsouzai commented Mar 25, 2020

dsouzai commented Mar 25, 2020

dsouzai commented Mar 31, 2020

dsouzai commented Apr 1, 2020

mstoodle commented Apr 24, 2020

dsouzai commented Apr 26, 2020

mstoodle commented Apr 27, 2020

Consolidate AOT Relocation Records #7601

Consolidate AOT Relocation Records #7601

Conversation

dsouzai commented Oct 25, 2019

dsouzai commented Oct 25, 2019

dsouzai commented Oct 26, 2019

mstoodle Nov 7, 2019

Choose a reason for hiding this comment

dsouzai Dec 17, 2019

Choose a reason for hiding this comment

mstoodle left a comment

Choose a reason for hiding this comment

dsouzai commented Dec 17, 2019

dsouzai commented Mar 25, 2020

dsouzai commented Mar 25, 2020

dsouzai commented Mar 25, 2020

dsouzai commented Mar 31, 2020

dsouzai commented Apr 1, 2020

mstoodle commented Apr 24, 2020

dsouzai commented Apr 26, 2020

mstoodle commented Apr 27, 2020