Delay appending VirtualThread objects to liveVirtualThreadList #16293

babsingh · 2022-11-08T18:49:28Z

Currently, VirtualThread objects are appended to liveVirtualThreadList
in JVM_VirtualThreadMountBegin.

To assist GC with synchronizing while scanning continuations and
VirtualThread objects, VirtualThread objects are appended to
liveVirtualThreadList in JVM_VirtualThreadMountEnd.

This delay will prevent race conditions for the GC while scanning the
VirtualThread objects.

Related: #16290
Related: #16259

Signed-off-by: Babneet Singh sbabneet@ca.ibm.com

Currently, VirtualThread objects are appended to liveVirtualThreadList in JVM_VirtualThreadMountBegin. To assist GC with synchronizing while scanning continuations and VirtualThread objects, VirtualThread objects are appended to liveVirtualThreadList in JVM_VirtualThreadMountEnd. This delay will prevent race conditions for the GC while scanning the VirtualThread objects. Related: eclipse-openj9#16259 Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>

babsingh · 2022-11-08T19:00:18Z

Running a personal build to verify these changes: https://hyc-runtimes-jenkins.swg-devops.com/view/OpenJ9%20-%20Personal/job/Pipeline-Build-Test-Personal/14740/.

fyi @LinHu2016 @amicic @dmitripivkine @tajila @fengxue-IS @pshipton @JasonFengJ9

babsingh · 2022-11-09T20:49:04Z

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_extended.openjdk_aarch64_mac_Personal/7/consoleFull

These are known failures, which are reported in #15967.

## java/security/cert/CertPathBuilder/targetConstraints/BuildEEBasicConstraints.java

14:50:35  sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
14:50:35  	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
14:50:35  	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
14:50:35  	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
14:50:35  	at jdk.test.lib.security.CertUtils.build(CertUtils.java:604)
14:50:35  	at BuildEEBasicConstraints.main(BuildEEBasicConstraints.java:77)
14:50:35  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
14:50:35  	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
14:50:35  	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
14:50:35  	at java.base/java.lang.Thread.run(Thread.java:1573)

## java/security/cert/pkix/policyChanges/TestPolicy.java

14:50:36  java.security.cert.CertPathValidatorException: validity check failed
14:50:36  	at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:135)
14:50:36  	at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:224)
14:50:36  	at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:144)
14:50:36  	at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:83)
14:50:36  	at java.base/java.security.cert.CertPathValidator.validate(CertPathValidator.java:309)
14:50:36  	at TestPolicy.main(TestPolicy.java:102)
14:50:36  	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
14:50:36  	at java.base/java.lang.reflect.Method.invoke(Method.java:578)
14:50:36  	at com.sun.javatest.regtest.agent.MainWrapper$MainThread.run(MainWrapper.java:127)
14:50:36  	at java.base/java.lang.Thread.run(Thread.java:1573)
14:50:36  Caused by: java.security.cert.CertificateExpiredException: NotAfter: Mon Nov 07 06:57:03 EST 2022
14:50:36  	at java.base/sun.security.x509.CertificateValidity.valid(CertificateValidity.java:277)
14:50:36  	at java.base/sun.security.x509.X509CertImpl.checkValidity(X509CertImpl.java:619)
14:50:36  	at java.base/sun.security.provider.certpath.BasicChecker.verifyValidity(BasicChecker.java:190)
14:50:36  	at java.base/sun.security.provider.certpath.BasicChecker.check(BasicChecker.java:144)
14:50:36  	at java.base/sun.security.provider.certpath.PKIXMasterCertPathValidator.validate(PKIXMasterCertPathValidator.java:125)
14:50:36  	... 9 more

babsingh · 2022-11-09T20:50:50Z

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_extended.functional_x86-64_linux_Personal/35/tapResults/

GC failure.

## SyntheticGCWorkload_concurrentSlackAuto_10M_J9_0_FAILED

Legend: 
. = Scavenge
, = Global
f = Failed flip
F = Failed tenure
p = Percolate collect
c = Concurrent aborted
| = Minute marker
x = Warning: aborted collection due to insufficient free space
s = Warning: scan cache overflow (storage acquired from heap)
X = Other warning
T = Total throughput
S = Concurrent slack (-1 = auto)
? = Unknown event
---------------------------------------------------------
/home/jenkins/workspace/Test_openjdk19_j9_extended.functional_x86-64_linux_Personal_testList_1/aqa-tests/TKG/../TKG/output_16679385495615/SyntheticGCWorkload_concurrentSlackAuto_10M_J9_0/config_10M_0.3.xml_2022-11-08T_12-36-07_verbose_original.xml: 
.................,.......F|,.......F,...f.....F,........F,........F,.|......fFxpc,.....f.F,........Fp,..X....F,....f.|...F,.....f.F,........fFxp,........F,.f...|...F,........F,.......F,...f....F,.......F,|.......fFxp,......F,..f......F,........Fpc,..|.....F,......F,.....f.F,........F,...|.....Fp,......f.F,........fFxp,......F,..f.|.....F,........Fp,.....f.F,.......F,.......fFxp,|......F,.......f.F,........F,.......F,...|..
.	,	x	f	F	p	c	X	S	T	
303	40	5	17	39	9	2	1	0	31.9GB
---------------------------------------------------------
/home/jenkins/workspace/Test_openjdk19_j9_extended.functional_x86-64_linux_Personal_testList_1/aqa-tests/TKG/../TKG/output_16679385495615/SyntheticGCWorkload_concurrentSlackAuto_10M_J9_0/config_10M_0.3.xml_2022-11-08T_12-36-07_verbose_concurrentSlackAuto.xml: 
.................,........fFxp,.f.|........Fpc,.......F,.......F,......|..f.F,..........F,.........fFxp,........fFxp,..|....F,........F,........F,...f...|...F,..........F,.........F,....f.....F,..|.........F,.........fFxp,...X.....fFxp,.f.....|.F,.........F,.........fFxp,........fFxp,.f..|.....F,..f.....F,.......Fp,.......F,..f.|....f.F,........F,.........F,........F,.|.......fFxp,....f....F,......f...F,.......f.F
.	,	x	f	F	p	c	X	S	T	
300	34	8	20	34	10	1	1	-2	31.4GB
---------------------------------------------------------
Test results: 
Warning: aborted collection due to insufficient free space (x): before 5, after 8, ratio 1.6 > 0.15 --> failed
Percolate collect (p): before 9, after 10, ratio 1.1111111111111112 > 0.75 --> failed

amicic · 2022-11-09T20:52:42Z

GC failure.
SyntheticGCWorkload_concurrentSlackAuto_10M_J9_0_FAILED

unrelated

babsingh · 2022-11-09T20:53:28Z

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_sanity.openjdk_x86-64_linux_Personal/22/consoleFull

GC assertion triggered.

## jdk/java/lang/Thread/virtual/stress/GetStackTraceALot.java

15:41:13  20:41:07.097 0x7f8ac4009f00    j9mm.141    *   ** ASSERTION FAILED ** at /home/jenkins/workspace/Build_JDK19_x86-64_linux_Personal/omr/gc/base/standard/Scavenger.cpp:1657: ((false))

amicic · 2022-11-09T20:57:32Z

I'm ok with this change (and leaving removal from the list in unmount-being is ok, too).

Scavenger.cpp

Could be due to many sorts of root causes. We saw it here already, though: #16259 (comment)

amicic · 2022-11-09T21:00:47Z

Despite the assert failure (not being fixed), I think this is a reasonable change (and leaving removal from the list in unmount-begin is consistent - the steps in mount and unmount should mirror each other).

babsingh · 2022-11-09T21:03:33Z

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_extended.openjdk_x86-64_linux_Personal/20/testReport/

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_extended.openjdk_x86-64_windows_Personal/10/testReport/

These are known failures, which are reported in #15967.

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk19_j9_sanity.openjdk_x86-64_windows_Personal/10/consoleFull

This is a variant of the JVM_VirtualThreadUnmountBegin crash (#16259) on Windows.

### java/lang/Thread/virtual/ParkWithFixedThreadPool.java

19:12:33  j9port_isCompatible+0x1a08b (0x00007FFB7501CC3B [j9prt29+0x1cc3b])
19:12:33  J9VMDllMain+0x14fdcb (0x00007FFB71430DDB [j9gc_full29+0x150ddb])
19:12:33  omrthread_get_category+0xa42 (0x00007FFB7CF24232 [j9thr29+0x4232])
19:12:33  _o__realloc_base+0x60 (0x00007FFB8443FB80 [ucrtbase+0x1fb80])
19:12:33  BaseThreadInitThunk+0x14 (0x00007FFB861E84D4 [KERNEL32+0x84d4])
19:12:33  RtlUserThreadStart+0x21 (0x00007FFB86FD1791 [ntdll+0x51791])

babsingh · 2022-11-09T21:06:17Z

This PR is ready for review. I don't see any new failures introduced by these changes. @gacholio Can you please review?

@amicic Will this PR + #16290 resolve the race condition? or do we require more changes?

runtime/j9vm/javanextvmi.cpp

amicic · 2022-11-09T21:14:56Z

This PR is ready for review. I don't see any new failures introduced by these changes. @gacholio Can you please review?

@amicic Will this PR + #16290 resolve the race condition? or do we require more changes?

#16290 should fix a race and some problems related to walking VT stack etc, but I don't think it will fix problems with VT list (what appears more to be we GC a live VT object)

gacholio · 2022-11-09T21:16:34Z

jenkins test sanity zlinux jdk19

eclipse-openj9/openj9#16212 was fixed by 1. eclipse-openj9/openj9#16290 2. eclipse-openj9/openj9#16293 eclipse-openj9/openj9#16275 is a duplicate of eclipse-openj9/openj9#16212. eclipse-openj9/openj9#16229 was fixed by eclipse-openj9/openj9#16323. FramePop/framepop02 fails with another issue, which is reported in eclipse-openj9/openj9#16346. Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>

eclipse-openj9/openj9#16212 was fixed by 1. eclipse-openj9/openj9#16290; and 2. eclipse-openj9/openj9#16293. eclipse-openj9/openj9#16275 is a duplicate of eclipse-openj9/openj9#16212. eclipse-openj9/openj9#16229 was fixed by eclipse-openj9/openj9#16323. FramePop/framepop02 fails with another issue, which is reported in eclipse-openj9/openj9#16346. Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>

eclipse-openj9/openj9#16212 was fixed by 1. eclipse-openj9/openj9#16290; and 2. eclipse-openj9/openj9#16293. eclipse-openj9/openj9#16275 is a duplicate of eclipse-openj9/openj9#16212. eclipse-openj9/openj9#16229 was fixed by eclipse-openj9/openj9#16323. FramePop/framepop02 fails with another issue, which is reported in eclipse-openj9/openj9#16346. Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com> Signed-off-by: Babneet Singh <sbabneet@ca.ibm.com>

babsingh marked this pull request as ready for review November 9, 2022 21:04

babsingh requested a review from gacholio November 9, 2022 21:06

gacholio approved these changes Nov 9, 2022

View reviewed changes

gacholio self-requested a review November 9, 2022 21:10

gacholio reviewed Nov 9, 2022

View reviewed changes

runtime/j9vm/javanextvmi.cpp Show resolved Hide resolved

gacholio merged commit 78db7ef into eclipse-openj9:master Nov 10, 2022

babsingh mentioned this pull request Nov 21, 2022

Re-enable tests adoptium/aqa-tests#4162

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Delay appending VirtualThread objects to liveVirtualThreadList #16293

Delay appending VirtualThread objects to liveVirtualThreadList #16293

babsingh commented Nov 8, 2022 •

edited

babsingh commented Nov 8, 2022

babsingh commented Nov 9, 2022 •

edited

babsingh commented Nov 9, 2022

amicic commented Nov 9, 2022

babsingh commented Nov 9, 2022

amicic commented Nov 9, 2022

amicic commented Nov 9, 2022

babsingh commented Nov 9, 2022

babsingh commented Nov 9, 2022 •

edited

amicic commented Nov 9, 2022 •

edited

gacholio commented Nov 9, 2022

Delay appending VirtualThread objects to liveVirtualThreadList #16293

Delay appending VirtualThread objects to liveVirtualThreadList #16293

Conversation

babsingh commented Nov 8, 2022 • edited

babsingh commented Nov 8, 2022

babsingh commented Nov 9, 2022 • edited

babsingh commented Nov 9, 2022

amicic commented Nov 9, 2022

babsingh commented Nov 9, 2022

amicic commented Nov 9, 2022

amicic commented Nov 9, 2022

babsingh commented Nov 9, 2022

babsingh commented Nov 9, 2022 • edited

amicic commented Nov 9, 2022 • edited

gacholio commented Nov 9, 2022

babsingh commented Nov 8, 2022 •

edited

babsingh commented Nov 9, 2022 •

edited

babsingh commented Nov 9, 2022 •

edited

amicic commented Nov 9, 2022 •

edited