Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release Triage for Temurin JDK21 #4773

Closed
smlambert opened this issue Sep 19, 2023 · 12 comments
Closed

Release Triage for Temurin JDK21 #4773

smlambert opened this issue Sep 19, 2023 · 12 comments
Assignees

Comments

@smlambert
Copy link
Contributor

smlambert commented Sep 19, 2023

https://ci.adoptium.net/job/build-scripts/job/release-openjdk21-pipeline/1/ - all platforms except x86-64_mac
https://ci.adoptium.net/job/build-scripts/job/release-openjdk21-pipeline/2/ - x86-64_mac
https://ci.adoptium.net/job/build-scripts/job/release-openjdk21-pipeline/6/ - Alpine/aarch64

Platforms Details Status ✔️ or ❌
aarch64_linux link ✔️
aarch64_mac link ✔️
arm_linux link ✔️
ppc64_aix link ✔️
ppc64le_linux link ✔️
s390x_linux link ✔️
x86-64_alpine-linux link ✔️
x86-64_linux link ✔️
x86-64_mac link ✔️
x86-64_windows link ✔️
aarch64_alpine-linux link ✔️
@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

Rerun some of the failing top-level targets again in AQA_Test_Pipeline, because the build pipelines were launched before cherry-picked backports of test exclusions were merged.

Pipeline Targets rerun Platforms
AQA_Test_Pipeline/176 extended.openjdk, extended.perf, sanity.system x86-64_linux, x86-64_windows, aarch64_linux, aarch64_mac
AQA_Test_Pipeline/177 extended.openjdk, extended.perf, sanity.system x86-64_mac
AQA_Test_Pipeline/178 extended.openjdk, extended.perf, sanity.system x86-64_alpine-linux,arm_linux,ppc64_aix,s390x_linux,ppc64le_linux

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

aarch64_linux - ignore the ClassLoadingTest_5m failure and failed jobs are as following:

Test_openjdk21_hs_extended.openjdk_aarch64_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
jdk_net_1 => deep history 0/1 passed | possible issues

timeout rerun https://ci.adoptium.net/job/Grinder/7578/ - passed

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

aarch64_mac - ignore the ClassLoadingTest_5m failure no other failures

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

ppc64_aix -

Test_openjdk21_hs_sanity.openjdk_ppc64_aix ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode)

jdk_lang_1 => deep history 0/37 passed | possible issues
jdk_lang_0 => deep history 0/37 passed | possible issues
jdk_security4_0 => deep history 14/34 passed | possible issues
jdk_security4_1 => deep history 14/34 passed | possible issues
jdk_foreign_0 => deep history 8/36 passed | possible issues
jdk_foreign_1 => deep history 8/36 passed | possible issues

java/foreign/TestLargeSegmentCopy.java.TestLargeSegmentCopy
java/foreign/TestLargeSegmentCopy.java.TestLargeSegmentCopy
sun/security/krb5/auto/Cleaners.java.Cleaners
sun/security/krb5/auto/Cleaners.java.Cleaners
sun/security/krb5/auto/NoAddresses.java.NoAddresses
sun/security/krb5/auto/NoAddresses.java.NoAddresses
java/lang/String/concat/IntegerMinValue.java.IntegerMinValue
java/lang/String/concat/IntegerMinValue.java.IntegerMinValue
rerun on test-osuosl-aix72-ppc64-1 https://ci.adoptium.net/job/Grinder/7583/ -- sun/security/krb5/auto/NoAddresses passed.

Grinder_20230920145743.tap.zip

java/lang/String/concat/IntegerMinValue.java - https://bugs.openjdk.org/browse/JDK-8306007 , should be excluded or update to use the XLC17 compiler?.Action : should or could we update to XLC17? current xlc version : Version: 13.01.0003.0007
Both java/foreign/TestLargeSegmentCopy.java and sun/security/krb5/auto/Cleaners.java has been problemlisted and excluded in the nightly run. Not sure why it's running in the release run.

rerun https://ci.adoptium.net/job/Grinder/7595/
rerun https://ci.adoptium.net/job/Grinder/7597/

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

x86-64_linux - ignore the ClassLoadingTest_5m failure no other failures

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

x86-64_mac - ignore the ClassLoadingTest_5m failure and failed jobs are as following:

Test_openjdk21_hs_sanity.openjdk_x86-64_mac ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_util_0 => deep history 17/38 passed | possible issues
jdk_util_1 => deep history 17/38 passed | possible issues

Know issue #4751 rerun on mac12 https://ci.adoptium.net/job/Grinder/7577/ - passed as expected.

@smlambert
Copy link
Contributor Author

smlambert commented Sep 20, 2023

x86-64_windows - ignore the ClassLoadingTest_5m failure and no other failures.

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Sep 20, 2023

ppc64le_linux ✅

Test_openjdk21_hs_extended.openjdk_ppc64le_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_management_1 => deep history 0/1 passed | possible issues
jvm_compiler_1 => deep history 0/1 passed | possible issues
jdk_management_0 => deep history 0/1 passed | possible issues
jdk_tools_1 => deep history 0/1 passed | possible issues
jdk_tools_0 => deep history 0/1 passed | possible issues
jdk_jfr_0 => deep history 0/1 passed | possible issues
jdk_jfr_1 => deep history 0/1 passed | possible issues

Rerun https://ci.adoptium.net/job/Grinder/7580/console
jvm_compiler passed, the rest failed. Rerunning on many machines: grinders 7615 to 7623
https://ci.adoptium.net/job/Grinder/7616/console passed ✅
ppc64le_grinder_tap.zip

Test_openjdk21_hs_sanity.openjdk_ppc64le_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_util_0 => deep history 29/34 passed | possible issues
jdk_util_1 => deep history 29/34 passed | possible issues

Rerun https://ci.adoptium.net/job/Grinder/7581/console
Grinder_tap.zip

Test_openjdk21_hs_sanity.system_ppc64le_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

ClassLoadingTest_5m_1 => deep history 1/33 passed | possible issues
ClassLoadingTest_5m_0 => deep history 1/33 passed | possible issues

Rerun with updated system test branch https://ci.adoptium.net/job/Grinder/7582/console
Can ignore classloading test failures

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Sep 20, 2023

s390x_linux ✅

Test_openjdk21_hs_sanity.system_s390x_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

TestJlmRemoteClassAuth_0 => deep history 32/35 passed | possible issues
TestJlmRemoteClassNoAuth_0 => deep history 33/35 passed | possible issues
TestJlmRemoteClassAuth_1 => deep history 33/35 passed | possible issues
TestJlmRemoteClassNoAuth_1 => deep history 33/35 passed | possible issues
TestJlmRemoteMemoryAuth_0 => deep history 33/35 passed | possible issues
TestJlmRemoteMemoryAuth_1 => deep history 33/35 passed | possible issues
TestJlmRemoteNotifierProxyAuth_1 => deep history 33/35 passed | possible issues
TestJlmRemoteMemoryNoAuth_0 => deep history 33/35 passed | possible issues
TestJlmRemoteMemoryNoAuth_1 => deep history 33/35 passed | possible issues
TestJlmRemoteNotifierProxyAuth_0 => deep history 33/35 passed | possible issues
TestJlmRemoteThreadAuth_1 => deep history 32/35 passed | possible issues
TestJlmRemoteThreadAuth_0 => deep history 33/35 passed | possible issues
TestJlmRemoteThreadNoAuth_0 => deep history 33/35 passed | possible issues
ClassLoadingTest_5m_1 => deep history 2/35 passed | possible issues
TestJlmRemoteThreadNoAuth_1 => deep history 33/35 passed | possible issues
ClassLoadingTest_5m_0 => deep history 2/35 passed | possible issues
NioLoadTest_5m_0 => deep history 33/35 passed | possible issues
NioLoadTest_5m_1 => deep history 33/35 passed | possible issues

Ignoring the classloading test failures, running https://ci.adoptium.net/job/Grinder/7585/console

Waiting for extended.openjdk to finish running https://ci.adoptium.net/job/Test_openjdk21_hs_extended.openjdk_s390x_linux/17/

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Sep 20, 2023

arm_linux ✅ (2 outstanding test failures are known upstream issues, excluding here #4785)

Test_openjdk21_hs_extended.functional_arm_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

CryptoTests_0 => deep history 0/32 passed | possible issues

Rerunning https://ci.adoptium.net/job/Grinder/7586/console
Still fails due to timeout, open issue #4336
Comment in linked issue suggests its intermittent. Rerunning on all arm_linux machines for 10/10 iterations: grinders 7625 to 7628
Seeing intermittent passes: https://ci.adoptium.net/job/Grinder/7628/tapTestReport/ has 5 passes of CryptoTests_0
Cryptotests_grinder.zip

Test_openjdk21_hs_sanity.openjdk_arm_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_lang_2 => deep history 4/37 passed | possible issues

Rerunning https://ci.adoptium.net/job/Grinder/7593/console

java/lang/ScopedValue/StressStackOverflow.java

java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalStateException: Subtask not completed or did not complete successfully
	at StressStackOverflow.lambda$run$1(StressStackOverflow.java:208)
	at java.base/jdk.internal.vm.ScopedValueContainer.runWithoutScope(ScopedValueContainer.java:112)
	at java.base/jdk.internal.vm.ScopedValueContainer.run(ScopedValueContainer.java:98)
	at java.base/java.lang.ScopedValue$Carrier.run(ScopedValue.java:510)

Deep history shows that it started failing in May
Known failure on x64 linux https://bugs.openjdk.org/browse/JDK-8313260. Passes on jdk22, needs a backport

Test_openjdk21_hs_extended.openjdk_arm_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_jfr_2 => deep history 0/1 passed | possible issues
jdk_io_2 => deep history 0/1 passed | possible issues

Rerunning https://ci.adoptium.net/job/Grinder/7592/console

java/io/File/GetXSpace.java

FAILED: '/ usable space': 97777729536 > 1139773440
Warning: us > s.free()
FAILED: '/ usable vs. free space': 97777729536 > 3305226240
6 tests: 6 failure(s); first: java.lang.RuntimeException: '/ total space': 125693825024 != 1139773440
java.lang.RuntimeException: 2 test(s) failed
	at GetXSpace.main(GetXSpace.java:485)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at com.sun.javatest.regtest.agent.MainWrapper$MainTask.run(MainWrapper.java:138)
	at java.base/java.lang.Thread.run(Thread.java:1583)

Rerunning on multiple machines, grinders 7599 to 7602
Failed in all grinders 10/10 times
Known linux failure https://bugs.openjdk.org/browse/JDK-8251017

Test_openjdk21_hs_sanity.system_arm_linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

ClassLoadingTest_5m_2 => deep history 2/34 passed | possible issues

Classloading failures can be ignored

@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Sep 20, 2023

x86-64_alpine-linux ✅

Test_openjdk21_hs_sanity.openjdk_x86-64_alpine-linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_security4_0 => deep history 29/33 passed | possible issues
jdk_security4_1 => deep history 29/33 passed | possible issues

Rerunning https://ci.adoptium.net/job/Grinder/7589/console
grinder_tap.zip

Test_openjdk21_hs_extended.openjdk_x86-64_alpine-linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

jdk_net_1 => deep history 0/1 passed | possible issues
jdk_build_0 => deep history 0/1 passed | possible issues
jdk_build_1 => deep history 0/1 passed | possible issues

Rerunning https://ci.adoptium.net/job/Grinder/7590/console

jdk_build tests pass, jdk_net tests fail. Needs test image

Rerunning with test image https://ci.adoptium.net/job/Grinder/7598/console
Still one test failure:

java/net/httpclient/HttpClientLocalAddrTest.java
Program `/home/jenkins/workspace/Grinder/openjdkbinary/j2sdk-image/bin/java' timed out (timeout set to 960000ms, elapsed time including timeout handling was 960278ms).   

Rerunning on all available machines 10/10 iterations, grinders 7633 to 7636
https://ci.adoptium.net/job/Grinder/7636/console passed ✅
x64alpine_grinder_tap.zip

Test_openjdk21_hs_sanity.system_x86-64_alpine-linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

ClassLoadingTest_5m_0 => deep history 2/36 passed | possible issues
ClassLoadingTest_5m_1 => deep history 2/36 passed | possible issues

Classloading tests can be ignored

@adoptium adoptium deleted a comment from smlambert Sep 20, 2023
@adoptium adoptium deleted a comment from smlambert Sep 20, 2023
@adoptium adoptium deleted a comment from smlambert Sep 20, 2023
@adoptium adoptium deleted a comment from smlambert Sep 20, 2023
@Haroon-Khel
Copy link
Contributor

Haroon-Khel commented Sep 20, 2023

aarch64_alpine-linux ✅

extended.openjdk, extended.perf, sanity.system rerun on aarch64_alpine-linux https://ci.adoptium.net/job/AQA_Test_Pipeline/179/console
AQAaarch64alpinetap.zip

All other tests passed

Test_openjdk21_hs_extended.openjdk_aarch64_alpine-linux ⚠️ UNSTABLE ⚠️
Rerun failed

java -version
openjdk version "21" 2023-09-19 LTS
OpenJDK Runtime Environment Temurin-21+35 (build 21+35-LTS)
OpenJDK 64-Bit Server VM Temurin-21+35 (build 21+35-LTS, mixed mode, sharing)

hotspot_serviceability_jvmti_1 => deep history 0/1 passed | possible issues
jvm_native_sanity_1 => deep history 0/1 passed | possible issues
jvm_native_sanity_0 => deep history 0/1 passed | possible issues
jvm_compiler_0 => deep history 0/1 passed | possible issues
hotspot_serviceability_jvmti_0 => deep history 0/1 passed | possible issues
jvm_compiler_1 => deep history 0/1 passed | possible issues
runtime_nestmate_1 => deep history 0/1 passed | possible issues
jdk_io_0 => deep history 0/1 passed | possible issues
runtime_nestmate_0 => deep history 0/1 passed | possible issues
jdk_net_1 => deep history 0/1 passed | possible issues
jdk_net_0 => deep history 0/1 passed | possible issues
jdk_io_1 => deep history 0/1 passed | possible issues
jdk_nio_0 => deep history 0/1 passed | possible issues
jdk_management_1 => deep history 0/1 passed | possible issues
jdk_management_0 => deep history 0/1 passed | possible issues
jdk_nio_1 => deep history 0/1 passed | possible issues
jdk_instrument_0 => deep history 0/1 passed | possible issues
jdk_jdi_0 => deep history 0/1 passed | possible issues
jdk_instrument_1 => deep history 0/1 passed | possible issues
jdk_jdi_1 => deep history 0/1 passed | possible issues
jdk_native_sanity_0 => deep history 0/1 passed | possible issues
jdk_native_sanity_1 => deep history 0/1 passed | possible issues

Rerunning with test image, grinders 7629 to 7632
In https://ci.adoptium.net/job/Grinder/7630/console, all but java/net/httpclient/HttpClientLocalAddrTest.java.HttpClientLocalAddrTest pass
Rerunning this test 10/10 times on all available machines, grinders 7642 to 7645
Passes in this grinder https://ci.adoptium.net/job/Grinder/7643/ on test-docker-alpine313-aarch64-1
aarch64alpineHTTPrerun.zip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: Done
Development

No branches or pull requests

3 participants