Avoid unnecessary copying of system lib files #5418

annaibm · 2024-07-04T16:05:23Z

Ensure system lib files are only copied when IS_SVT_TESTREPO is set to true
Prevent unnecessary copying of system lib files to systemtest_prereqs directory

related:https://github.ibm.com/runtimes/backlog/issues/1457

annaibm · 2024-07-05T13:34:00Z

Grinder link:
TARGET: extended.system
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41870/console
TARGET: MiniMix_5m
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41768/
TARGET: jcstress_BooleanFencedTest
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41882/ -windows

TARGET: SharedClasses.SCM01.SingleCL
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41878/ -JDK21 on m/c sles15x86-rtp-rt8-1.fyre.ibm.com
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41880/ -JDK11 on m/c sles15x86-rtp-rt8-1.fyre.ibm.com

Grinder links with:
VENDOR_TEST_REPOS: git@github.ibm.com:runtimes/SVTTestRepo.git
TARGET: jssejce.JCE_2
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19203/

https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19223/

Non system test
TARGET: jdk_math
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41881/

annaibm · 2024-07-05T14:25:08Z

System lib files are copied only when isSVTTestRepo is set to true, i.e when we use VENDOR_TEST_REPOS: git@github.ibm.com:runtimes/SVTTestRepo.git
Grinder link: https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19203/

[echo] isSVTTestRepo is true
17:16:46      [mkdir] Created dir: /home/jenkins/workspace/Grinder_CR/jvmtest/system/systemtest_prereqs
17:16:46      [mkdir] Created dir: /home/jenkins/workspace/Grinder_CR/aqa-tests/systemtest_prereqs
17:16:46       [copy] Copying 11 files to /home/jenkins/workspace/Grinder_CR/jvmtest/system/systemtest_prereqs
17:16:46       [copy] Copying 11 files to /home/jenkins/workspace/Grinder_CR/aqa-tests/systemtest_prereqs
17:16:46

System lib files are not copied:
Grinder: https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41870/console

common_dist:
17:25:48       [copy] Copying 330 files to /home/jenkins/workspace/Grinder_testList_0/jvmtest/system/STF
17:25:49       [copy] Copying 1311 files to /home/jenkins/workspace/Grinder_testList_0/jvmtest/system/aqa-systemtest
17:25:49       [copy] Copying 294 files to /home/jenkins/workspace/Grinder_testList_0/jvmtest/system/openj9-systemtest
17:25:50       [echo] env.IS_SVT_TESTREPO is ${env.IS_SVT_TESTREPO}
17:25:50       [echo] isSVTTestRepo is false

buildenv/jenkins/JenkinsfileBase

system/common.xml

annaibm · 2024-07-09T13:44:17Z

Grinder link:
VENDOR_TEST_REPOS: git@github.ibm.com:runtimes/SVTTestRepo.git
TARGET: jssejce.JCE_2
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19279/consoleFull

Grinder link:
TARGET: extended.system
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/41959/

system/common.xml

buildenv/jenkins/JenkinsfileBase

LongyuZhang

LGTM

llxia · 2024-07-11T15:24:36Z

system/common.xml

+		<condition property="IS_SVT_TESTREPO" value="true" else="false">
+			<and>
+				<isset property="env.IS_SVT_TESTREPO"/>
+				<equals arg1="${env.IS_SVT_TESTREPO}" arg2="true"/>
+			</and>
 		</condition>


Do we need this condition check? Can we do it at line 364?

<equals arg1="${env.IS_SVT_TESTREPO}" arg2="true"/>

Sure , I will try doing there.

llxia · 2024-07-11T20:36:17Z

Please provide Grinder link based on the latest change.

llxia · 2024-07-12T13:07:24Z

buildenv/jenkins/JenkinsfileBase

@@ -112,7 +112,7 @@ def setupEnv() {

 	env.LIGHT_WEIGHT_CHECKOUT = (params.LIGHT_WEIGHT_CHECKOUT == false) ? params.LIGHT_WEIGHT_CHECKOUT : true
 	env.GENERATE_JOBS = params.GENERATE_JOBS ?: false
-	env.isSVTTestRepo=false
+	env.IS_SVT_TESTREPO = ""


I do not think this line is needed.

ok,I was providing this line to check this variable is set or not set. Will try removing this line and test.

llxia · 2024-07-12T13:08:36Z

buildenv/jenkins/JenkinsfileBase

@@ -511,6 +511,8 @@ def setup() {
 		VENDOR_TEST_BRANCHES = (params.VENDOR_TEST_BRANCHES) ? "--vendor_branches \"${params.VENDOR_TEST_BRANCHES}\"" : ""
 		VENDOR_TEST_DIRS = (params.VENDOR_TEST_DIRS) ? "--vendor_dirs \"${params.VENDOR_TEST_DIRS}\"" : ""
 		VENDOR_TEST_SHAS = (params.VENDOR_TEST_SHAS) ? "--vendor_shas \"${params.VENDOR_TEST_SHAS}\"" : ""
+		env.IS_SVT_TESTREPO = params.VENDOR_TEST_REPOS.contains('SVTTestRepo') ? true : false


This may fail if params.VENDOR_TEST_REPOS is not set.

#5418 (comment) passed because params.VENDOR_TEST_REPOS is set to empty, not undefined. I think we should do

env.IS_SVT_TESTREPO = (params.VENDOR_TEST_REPOS && params.VENDOR_TEST_REPOS.contains('SVTTestRepo')) ? true : false

annaibm · 2024-07-12T14:41:46Z

Grinder links:
TARGET: MiniMix_5m, when VENDOR_TEST_REPOS is not set
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/42038/
TARGET: extended.system, VENDOR_TEST_REPOS is set
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/42034/

Not copying

 
09:27:08  common_dist:
09:27:08       [copy] Copying 330 files to /home/jenkins/workspace/Grinder/jvmtest/system/STF
09:27:08       [copy] Copying 1311 files to /home/jenkins/workspace/Grinder/jvmtest/system/aqa-systemtest
09:27:09       [copy] Copying 294 files to /home/jenkins/workspace/Grinder/jvmtest/system/openj9-systemtest

When VENDOR_TEST_REPOS: git@github.ibm.com:runtimes/SVTTestRepo.git
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19670/consoleFull

build:
11:03:16  
11:03:16  common_dist:
11:03:16       [copy] Copying 336 files to C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\STF
11:03:20       [copy] Copying 1203 files to C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\aqa-systemtest
11:03:37       [copy] Copied 510 empty directories to 6 empty directories under C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\aqa-systemtest
11:03:37       [copy] Copying 298 files to C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\openj9-systemtest
11:03:39      [mkdir] Created dir: C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\systemtest_prereqs
11:03:39      [mkdir] Created dir: C:\Users\jenkins\workspace\Grinder_CR\aqa-tests\systemtest_prereqs
11:03:39       [copy] Copying 171 files to C:\Users\jenkins\workspace\Grinder_CR\jvmtest\system\systemtest_prereqs
11:03:45       [copy] Copying 171 files to C:\Users\jenkins\workspace\Grinder_CR\aqa-tests\systemtest_prereqs
11:03:49

- Ensure system lib files are only copied when isSVTTestRepo is set to true - Prevent unnecessary copying of system lib files to systemtest_prereqs directory related:https://github.ibm.com/runtimes/backlog/issues/1457 Signed-off-by: Anna Babu Palathingal <anna.bp@ibm.com>

annaibm · 2024-07-15T15:31:54Z

Grinder links:
TARGET: MiniMix_5m, when VENDOR_TEST_REPOS is not set
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/42043/

When VENDOR_TEST_REPOS: git@github.ibm.com:runtimes/SVTTestRepo.git
TARGET: jssejce.JCE_2
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19805/
TARGET: cookie
https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder_CR/19804/

annaibm force-pushed the copysystemlib branch from 80babed to 1a8b2db Compare July 4, 2024 21:07

annaibm force-pushed the copysystemlib branch from 1a8b2db to d282f09 Compare July 5, 2024 14:17

LongyuZhang reviewed Jul 8, 2024

View reviewed changes

buildenv/jenkins/JenkinsfileBase Outdated Show resolved Hide resolved

system/common.xml Outdated Show resolved Hide resolved

annaibm force-pushed the copysystemlib branch from caa41e8 to 3a379ca Compare July 8, 2024 20:29

annaibm requested a review from LongyuZhang July 8, 2024 20:34

LongyuZhang reviewed Jul 9, 2024

View reviewed changes

system/common.xml Outdated Show resolved Hide resolved

annaibm force-pushed the copysystemlib branch from bd87d26 to 6a22457 Compare July 9, 2024 20:06

annaibm requested a review from LongyuZhang July 9, 2024 20:17

LongyuZhang reviewed Jul 9, 2024

View reviewed changes

buildenv/jenkins/JenkinsfileBase Outdated Show resolved Hide resolved

annaibm force-pushed the copysystemlib branch from 7e08272 to 4df3130 Compare July 10, 2024 13:51

annaibm requested a review from LongyuZhang July 10, 2024 14:19

LongyuZhang approved these changes Jul 10, 2024

View reviewed changes

annaibm marked this pull request as ready for review July 10, 2024 15:34

llxia reviewed Jul 11, 2024

View reviewed changes

annaibm force-pushed the copysystemlib branch from 4df3130 to b4f8d71 Compare July 11, 2024 20:17

annaibm requested a review from llxia July 11, 2024 20:29

llxia reviewed Jul 12, 2024

View reviewed changes

annaibm force-pushed the copysystemlib branch from b4f8d71 to b19a350 Compare July 12, 2024 14:01

annaibm requested a review from llxia July 12, 2024 15:23

annaibm force-pushed the copysystemlib branch from b19a350 to b5e4303 Compare July 15, 2024 14:25

annaibm force-pushed the copysystemlib branch from b5e4303 to 877dfcc Compare July 15, 2024 14:27

llxia approved these changes Jul 15, 2024

View reviewed changes

llxia merged commit 38db5e7 into adoptium:master Jul 15, 2024
2 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid unnecessary copying of system lib files #5418

Avoid unnecessary copying of system lib files #5418

annaibm commented Jul 4, 2024 •

edited

Loading

annaibm commented Jul 5, 2024 •

edited

Loading

annaibm commented Jul 5, 2024

annaibm commented Jul 9, 2024 •

edited

Loading

LongyuZhang left a comment

llxia Jul 11, 2024

annaibm Jul 11, 2024

llxia commented Jul 11, 2024

llxia Jul 12, 2024

annaibm Jul 12, 2024

llxia Jul 12, 2024

llxia Jul 15, 2024

annaibm commented Jul 12, 2024 •

edited

Loading

annaibm commented Jul 15, 2024 •

edited

Loading

Avoid unnecessary copying of system lib files #5418

Avoid unnecessary copying of system lib files #5418

Conversation

annaibm commented Jul 4, 2024 • edited Loading

annaibm commented Jul 5, 2024 • edited Loading

annaibm commented Jul 5, 2024

annaibm commented Jul 9, 2024 • edited Loading

LongyuZhang left a comment

Choose a reason for hiding this comment

llxia Jul 11, 2024

Choose a reason for hiding this comment

annaibm Jul 11, 2024

Choose a reason for hiding this comment

llxia commented Jul 11, 2024

llxia Jul 12, 2024

Choose a reason for hiding this comment

annaibm Jul 12, 2024

Choose a reason for hiding this comment

llxia Jul 12, 2024

Choose a reason for hiding this comment

llxia Jul 15, 2024

Choose a reason for hiding this comment

annaibm commented Jul 12, 2024 • edited Loading

annaibm commented Jul 15, 2024 • edited Loading

annaibm commented Jul 4, 2024 •

edited

Loading

annaibm commented Jul 5, 2024 •

edited

Loading

annaibm commented Jul 9, 2024 •

edited

Loading

annaibm commented Jul 12, 2024 •

edited

Loading

annaibm commented Jul 15, 2024 •

edited

Loading