Created a general function for retrying commands #7208

JennyChenyj · 2019-09-24T15:57:45Z

[skip ci]

Signed-off-by: Jenny Chen Jenny.Chen@ibm.com

JennyChenyj · 2019-09-24T15:58:24Z

Hi @AdamBrousseau. Thanks!

JennyChenyj · 2019-09-24T18:01:29Z

Hi @AdamBrousseau! The order has been changed. Thanks!

AdamBrousseau · 2019-09-24T18:11:30Z

buildenv/jenkins/common/pipeline-functions.groovy

+    } else {
+        ret = true
+    }
+    command()


Indenting needs to be fixed

AdamBrousseau · 2019-09-24T18:12:05Z

buildenv/jenkins/common/pipeline-functions.groovy

@@ -606,6 +606,18 @@ def generate_test_jobs(TARGET_NAMES, SPEC, ARTIFACTORY_SERVER, ARTIFACTORY_REPO)
    }
 }

+def retry_and_delay(command, numRetries = 3, waitTime = 60, units = 'SECONDS') {


We should add a comment about how to use this function.

AdamBrousseau · 2019-09-24T18:12:28Z

buildenv/jenkins/common/pipeline-functions.groovy

+    def ret = false
+    retry(numRetries) {
+    if (ret) {
+        sleep time: waitTime,unit: units


Whitepsace after comma.

Thanks for find those mistakes! I have fixed them. Thanks for checking it again~

AdamBrousseau

Looks good. Once #7110 and #7145 are merged we can replace those retries with a call to this function.

pshipton · 2019-09-25T13:43:26Z

They are merged now.

JennyChenyj · 2019-09-25T15:14:42Z

They are merged now.

Thanks!

JennyChenyj · 2019-09-25T16:56:09Z

Hey @AdamBrousseau! I replaced both of them. Thanks for checking! :)

AdamBrousseau · 2019-09-25T17:19:03Z

Jenkins compile plinux jdk11

JennyChenyj · 2019-09-25T18:46:42Z

Hey @AdamBrousseau! Sorry, I made a mistake of calling the function. I fixed it. Thanks!

AdamBrousseau · 2019-09-25T19:05:22Z

Jenkins compile plinux jdk11

AdamBrousseau · 2019-09-25T19:10:01Z

buildenv/jenkins/common/variables-functions.groovy

-       }
-       jobDsl targets: templatePath, ignoreExisting: false, additionalParameters: params
-    }
+    retry_and_delay({jobDsl targets: templatePath, ignoreExisting: false, additionalParameters: params}, 3, 120)


You'll have to check if pipeline-functions.groovy is loaded at this point in the build and what variable it's loaded into. Same comment for build.groovy. Look for a load step
ex buildFile = load 'buildenv/jenkins/common/pipeline-functions.groovy'

Hey Adam! I think pipeline-functions.groovy is not loaded when it fails. I checked the console output and there is no such line
load 'buildenv/jenkins/common/pipeline-functions.groovy'. Thanks!

Right, so we'll have to make sure we load that library in those files so we can utilize this new function.

JennyChenyj · 2019-09-27T15:20:49Z

Hey @AdamBrousseau! Thanks for checking it!

AdamBrousseau · 2019-09-30T14:09:21Z

Jenkins compile plinux jdk11

AdamBrousseau · 2019-09-30T15:12:19Z

Jenkins compile plinux jdk11

AdamBrousseau · 2019-09-30T16:03:11Z

Jenkins compile plinux jdk11

AdamBrousseau

LGTM. PR build tests the 2 function calls.

pshipton · 2019-09-30T18:27:14Z

The commits can be squashed, I don't think we need to have all three merged. Also, a different formatting for pipelineFunctions.retry_and_delay({server.upload spec: uploadSpec, buildInfo: buildInfo; server.publishBuildInfo buildInfo}, 3, 300) should be easier to read. Something like

pipelineFunctions.retry_and_delay({
    server.upload spec: uploadSpec, buildInfo: buildInfo;
    server.publishBuildInfo buildInfo},
    3, 300)

JennyChenyj · 2019-09-30T18:29:32Z

The commits can be squashed, I don't think we need to have all three merged. Also, a different formatting for pipelineFunctions.retry_and_delay({server.upload spec: uploadSpec, buildInfo: buildInfo; server.publishBuildInfo buildInfo}, 3, 300) should be easier to read. Something like
pipelineFunctions.retry_and_delay({
    server.upload spec: uploadSpec, buildInfo: buildInfo;
    server.publishBuildInfo buildInfo},
    3, 300)

Thanks for the advice! I will make the change now.

Rewrote the retries for uploading to artifactory and creating jobs [skip ci] Signed-off-by: Jenny Chen <Jenny.Chen@ibm.com>

JennyChenyj · 2019-10-04T13:27:36Z

Hi @pshipton! I have made the change. Thanks!

JennyChenyj changed the title ~~Created a general function for retrying commands~~ WIP Created a general function for retrying commands Sep 24, 2019

JennyChenyj mentioned this pull request Sep 24, 2019

Wrapped job creation to reduce ConcurrentModificationException #7145

Merged

JennyChenyj force-pushed the AddRetryWait branch from 27b74e1 to 8904a3f Compare September 24, 2019 17:51

AdamBrousseau reviewed Sep 24, 2019

View reviewed changes

buildenv/jenkins/common/pipeline-functions.groovy Outdated

} else {

ret = true

}

command()

Copy link

Contributor

AdamBrousseau Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indenting needs to be fixed

AdamBrousseau reviewed Sep 24, 2019

View reviewed changes

JennyChenyj force-pushed the AddRetryWait branch 3 times, most recently from 8578e6d to bf8f94f Compare September 24, 2019 18:40

AdamBrousseau reviewed Sep 25, 2019

View reviewed changes

pshipton added the comp:build label Sep 25, 2019

JennyChenyj force-pushed the AddRetryWait branch 2 times, most recently from 07fcdcb to e64db12 Compare September 25, 2019 16:55

JennyChenyj force-pushed the AddRetryWait branch from e64db12 to 1cbf0c3 Compare September 25, 2019 18:45

AdamBrousseau reviewed Sep 25, 2019

View reviewed changes

JennyChenyj force-pushed the AddRetryWait branch 2 times, most recently from 555548e to 8e47549 Compare September 27, 2019 15:20

JennyChenyj force-pushed the AddRetryWait branch from 8e47549 to df0abcb Compare September 30, 2019 14:59

JennyChenyj force-pushed the AddRetryWait branch from df0abcb to 1670424 Compare September 30, 2019 16:01

AdamBrousseau approved these changes Sep 30, 2019

View reviewed changes

pshipton changed the title ~~WIP Created a general function for retrying commands~~ Created a general function for retrying commands Sep 30, 2019

pshipton approved these changes Sep 30, 2019

View reviewed changes

pshipton self-assigned this Sep 30, 2019

JennyChenyj force-pushed the AddRetryWait branch 2 times, most recently from 44068b6 to acc6fb5 Compare September 30, 2019 18:52

Created a general function of retrying commands with delay options

490a5ef

Rewrote the retries for uploading to artifactory and creating jobs [skip ci] Signed-off-by: Jenny Chen <Jenny.Chen@ibm.com>

JennyChenyj force-pushed the AddRetryWait branch from acc6fb5 to 490a5ef Compare September 30, 2019 18:55

pshipton merged commit ddbc3da into eclipse-openj9:master Oct 4, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Created a general function for retrying commands #7208

Created a general function for retrying commands #7208

JennyChenyj commented Sep 24, 2019

JennyChenyj commented Sep 24, 2019

JennyChenyj commented Sep 24, 2019

AdamBrousseau Sep 24, 2019

AdamBrousseau Sep 24, 2019

AdamBrousseau Sep 24, 2019

JennyChenyj Sep 24, 2019

AdamBrousseau left a comment

pshipton commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019

AdamBrousseau commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019 •

edited

AdamBrousseau commented Sep 25, 2019

AdamBrousseau Sep 25, 2019

JennyChenyj Sep 25, 2019 •

edited

AdamBrousseau Sep 27, 2019

JennyChenyj commented Sep 27, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau left a comment

pshipton commented Sep 30, 2019

JennyChenyj commented Sep 30, 2019

JennyChenyj commented Oct 4, 2019

Created a general function for retrying commands #7208

Created a general function for retrying commands #7208

Conversation

JennyChenyj commented Sep 24, 2019

JennyChenyj commented Sep 24, 2019

JennyChenyj commented Sep 24, 2019

AdamBrousseau Sep 24, 2019

Choose a reason for hiding this comment

AdamBrousseau Sep 24, 2019

Choose a reason for hiding this comment

AdamBrousseau Sep 24, 2019

Choose a reason for hiding this comment

JennyChenyj Sep 24, 2019

Choose a reason for hiding this comment

AdamBrousseau left a comment

Choose a reason for hiding this comment

pshipton commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019

AdamBrousseau commented Sep 25, 2019

JennyChenyj commented Sep 25, 2019 • edited

AdamBrousseau commented Sep 25, 2019

AdamBrousseau Sep 25, 2019

Choose a reason for hiding this comment

JennyChenyj Sep 25, 2019 • edited

Choose a reason for hiding this comment

AdamBrousseau Sep 27, 2019

Choose a reason for hiding this comment

JennyChenyj commented Sep 27, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau commented Sep 30, 2019

AdamBrousseau left a comment

Choose a reason for hiding this comment

pshipton commented Sep 30, 2019

JennyChenyj commented Sep 30, 2019

JennyChenyj commented Oct 4, 2019

JennyChenyj commented Sep 25, 2019 •

edited

JennyChenyj Sep 25, 2019 •

edited