Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set up automation to comment on PRs w/ links to downstream jobs #21795

Closed
nickboldt opened this issue Oct 31, 2022 · 13 comments
Closed

Set up automation to comment on PRs w/ links to downstream jobs #21795

nickboldt opened this issue Oct 31, 2022 · 13 comments
Assignees
@nickboldt
Labels
area/ci CI build and releases, PR testing, & whitelabel/productization issues kind/enhancement A feature request - must adhere to the feature request template. severity/P2 Has a minor but important impact to the usage or development of the system. sprint/current

Comments

@nickboldt
Copy link
Contributor

nickboldt commented Oct 31, 2022 •
edited

Is your enhancement related to a problem? Please describe

Purpose here is to automate the flow of information so that when a PR merge / commit happens upstream, that SHA / branch can be more easily associated w/ a downstream DS 3.x build.

Repos affected include:

  • che-incubator/*
  • eclipse-che/*
  • redhat-developer/devspaces (registries)
  • redhat-developer/devspaces-theia

For these repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

  • redhat-developer/devspaces-images
  • redhat-developer/devspaces-chectl

Describe the solution you'd like

Downstream Jenkins job could...

  • check out upstream code

  • find pull request #

  • use GH api to comment on the PR with links to the job URL (console + changes); see also https://kb.novaordis.com/index.php/Jenkins_currentBuild

  • when the job triggers its downstream process, could then pass the upstream PR URL to the next job, and add more comments as each build completes.

  • you would then end up with 2-4 comments on the PR for each build, ending with a link to the quay image (and possibly the IIBs associated w/ the build too)

Describe alternatives you've considered

GHA that watches for changes in quay?

Additional context

Discussed during this current sprint's retrospective last week
@nickboldt nickboldt added the kind/enhancement A feature request - must adhere to the feature request template. label Oct 31, 2022
@nickboldt nickboldt changed the title Set up GH Actions to comment on upstream and downstream PRs Set up automation to comment on PRs w/ links to downstream jobs Oct 31, 2022
@che-bot che-bot added the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Oct 31, 2022
@nickboldt nickboldt added severity/P2 Has a minor but important impact to the usage or development of the system. area/ci CI build and releases, PR testing, & whitelabel/productization issues labels Oct 31, 2022
@dkwon17 dkwon17 removed the status/need-triage An issue that needs to be prioritized by the curator responsible for the triage. See https://github. label Nov 1, 2022
@nickboldt nickboldt mentioned this issue Nov 1, 2022
Closed
73 tasks
@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 16, 2022 •
edited

Initial prototype: jenkinsfile to check out a project from a branch, switch to a tag if applicable, and then using that SHA, comment on any applicable pull request.

https://gist.github.com/nickboldt/efb8d1b3a60c079e46b5b7aab4412e22

Tested with: SHA = 58d8b4407ab5d2b9a696236202308d92d3e25340
and comment: redhat-developer/devspaces#848 (comment)

TODO:

Phase 2:

  • extract JIRA IDs from the PR / commits, and comment on JIRAs w/ links to builds

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022
@nickboldt
chore: add methods for PR comment automation (eclipse-che/che#21795)
7bcb35d 
Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2
Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt nickboldt mentioned this issue Nov 17, 2022
Merged
9 tasks
@devstudio-release devstudio-release mentioned this issue Nov 17, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022
@nickboldt
chore: add methods for PR comment automation (eclipse-che/che#21795)
4c1fa8e 
Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2
Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 17, 2022 •
edited

new methods are added to util2.groovy via this PR: https://github.com/redhat-developer/devspaces/pull/850/files

  • commentOnPullRequestBuildLinks
  • commentOnPullRequestBuildDescription

Plus utilities:

  • commentOnPullRequest(String comments_url, String message)
  • commentOnPullRequest(String ownerRepo, String SHA, String message)
  • defaultPullRequestComment
  • prepareHTMLStringForJSON

DONE:

TODO:

  • use new methods in various jobs
  • update job to pass PR_Comment_URL to downstream jobs
  • update downstream jobs check if PR_Comment_URL already set; use if found else generate a new one if not

Phase 2:

  • extract JIRA IDs from the PR / commits, and comment on JIRAs w/ links to builds

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022
@nickboldt
chore: add methods for PR comment automation (eclipse-che/che#21795)
9a9030d 
Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2
Signed-off-by: Nick Boldt <nboldt@redhat.com>
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 17, 2022
@nickboldt
chore: add methods for PR comment automation (eclipse-che/che#21795) (#…
fff7ef1 
…850)

Change-Id: Iad6e4c9def1d1340e97c1059c5d02e40ad40eaa2
Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt
Copy link
Contributor Author

Enabling PR comments for template, sync-to-downstream, get-sources-rhpkg, and push-latest jobs for 3.x:

https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1409

Affected (13):

  • code
  • configbump
  • dashboard
  • devfileregistry
  • idea
  • imagepuller
  • machineexec
  • operator-bundle
  • operator
  • plugin reg
  • server
  • trafik
  • udi

TODO:

  • dsc 3.x (triggered by sync-to-downstream if operator-bundle build)
  • apply changes to 3.3 jobs

Other jobs to consider enabling:

  • copyIIBsToQuay
  • update-digests
  • theia-akamai
  • theia-sources

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
update che-devfile-registry/webhook_trigger.txt to cause a build to t…
2c4e94b 
…rigger so we can see what happens in eclipse-che/che#21795

Change-Id: I803297818611a30938f0f7ad15e33a39d73a817b
Signed-off-by: Nick Boldt <nboldt@redhat.com>
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
4c3096f 
…eclipse-che/che#21795

Change-Id: I405ed5bba2a57be36f968f2de8c4c1c555ebb714
Signed-off-by: Nick Boldt <nboldt@redhat.com>
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
afca9df 
…eclipse-che/che#21795 [devfile reg]

Change-Id: I51469f618ffcef37d1e684ad6bef9eb1d598e665
Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
4b489c1 
…eclipse-che/che#21795 [devfile reg] (#851)

Change-Id: I51469f618ffcef37d1e684ad6bef9eb1d598e665
Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
05420ac 
…eclipse-che/che#21795 [plugin reg]

Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252
Signed-off-by: Nick Boldt <nboldt@redhat.com>
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
4cfad4c 
…eclipse-che/che#21795 [plugin reg] (#852)

Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252
Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt
Copy link
Contributor Author

Testing comments on PRs from Jenkins in these two PRs:

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
Revert "no-op change to cause a build to trigger so we can see what h…
fdc5dde 
…appens in eclipse-che/che#21795 [plugin reg] (#852)"

This reverts commit 4cfad4c.
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
Revert "no-op change to cause a build to trigger so we can see what h…
6b4ac6a 
…appens in eclipse-che/che#21795 [plugin reg] (#852)" (#853)

This reverts commit 4cfad4c.
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
c6007f8 
…eclipse-che/che#21795 [plugin reg]

Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252
Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
6304352 
…eclipse-che/che#21795 [plugin reg] (#856)

Change-Id: I59a88d6f55d54f7eda87b0b91e5baec15c3e9252
Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>

Signed-off-by: Nick Boldt <nboldt@redhat.com>
@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 18, 2022 •
edited

Didn't work, need full 40-char SHAs:

https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1410

And need to check .merge_commit_sha:

redhat-developer/devspaces@fe653f6

Trying again...

redhat-developer/devspaces#856

image

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
abef6ea 
…eclipse-che/che#21795 [plugin reg]

This reverts commit c6007f8.
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
9f3c97f 
…eclipse-che/che#21795 [plugin reg] (#857)

This reverts commit c6007f8.
@nickboldt
Copy link
Contributor Author

Worked for plugin reg comment, but didn't work for sync-to-downstream.

Fix: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1414/diffs

Testing:

https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/pluginregistry_3.x/125/ --> redhat-developer/devspaces#857

nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
93b6c9d 
…eclipse-che/che#21795 [plugin reg]

This reverts commit 9f3c97f.
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
ed3a9fc 
…eclipse-che/che#21795 [plugin reg] (#858)

This reverts commit 9f3c97f.
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
3102bb5 
…eclipse-che/che#21795 [plugin reg] (#857)

This reverts commit c6007f8.
@nickboldt nickboldt mentioned this issue Nov 18, 2022
Merged
9 tasks
nickboldt added a commit to redhat-developer/devspaces that referenced this issue Nov 18, 2022
@nickboldt
no-op change to cause a build to trigger so we can see what happens in …
786cadc 
…eclipse-che/che#21795 [plugin reg] (#857) (#859)

This reverts commit c6007f8.
@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 18, 2022 •
edited

Working for the test example eclipse-che/che-machine-exec#225

image

However the links to the job need to be regex-replaced so they are absolute links, not relative ones.

TODO:

  • fix links in comments to be absolute refs to Jenkins, not relative paths that are 404'd on github.com

  • add logic to copyIIBsToQuay so the list of IIBs appears in the PR

  • add logic to dsc 3.x job (triggered by sync-to-downstream if operator-bundle build)

  • add logic & webhook to operator-bundle job ?

Then:

  • backport changes to 3.3 jobs (if necessary?)

For these 2 repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

  • redhat-developer/devspaces-images

  • redhat-developer/devspaces-chectl

Other jobs to consider enabling:

  • update-digests (to connect bundle builds to the builds that triggered them)

  • theia-sources, theia-akamai (deprecated but still useful?) [Artem says don't bother for now] :D

@nickboldt
Copy link
Contributor Author

  • Improved links to Jenkins so they all work now.
  • added status badges and indented build status so it's easier to see start/end comments

image

@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 21, 2022 •
edited

Testing with: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/push-latest-container-to-quay_3.x/1099/

Results pushed to: eclipse-che/che-machine-exec#229 ... but also to eclipse-che/che-operator#1564 (comment)

so we need to tweak the template_3.x job so that it checks if comments_url is set BEFORE just searching for the latest commit from the current source repo, instead of this:

            // comment on the PR with links to this job
            comments_url=util.commentOnPullRequestBuildLinks(SOURCE_REPO,util.getLastCommitSHA(SOURCE_DIR,40))
            println("Computed PR comment URL: " + comments_url)
            comments_url=comments_url.replaceAll("#.+","")

@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 22, 2022 •
edited

Fix: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1427 + https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1428 + https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/merge_requests/1429

Have verified that traefik rebuilds (because they're keyed to a tag, not a branch) will no longer put comments on PRs, per redhat-developer/devspaces-images#357 and traefik/traefik#9515 (comment)

Building: https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/push-latest-container-to-quay_3.x/1103/console

Results should be here: eclipse-che/che-machine-exec#229 (comment) ... but not eclipse-che/che-operator#1564 ... so there's still a bug causing existing comments_url values to be overwritten :(

DONE:

  • add comments_url logic to dsc, operator-bundle, update-digests

For these 2 repos, we will need a secondary webhook setup that can watch for config changes but NOT changes already synced from upstream, eg., a config change like redhat-developer/devspaces-images#353

  • redhat-developer/devspaces-images, eg., for sync*.sh changes in operator or operator-bundle
  • redhat-developer/devspaces-chectl

==> moved to https://issues.redhat.com/browse/CRW-3550

  • ensure copyIIBs job comments 2x on the PR (start w/ links and end w/ status)

image

This was referenced Nov 22, 2022
Closed
Closed
@nickboldt
Copy link
Contributor Author

this PR eclipse-che/che-machine-exec#230 shows a nearly perfect build record:

  • machineexec -> sync-to-down -> get-sources-rhpkg
  • push-latest
  • copyIIBs (job failed, unrelated reasons)

Then a second build cycle started for operator-bundle:

  • operator-bundle (missing the initial comment, but the status at the end appears)
  • sync-to-down (missing the initial comment, but the status at the end appears)
  • get-sources-rhpkg (missing the initial comment, but the status at the end appears)
  • dsc (both link and description messages)

@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 22, 2022 •
edited

even better build record: eclipse-che/che-server#386 (dsc job failed, copyIIB successful 2x!)

DONE:

@nickboldt
Copy link
Contributor Author

nickboldt commented Nov 23, 2022 •
edited

Followup enhancements:

  • CRW-3581 enable github to jira integration (openshift-ci)
  • CRW-3601 in PR comments, switch from comments API to review-comments so we can update 'job started' to 'job done' messages

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@nickboldt nickboldt

Labels
area/ci CI build and releases, PR testing, & whitelabel/productization issues kind/enhancement A feature request - must adhere to the feature request template. severity/P2 Has a minor but important impact to the usage or development of the system. sprint/current
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@nickboldt @ibuziuk @che-bot @dkwon17