Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change GH api used to build release notes; plus refactoring #11645

Merged
merged 1 commit into from
Jul 7, 2023

Conversation

amaltaro
Copy link
Contributor

@amaltaro amaltaro commented Jul 5, 2023

Fixes #11640

Status

ready

Description

Some overall refactoring of this script, it's worth highlighting though:

  • changed GH API from search issues to commits pulls;
  • less temp files.

Is it backward compatible (if not, which system it affects?)

YES

Related PRs

None

External dependencies / deployment changes

None

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: succeeded
    • 1 tests no longer failing
    • 3 changes in unstable tests
  • Python3 Pylint check: succeeded
  • Pylint py3k check: succeeded
  • Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14302/artifact/artifacts/PullRequestReport.html

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amaltaro There are few things I have noticed in this PR. Please take a look before merging. Other than that, it looks good and the results it produces are quite sensible and seem to be solving the issue.


# Grab all the commit hashes, subject and author since the last release
TMP_HASHES_SUBJ_AUTHOR=$(mktemp -t wmcore_hashes.${LASTVERSION}.XXXXX)
git log --no-merges --pretty=format:'%H %s (%an)' ${LASTCOMMIT}.. >> $TMP_HASHES_SUBJ_AUTHOR
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alan, this line does not generate anything until it has the two dots at the end (..), while removing this results in the following output:

$ git log -3 --no-merges  --pretty=format:'%H %s (%an)' ${LASTCOMMIT} 
55d76e3022bd498258767f9c187b4fa02d7a17af 2.2.2rc5 (Alan Malta Rodrigues)
0524a82b734d351b998829ad555704300da78b13 2.2.2rc4 (Alan Malta Rodrigues)
a5a824ab603c060600f7ed3215618f2339560911 Add Docker context path to docker/build-push-action@v1 (Todor Ivanov)

Copy link
Contributor Author

@amaltaro amaltaro Jul 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that is correct. But is it just an observation or are you actually requesting any changes here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry Alan, my bad, I was not considering the fact that $LASTCOMMIT is actually the limiting commit freventing from going through the whole repository history. You may ignore my comment bellow as well.

PR=$(curl -s https://api.github.com/repos/dmwm/WMCore/commits/$HASH_ID/pulls | grep -Po '\"html_url\": \"https://github.com/dmwm/WMCore/pull/\K[0-9]+' | sort | uniq)
# remove hash_id from the commit line
commitline=$(echo $commitline | sed 's+$HASH_ID+ -+')
echo "$commitline #$PR" >> TMP_CHANGES
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing $ sign: $TMP_CHANGES

PR=$(curl -s https://api.github.com/repos/dmwm/WMCore/commits/$HASH_ID/pulls | grep -Po '\"html_url\": \"https://github.com/dmwm/WMCore/pull/\K[0-9]+' | sort | uniq)
# remove hash_id from the commit line
commitline=$(echo $commitline | sed 's+$HASH_ID+ -+')
echo "$commitline #$PR" >> TMP_CHANGES
done
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cycle goes to beginning of time - iterates through the whole repository history and does not stop at some specific commit. I am not sure if this is the expected behavior, but it takes quite a lot of time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The use case is: given a recent list of commit hash ids, fetch their equivalent pull request numbers. Building the release notes takes around a second, so I it's not clear to me what issue you are reporting here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not a problem here... see my comment above

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: succeeded
  • Python3 Pylint check: succeeded
  • Pylint py3k check: succeeded
  • Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14304/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

amaltaro commented Jul 6, 2023

@todor-ivanov I fixed one missing dollar sign that you noted - I think it was in the middle of your review - so you might want to have another look.
Nonetheless, before merging it, I will wait for further commits to get mergeed into the repository and run a final and clean real test before incorporating it.

Copy link
Contributor

@todor-ivanov todor-ivanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @amaltaro this looks perfect now

revert editor

add missing dollar sign

sed with double quotes instead of single

Replace format option from n to N for the author
@amaltaro
Copy link
Contributor Author

amaltaro commented Jul 7, 2023

Now that we have more commits at the HEAD, I could execute the final tests. I had to update 2 lines in the previous code:

  1. the command with sed was not properly expanding the $HASH_ID. Resolved by replacing single quotes with double quotes.
  2. not clear whether it really makes any difference, but when formatting git log to retrieve the author, I switched from %an to %aN. Which is what the script already uses.

Documentation on these log format can be found at: https://devhints.io/git-log-format

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: succeeded
  • Python3 Pylint check: succeeded
  • Pylint py3k check: succeeded
  • Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14312/artifact/artifacts/PullRequestReport.html

@cmsdmwmbot
Copy link

Jenkins results:

  • Python3 Unit tests: failed
    • 1 new failures
    • 1 changes in unstable tests
  • Python3 Pylint check: succeeded
  • Pylint py3k check: succeeded
  • Pycodestyle check: succeeded

Details at https://cmssdt.cern.ch/dmwm-jenkins/view/All/job/DMWM-WMCore-PR-test/14313/artifact/artifacts/PullRequestReport.html

@amaltaro
Copy link
Contributor Author

amaltaro commented Jul 7, 2023

This is working fine now. Merging

@amaltaro amaltaro merged commit 70f7737 into dmwm:master Jul 7, 2023
2 of 4 checks passed
@amaltaro amaltaro changed the title Change GH api to build release notes; plus refactoring Change GH api used to build release notes; plus refactoring Jul 17, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Script to build the CHANGES file - release notes - is malfunctioning
3 participants