Community repos and Mandrel release automation

[Karm]

Hello,

mandrel-ops.java

This begun as a meshup of various incarnations of release-scripts/mandrel-release.java in January.
Then I went back to it with April CPU and this time around I haven't done any step of the release manually.
Thx @jerboaa for watching over e.g. misunderstandings like graalvm/mandrel#958 (comment) that I took one by one and incorporated into the correct form here.

This PR is not a giant "proposal of a process", it is the process as we already did it, documented in README.md.

README.md

The doc, aided by Google Gemini and reviewed by me, enumerates the available command args.
At the beginning of the doc, I wrote down the process as a numbered list, including thing like the convention that we wait with "Unmark" PR merging until at least one vendor does a successful release, e.g. noted here: graalvm/graalvm-community-jdk21u#278 (review) or here graalvm/graalvm-community-jdk25u#37 (comment).

I'd like to kindly ask you to scroll the list from "Before you start" point to the "Sync Upstream". It enumerates the functions and gives an overview.

test-mandrel-ops.java

Ever since the inception of the now surpassed mandrel-release.java it has been a huge pain for me to change anything and test it on dummy/fake repos. This test creates dummy repos with some basic test content, e.g. suite.py to test version bumping and conflict resolution, Quarkus Images yaml to test updating sha256 for Quarkus Images PR etc.

Each part of this April release was done with this script, although not exactly this version as I have been tinkering with it and hopefully improving it. Hence the test suite.

Last but not least, the sync-upstream command that e.g. opened
graalvm/mandrel#971
graalvm/mandrel#972
should be usable for general sync PR from community repos to Mandrel using the --since flag, but I used that only in the testing fake repos, not in production yet.

I realize it's a lot, but it works and it shouldn't be a huge pain to review 🙏

Cheers
K.

[Karm]

Issues:

• sync upstream should add PR to the open milestone
• release: False should not be added to suite.py where is wasn't in the first place

[Karm]

Issues:

* sync upstream should add PR to the open milestone

* release: False should not be added to suite.py where is wasn't in the first place

Fixed in b3f4c7b

[Karm]

Issue:

Make sync-upstream handle empty changeset and test --since with an empty chnage set.

[zakkak]

Nitpick: draft-release might be a better name for this command.

[Karm]

Not sure...it ends up as Draft, but it's actively uploading the artifacts already.

[zakkak]

No strong opinion, I am just more afraid of running publish-release vs draft-release :)

[zakkak]

Looking at the README file it looks like there is quite some repetition of options/parameters across different commands. I believe introducing an options file and passing it through @-file would help reduce this duplication and simplify both the documentation and the actual workflow. I expect to have one such options file for each release train.

[Karm]

@zakkak Thx for the feedback, we will be back with you shortly! :-D

[zakkak]

@zakkak Thx for the feedback, we will be back with you shortly! :-D

FWIW anything you believe should be handled in a separate PR feel free to tell me so and I will create an issue for it.

[zakkak]

@zakkak Thx for the feedback, we will be back with you shortly! :-D

Hi @Karm, are you still on the line? :)

[Karm]

Looking at the README file it looks like there is quite some repetition of options/parameters across different commands. I believe introducing an options file and passing it through @-file would help reduce this duplication and simplify both the documentation and the actual workflow. I expect to have one such options file for each release train.

I agree, but I'd like to push it to another PR...

#525

[Karm]

@zakkak

I would love it if it could automatically detect the latest merge point and have the --since parameter be optional.

Done in 2dcc6c3

e.g. graalvm/mandrel#985 was generated as:

$ ./mandrel-ops.java sync-upstream --dir /home/karm/tmp/mandrel_25.0 --fork Karm/graal --repo graalvm/mandrel --base-branch mandrel/25.0 --upstream-url https://github.com/graalvm/graalvm-community-jdk25u.git --upstream-branch master
[jbang] Building jar for mandrel-ops.java...
Fetching master from https://github.com/graalvm/graalvm-community-jdk25u.git
No --since provided. Calculating the merge base between downstream and upstream...
Calculated merge base: 3d94ea2b3a1a53081182faac0eaed93db1ea0cad ([CI] Update GitHub Actions(deps): Bump actions/checkout)
Merging upstream branch into current branch.
GPG Key (fingerprint d72501ba9a2a624b000f38858cdbce4379381fc4)
Passphrase: 
Pushing branch to fork git@github.com:Karm/graal.git
Assigning Sync PR to milestone: 25.0.4.0
PR created: https://github.com/graalvm/mandrel/pull/985

It autodetects the appropriate merge root. The script assumes we don't don force pushes and that the history is sane, which is O.K. as we have protection rules.

There are 2 cases when the script gives up and asks for an explicit --since:

1. When it detects a cherry-pick in the history
2. When it detects a squash commit in the history

[Karm]

Test script tests for the --since autodetection too ✔️

[zakkak]

Thank you @Karm