Refactor build and release workflows

[aalmiray]

Refactors duplication found in verify.yaml by using a matrix.
Configures the JReleaser Maven plugin to handle Git releases.

Fixes #542

[aalmiray]

Converting to draft as there's still work to be done.

Requires a release of jreleaser/jreleaser#658
Missing configuration for publishing Homebrew formula to a homebrew tap.
Missing GH workflow for prerelease (before a candidate is voted) and release (update a successfully voted candidate)

[aalmiray]

Added Homebrew and Sdkman configuration

Both release-candidate and release workflows expect that a tag has been already created (following the Apache release process) and thus they must be triggered manually. If you feel like release-candidate should be triggered with a tag then we can update the trigger condition. However the release workflow may remain with a manual trigger as it should only be run once a vote has been successfully passed.

[ppalaga]

Thanks a lot for your work, @aalmiray !

If you feel like release-candidate should be triggered with a tag then we can update the trigger condition.

Id vote for triggering by a tag, but how about the third option: let release-candidate create the tag itself?
The tagging steps we used to use so far are in https://github.com/apache/maven-mvnd/blob/master/build/release.sh - that file could be removed then.

[aalmiray]

Id vote for triggering by a tag, but how about the third option: let release-candidate create the tag itself?

This is of course possible but here I wonder how will the workflow interact with the Apache Maven release process? I'm not familiar with it so that's why I did not push to automate that part just yet. Who and when is responsible for tagging a repository before a vote is started?

If the tag happens earlier in the process and IF this workflow creates the tag then the changelog will be ready before the e-mail vote is sent, right? Perhaps @hboutemy may help us clarify this point 😄

[gnodet]

Id vote for triggering by a tag, but how about the third option: let release-candidate create the tag itself?

This is of course possible but here I wonder how will the workflow interact with the Apache Maven release process? I'm not familiar with it so that's why I did not push to automate that part just yet. Who and when is responsible for tagging a repository before a vote is started?

If the tag happens earlier in the process and IF this workflow creates the tag then the changelog will be ready before the e-mail vote is sent, right? Perhaps @hboutemy may help us clarify this point 😄

I think we can somehow mimic the process with pure maven: when mvn release:prepare release:perform is run, the invocation will change the version, create a tag, upload to nexus staging repo. A vote is then started and at the end, the staging repo is published to central and the release to its final destination in the ASF public distribution area.

So I think a manual trigger which would set the version and create a tag before building the binaries is fine. We then need to be able to review and vote on the packages which will be published during a next step.

[aalmiray]

Right. If the maven release plugin is used then a tag and a new version will be already set. The release-candidate workflow would have to be changed to be triggered on tag creation and keep JReleaser from creating a tag (as it's already there).

If the release-candidate workflow were the starting point then you'd have to change version from snapshot to release (in all poms), let JReleaser create the tag (or explicit tag in the workflow), issue a deploy to Maven Central (not using the release plugin), and to be honest it already sounds complicated, at least to me.

Starting the release process with the maven release plugin, having the release-candidate be triggered on tag creation sounds better to me from a minimalistic automation POV, less moving parts IMHO.

[gnodet]

Right. If the maven release plugin is used then a tag and a new version will be already set. The release-candidate workflow would have to be changed to be triggered on tag creation and keep JReleaser from creating a tag (as it's already there).

If the release-candidate workflow were the starting point then you'd have to change version from snapshot to release (in all poms), let JReleaser create the tag (or explicit tag in the workflow), issue a deploy to Maven Central (not using the release plugin), and to be honest it already sounds complicated, at least to me.

Starting the release process with the maven release plugin, having the release-candidate be triggered on tag creation sounds better to me from a minimalistic automation POV, less moving parts IMHO.

Sorry if I've been misunderstood. I did not imply we should keep the maven release process, but I was only describing the usual release process for simple maven based projects. So a manual trigger can be used as the starting point and have jreleaser create the tag itself as part of its release process. We just need to be able to vote on the packages (so they need to be built and staged) before being published.

[aalmiray]

Got it. But you do raise a point with moving the version to the next available number and publishing artifacts to Maven Central.

Version bump can happen with the versions plugin in which case you must set the number. Same goes by setting next snapshot. At least 2 different invocations of mvn. The release plugin does this for you in 1 invocation (assuming build success).

If mvnd does not deploy artifacts to Maven Central then the release plugin may still be used for handling the numbering scheme while disabling the deploy phase so that no artifacts are pushed to MC, isn't that the case?

So we have a few options:

• use the maven release plugin. it'll set the version to release, tag, bump version to next. the tag triggers the release-candidate workflow which only creates the git release with changelog.

• use manual steps with versions plugin to set the version. JReleaser tags and creates git release with changelog. you may or may not bump the version to next, again using the versions plugin.

Both options may be triggered from GH actions, matter of fact I'd say it's a must for option 2.

Option 1. can still be triggered locally by the release master as the release-candidate workflow would have a tag trigger that does not care if the tag was made locally or on CI.

[ppalaga]

Yes, you are right both release:prepare and versions:set (multiple invocations; combined with git tag) can be used to produce the same result. Yes, release:prepare means a little bit less typing and can therefore be preferred.

In your proposed solution, would release:prepare be called from within GH workflow, or locally? - I could live with both.

[aalmiray]

OK, I can adjust the workflows to make use of a tag trigger for release-candidate.

If release:prepare were to be invoked by a GH workflow then we'll need to reuse the custom GitHub Personal Access token, as otherwise a tag created with workflow A won't trigger workflow B, that is, the default ${{ secrets.GITHUB_TOKEN }} is not enough in this case.

Is that acceptable?

[aalmiray]

Refactored workflows after testing some settings that were not properly working.

• verify.yaml renamed to early-access.yml.
• there's now a 2 stage release-candidate workflow that's triggered manually. stage1 builds, updates version, and tags (using the release plugin). stage 2 creates the Git release with changelog on tag. The split lets the release master perform stag1 manually if they so choose so, while stage2 it's triggered automatically when the tag is pushed.
• release workflow builds binaries and updates the existing release.

Note the use of a GIT_ACCESS_TOKEN secret, this is a PAT required to trigger stage2 by stage1, also to push the brew formula to the given repository.

[ppalaga]

Looks good, thanks!

[aalmiray]

Have to release JReleaser 1.0.0-M1 before this PR can be merged. The release should happen within the next 2 weeks.

In the meantime, let me know of any adjustments should be made.

[aalmiray]

PR is ready for review.

Notes:

• publication to Homebrew requires a Github Personal access Token with write access to the mvndaemon organization. Currently the secrete name is expected to be GIT_ACCESS_TOKEN.
• Chocolatey publication is not yet configured as an account & API key is required. this account should be set up and owned by someone in the mvnd team (@ppalaga, @gnodet).

[aalmiray]

@ppalaga @gnodet I've refreshed this PR with latest changes from master.

Do you have some time to review the proposed changes?

[ppalaga]

Looks good, thanks!

[ppalaga]

@gnodet are there any plans to release any time soon?

[gnodet]

@gnodet are there any plans to release any time soon?

Yes, a release is long overdue. What's the process now ?

[aalmiray]

Once merged, the early-access.yml workflow will kick in and generate a prerelease titled "Release early-access" with the latest binaries. This is independent of the Apache Maven release process.

Then, for a regular release, a release candidate must be voted. Manually trigger the release-candidate-stage1.yml workflow to generate the binaries. This workflow creates a git release in prerelase mode if not mistaken. Once the vote passes then manually trigger the release.yml workflow.

BUT before you do trigger please review the settings for Homebrew and Sdkman. A personal Git access token is required to push the formula to the target Git repository.

My suggestion would be to merge this PR and let the early-access release do its work. We can then discuss the steps to post a release following the Apache Maven release process (of which I've never been a party of). We might need the help of an Apache Maven guide (@hboutemy?)

[hboutemy]

the process is here https://maven.apache.org/developers/release/maven-project-release-procedure.html
it should not change much even if JReleaser helps at automating a few parts (or we'll discover where it still requires future improvements)
What is important is to execute the vote for minimum 72h on a release (candidate) available somewhere in source-release archive form (be it Apache's repository or anywhere else): based on the vote result, this release (candidate) will be considered failed then dropped, or successful then it's only at that time that the full scale public publication will happen (the key Apache part being that sous-release archive is published on Apache dist)

[gnodet]

@aalmiray could you restore the test-jdk11 which run tests against PRs ?

[gnodet]

@aalmiray could you restore the test-jdk11 which run tests against PRs ?

Ah, sorry, I think tests are now run with the early-access script.

[gnodet]

I've reverted the merge. I've raised https://issues.apache.org/jira/browse/INFRA-23128 to whitelist the required action before we can attempt to merge again.