release: less interactive and allow running stages individually

[v1v]

UI

Then the version will be prompted earlier within the checkout.

Issues

Superseedes #2789

[ghost]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-09-28T09:39:18.177+0000

• Duration: 56 min 48 sec

Test stats 🧪

Test	Results
Failed	0
Passed	3152
Skipped	36
Total	3188

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• run benchmark tests : Run the benchmark tests.

• run jdk compatibility tests : Run the JDK Compatibility tests.

• run integration tests : Run the Agent Integration tests.

• run end-to-end tests : Run the APM-ITs.

• run windows tests : Build & tests on windows.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[github-actions]

/test

[jackshirazi]

yes please!

[cachedout]

💯 Nice one.

[SylvainJuge]

Hi,

From what I understand this PR allows to conditionally execute all the steps in the release process, which is already a huge win when things go bad, so there is already a benefit to merge it as-is.

However, I still think that the end goal should be to have a release pipeline with idempotent steps (some are already like the create_update_major_branch that only do something if required), do you know anything that would prevent us to achieve this ? I think that we could even reuse the "restart from stage" feature of Jenkins, but I don't have any experience with it.

[v1v]

do you know anything that would prevent us to achieve this?

IIUC, the issue is mainly the release process in java that requires to push changes with the release and the new version, and generate the changelog on the fly and push it. Those steps might be the ones causing conflicts with the idempotent, though we might need to look into this deeply as I might be wrong.

I personally prefer to delegate that responsibility to a Java Maven plugin in charge of the whole post-release process, this will avoid the caveats with any CI.

In any case, I think this proposal will help with the idea of running the build from a particular stage onwards in the future if that's needed.

"restart from stage" feature of Jenkins,

IIRC, it didn't work well with ephemeral workers. Additionally there are some pre-requisites:

All inputs to the Pipeline will be the same. This includes SCM information, build parameters, and the contents of any stash step calls in the original Pipeline, if specified.

We could give a go the restart and verify whether it works as expected, no strong opinion, though I prefer to play safe for the shake of reliability in the release process, so this PR might be enough, even though if the overhead is when filling the form, on the other side, there are no many releases per year - so far 7 in this year, so maybe it's not worth the invest to major changes knowing there are certain WiP to use a different CI vendor and we have not decided much.

What do you think?

[SylvainJuge]

What do you think?

I think it's fine to keep the proposal (this PR) as-is, even if the Jenkins-specific "restart from stage" might work with the known limitations.

What I wonder though (and could definitely be discussed separately) is how we could build the steps to make them 1) idempotent and 2) maybe reusable if we switch to another CI vendor. We can't delegate all the steps to maven as it's only in charge of the build (for example the github release is not part of it). In my opinion, an ideal release pipeline would only have the target branch and release version as parameters and would execute only the steps that are needed.