[BEAM-8337] Hard-code Flink versions.

[ibzib]

Also publish the RC images instead of rebuilding.

There were problems with the bash code for munging Flink versions, so I got rid of it and replaced with a hard-coded list. The release manager will have to manually validate that they're correct, but I think that's a good idea anyway.

@Hannah-Jiang @udim @mxm

---

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

• Choose reviewer(s) and mention them in a comment (R: @username).
• Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
• If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Flink	Gearpump	Samza	Spark
Go		---	---		---	---
Java
Python		---			---	---
XLang	---	---	---		---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

[mxm]

What were the problems? I'd prefer not hardcoding the versions. If there are problems with this, I'd reckon they are just a symptom of other non-deterministic patterns in the release scripts.

[udim]

What were the problems? I'd prefer not hardcoding the versions. If there are problems with this, I'd reckon they are just a symptom of other non-deterministic patterns in the release scripts.

The issues were with creating FLINK_VER as a bash array from subshell output. I recommend using strings instead. I don't mind hard-coded versions for now.

[mxm]

Seems fair to change this if the bash version approach is too fragile.

[mxm]

Looks like the indention is off here (3 instead of 2 spaces).

[ibzib]

We could've fixed the bash instead. I prefer this way because the publish script no longer has to either make a clone or be run from within an up to date clone, instead it can be run from anywhere.

…

On Fri, Jan 10, 2020 at 10:20 AM Maximilian Michels < ***@***.***> wrote: ***@***.**** approved this pull request. Seems fair to change this if the bash version approach is too fragile. ------------------------------ In release/src/main/scripts/publish_docker_images.sh <#10549 (comment)>: > FLINK_IMAGE_NAME=apachebeam/flink${ver}_job_server + + # Pull verified RC from dockerhub. + docker pull "${FLINK_IMAGE_NAME}:${RELEASE}_${RC_VERSION}" + + # Tag with ${RELEASE} and push to dockerhub. + docker tag "${FLINK_IMAGE_NAME}:${RELEASE}_${RC_VERSION}" "${FLINK_IMAGE_NAME}:${RELEASE}" Looks like the indention is off here (3 instead of 2 spaces). — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#10549?email_source=notifications&email_token=AD2L6NW5MTEVYQ7UWB5WFBDQ5C37BA5CNFSM4KFBDZ2KYY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOCRMGJQQ#pullrequestreview-341337282>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD2L6NS4R45EHTHKNBYV2ZLQ5C37BANCNFSM4KFBDZ2A> .

[Hannah-Jiang]

Looks good to me.
One comment is, do we need to update release guide as well?

[ibzib]

One comment is, do we need to update release guide as well?

I see that the scripts are also written in markdown within the release guide "run all steps manually" section. I'm not sure why that section's there, however. Is there a difference between that and the contents of the scripts?

[Hannah-Jiang]

One comment is, do we need to update release guide as well?

I see that the scripts are also written in markdown within the release guide "run all steps manually" section. I'm not sure why that section's there, however. Is there a difference between that and the contents of the scripts?

This script is used when release it manually, for example, the script failed during running and release manager release left part manually. So we need to keep it consistent. However, I think this part can be improved.

[ibzib]

So we need to keep it consistent

I'd prefer to keep only the full scripts as the single source-of-truth for what needs to be run. But that's out of scope for this PR.

[ibzib]

For now, I've updated the release guide. For future discussion/action on simplifying the release guide, I filed https://issues.apache.org/jira/browse/BEAM-9087.