Change release branching strategy to create release-1.x branch late

[saschagrunert]

Summary

Right now we're creating the release branch at the same date as we cut the v1.x.0-beta.0 release. After that release we will fast-forward (merge the latest changes from the master branch) periodically into the release branch manually.

The main idea is now to cut the release branch much later, whereas ideally we could get away from the branch fast-forward process.

Discussion

Reasons for creating the release branch early:

• It gives extra CI signal (redundant test of same commits on different branches)
• We have extra eyes on code coming from master to release branch (ie: look at all merged commits and affirm they should technically go to the release branch)
• We practice operationally the tooling and processes around shifting content from master to release branch

Open Questions

• What would be the right time to cut the release branch? (Code thaw might be too late, whereas code freeze might be sufficient)
• What needs to be done from a release engineering perspective?
• What are the dependencies (some projects might rely on early-cut Kubernetes release branches)?

[justaugustus]

@kubernetes/release-engineering -- I've left a review on the proposed 1.19 release schedule about this.
PTAL and add feedback there.

[justaugustus]

/priority important-soon
/assign @justaugustus @tpepper @cpanato @saschagrunert @hasheddan

[justaugustus]

• What would be the right time to cut the release branch? (Code thaw might be too late, whereas code freeze might be sufficient)

We decided on Code Freeze, starting in Kubernetes 1.19.
More details here: https://groups.google.com/d/topic/kubernetes-dev/IVpiIOZ4WcM/discussion

• What needs to be done from a release engineering perspective?

@saschagrunert -- I think you summed it up nicely in #1281 (comment).

Proposal for the branch cutting changes:

1. Add a new variant `beta` to the current `krel gcbmgr --type` flag, which is currently:
   `--type string            Release type (must be one of: 'prerelease', 'rc', 'official')`

I would propose here that we rename this flag to stage, supporting the following options:

• alpha
• beta
• rc
• official

The prerelease designation becomes fairly meaningless if stands to denote alpha.
It's better to just be explicit.

2. Pass that flag via the GCB substitutions to anago

Agreed.

3. Modify anago (this seems to be the funny part) to not enforce `alpha` releases on the master and allow betas by taking the new release type into account. I guess one (of some) critical code part(s) is here: https://github.com/kubernetes/release/blob/c600d12013979543fcfab24c977ffae158405a21/anago#L471-L492

Explicitly, the following branches need to support the following release stages:

• master: alphas and betas
• release-x.y: RCs and official

Finally, an official release would need to also cut the next RC.

I think that we have to consider whether we want to conditionally restrict these changes to Kubernetes 1.19 or enable them everywhere.

For simplicity, I prefer the latter.
As far as I can remember, we haven't cut additional betas/RCs on a release branch, outside of the minor cycle.

@saschagrunert -- Do you have bandwidth to tackle this within the next few days? We need to land the changes ahead of the 1.19.0-beta.0 release.

[saschagrunert]

@saschagrunert -- Do you have bandwidth to tackle this within the next few days? We need to land the changes ahead of the 1.19.0-beta.0 release.

Yes, I'll propose the first PR later today.

[justaugustus]

This still needs some work. Looking into it now.
/reopen

[k8s-ci-robot]

@justaugustus: Reopened this issue.

In response to this:

This still needs some work. Looking into it now.
/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[justaugustus]

command	link	expected	status
krel gcbmgr --stage --type alpha	https://console.cloud.google.com/cloud-build/builds/6c1441e5-d286-4617-a23c-120a848776be?project=kubernetes-release-test	pass	pass
krel gcbmgr --stage --type alpha --branch release-1.19	https://console.cloud.google.com/cloud-build/builds/21a0cdc8-fa07-4a92-8dbf-6ae19c310aa5?project=kubernetes-release-test	fail: no alphas on release branches	pass
krel gcbmgr --stage --type beta	https://console.cloud.google.com/cloud-build/builds/cf512ac0-3cd0-4d96-a919-b60896ff455d?project=kubernetes-release-test	pass	pass
krel gcbmgr --stage --type beta --branch release-1.19	https://console.cloud.google.com/cloud-build/builds/5c49bd39-3c11-4919-978b-73e743cbcc4a?project=kubernetes-release-test	fail: no betas on release branches	pass
krel gcbmgr --stage --type rc	https://console.cloud.google.com/cloud-build/builds/ca6586b2-f54d-4ab5-9ed5-1e08d16b8434?project=kubernetes-release-test	fail: no RCs on master	pass
krel gcbmgr --stage --type rc --branch release-1.19	https://console.cloud.google.com/cloud-build/builds/723d874d-9714-49ca-8388-94498b8920a7?project=kubernetes-release-test	pass	pass
krel gcbmgr --stage --type official	https://console.cloud.google.com/cloud-build/builds/fd25cb20-d237-4f95-9b11-e32e1b8eb092?project=kubernetes-release-test, https://console.cloud.google.com/cloud-build/builds/74e4d782-0efc-4bc3-b25a-3d36f520d7a3?project=kubernetes-release-test	fail: must specify release branch	pass
krel gcbmgr --stage --type official --branch release-1.19	https://console.cloud.google.com/cloud-build/builds/75a5ca86-0886-46f8-801a-acbbbdf5481a?project=kubernetes-release-test	fail: release branch does not exist yet	fail
krel gcbmgr --stage --type official --branch release-1.18	https://console.cloud.google.com/cloud-build/builds/f46f4cec-8d84-4ddc-99c2-d43a910ff0aa?project=kubernetes-release-test	pass	pass

[justaugustus]

Added some validation in krel gcbmgr for the scenarios above that should've failed: #1293