contrib: Add release helper scripts

[ upstream commit 5388518 ]

Signed-off-by: Joe Stringer <joe@cilium.io>
Signed-off-by: Michal Rostecki <mrostecki@opensuse.org>

.github/ISSUE_TEMPLATE/release_template.md

@@ -0,0 +1,72 @@
+---
+name: Release a new version of Cilium
+about: Create a checklist for an upcoming release
+title: 'vX.Y.Z release'
+labels: kind/release
+assignees: ''
+
+---
+
+## Pre-release
+
+- [ ] Create a [new project] for the next release version
+- [ ] Add build targets for the new release on [Docker Hub]
+  - All versions:
+    - [cilium](https://hub.docker.com/repository/docker/cilium/cilium/builds/edit)
+    - [operator](https://hub.docker.com/repository/docker/cilium/operator/builds/edit)
+    - [docker-plugin](https://hub.docker.com/repository/docker/cilium/docker-plugin/builds/edit)
+  - Cilium v1.8 or later:
+    - [operator-generic](https://hub.docker.com/repository/docker/cilium/operator-generic/builds/edit)
+    - [operator-aws](https://hub.docker.com/repository/docker/cilium/operator-aws/builds/edit)
+    - [operator-azure](https://hub.docker.com/repository/docker/cilium/operator-azure/builds/edit)
+    - [hubble-relay](https://hub.docker.com/repository/docker/cilium/hubble-relay/builds/edit)
+- [ ] Check that there are no [release blockers] for the targeted release version
+- [ ] Ensure that outstanding [backport PRs] are merged
+- [ ] Consider building new [cilium-runtime images] and bumping the base image
+      versions on this branch
+- [ ] Move any unresolved issues/PRs from old release project into the newly
+      created release project
+- [ ] Push a PR including the changes necessary for the new release:
+  - [ ] Pull latest branch
+  - [ ] Run `contrib/release/start-release.sh'
+  - [ ] (If applicable) Update the `cilium_version` and `cilium_tag` in
+        `examples/getting-started/Vagrantfile`
+  - [ ] Commit all changes with title `Prepare for release vX.Y.Z`
+  - [ ] Submit PR (`contrib/release/submit-release.sh`)
+- [ ] Merge PR
+- [ ] Create and push *both* tags to GitHub (`vX.Y.Z`, `X.Y.Z`)
+  - Pull latest branch locally and run `contrib/release/tag-release.sh`
+- [ ] Wait for docker builds to complete
+  - [cilium](https://hub.docker.com/repository/docker/cilium/cilium/builds)
+  - [operator](https://hub.docker.com/repository/docker/cilium/operator/builds)
+  - [docker-plugin](https://hub.docker.com/repository/docker/cilium/docker-plugin/builds)
+- [ ] Create helm charts artifacts in [Cilium charts] repository using
+      [cilium helm release tool] for both the `vX.Y.Z` release and `vX.Y` branch
+      & push to repository
+- [ ] Run sanity check of Helm install using connectivity-check script.
+      Suggested approach: Follow the full [GKE getting started guide].
+- [ ] Check draft release from [releases] page and publish the release
+- [ ] Announce the release in #general on Slack (only [@]channel for vX.Y.0)
+
+## Post-release
+
+- [ ] Prepare post-release changes to master branch using `contrib/release/bump-readme.sh`
+- [ ] Update the `stable` tags for each Cilium image on Docker Hub (if applicable)
+- [ ] Update external tools and guides to point to the new Cilium version:
+  - [ ] [kops]
+  - [ ] [kubespray]
+
+
+[release blockers]: https://github.com/cilium/cilium/labels/priority%2Frelease-blocker
+[backport PRs]: https://github.com/cilium/cilium/pulls?utf8=%E2%9C%93&q=is%3Aopen+is%3Apr+backports
+[new project]: https://github.com/cilium/cilium/projects/new
+[Cilium release-notes tool]: https://github.com/cilium/release
+[Docker Hub]: https://hub.docker.com/orgs/cilium/repositories
+[Cilium charts]: https://github.com/cilium/charts
+[releases]: https://github.com/cilium/cilium/releases
+[Stable releases]: https://github.com/cilium/cilium#stable-releases
+[kops]: https://github.com/kubernetes/kops/
+[kubespray]: https://github.com/kubernetes-sigs/kubespray/
+[cilium helm release tool]: https://github.com/cilium/charts/blob/master/prepare_artifacts.sh
+[GKE getting started guide]: https://docs.cilium.io/en/stable/gettingstarted/k8s-install-gke/
+[cilium-runtime images]: https://quay.io/repository/cilium/cilium-runtime

Documentation/contributing/release.rst

@@ -96,61 +96,56 @@ If you intent to release a new minor release, see the
           used in the Cilium development process. See :ref:`dev_env` for
           detailed instructions about setting up said VM.
 
-#. Ensure that the necessary backports have been completed and merged. See
-   :ref:`backport_process`.
-#. Checkout the desired stable branch and pull it:
+GitHub template process
+~~~~~~~~~~~~~~~~~~~~~~~
 
-   ::
+#. File a `new release issue <https://github.com/cilium/cilium/issues/new?assignees=&labels=kind%2Frelease&template=release_template.md&title=vX.Y.Z+release>`_
+   on GitHub, updating the title to reflect the version that will be released.
 
-       git checkout v1.0; git pull
+#. Follow the steps in the issue template to prepare the release.
 
-#. Create a branch for the release pull request:
+Reference steps for the template
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-   ::
+#. Ensure that the necessary backports have been completed and merged. See
+   :ref:`backport_process`.
 
-       git checkout -b pr/prepare-v1.0.3
+   #. Update GitHub project and create ``vX.Y.Z+1`` project if applicable.
+   #. Update PRs / issues that were added to the ``vX.Y.Z`` project, but didn't
+      make it into this release into the ``vX.Y.Z+1`` project.
 
-#. Update the ``VERSION`` file to represent ``X.Y.Z+1``
-#. If this is the first release after creating a new release branch. Adjust the
-   image pull policy for all ``.sed`` files in ``install/kubernetes/cilium/values.yaml`` from
-   ``Always`` to ``IfNotPresent``.
-#. Update the image tag versions in the examples:
+#. Create a new project named "X.Y.Z+1" to automatically track the backports
+   for that particular release. `Direct Link: <https://github.com/cilium/cilium/projects/new>`_
 
-   ::
+#. Checkout the desired stable branch and pull it:
 
-       make -C install/kubernetes clean all
+   ::
 
-#. Update the ``cilium_version`` and ``cilium_tag`` variables in
-   ``examples/getting-started/Vagrantfile``
+       git checkout v1.0; git pull
 
-#. Update the ``AUTHORS file``
+#. Run the release preparation script:
 
    ::
 
-       make update-authors
-
+      contrib/release/start-release.sh
 
-   .. note::
+  .. note::
 
        Check to see if the ``AUTHORS`` file has any formatting errors (for
        instance, indentation mismatches) as well as duplicate contributor
        names, and correct them accordingly.
 
-
-#. Generate the release notes by running the instructions provided in github.com/cilium/release
-
-#. Add the generated release notes in the ``CHANGELOG.md`` file
+#. Update the ``cilium_version`` and ``cilium_tag`` variables in
+   ``examples/getting-started/Vagrantfile``
 
 #. Add all modified files using ``git add`` and create a pull request with the
-   title ``Prepare for release v1.0.3``. Add the backport label to the PR which
-   corresponds to the branch for which the release is being performed, e.g.
-   ``backport/1.0``.
+   title ``Prepare for release v1.0.3``.
 
-   .. note::
+#. Prepare a pull request for the changes:
 
-       Make sure to create the PR against the desired stable branch. In this
-       case ``v1.0``
+   ::
 
+      contrib/release/submit-release.sh
 
 #. Follow standard procedures to get the aforementioned PR merged into the
    desired stable branch. See :ref:`submit_pr` for more information about this
@@ -162,12 +157,11 @@ If you intent to release a new minor release, see the
 
        git checkout v1.0; git pull
 
-#. Create release tags:
+#. Create and push release tags to GitHub:
 
    ::
 
-       git tag -a v1.0.3 -m 'Release v1.0.3'
-       git tag -a 1.0.3 -m 'Release 1.0.3'
+      contrib/release/tag-release.sh
 
    .. note::
 
@@ -177,62 +171,81 @@ If you intent to release a new minor release, see the
        ``x.y.z`` For more information about how ReadTheDocs does versioning, you can
        read their `Versions Documentation <https://docs.readthedocs.io/en/latest/versions.html>`_.
 
-#. Build the binaries and push it to the release bucket:
+#. Wait for DockerHub to prepare all docker images.
 
-   ::
+#. `Publish a GitHub release <https://github.com/cilium/cilium/releases/>`_:
 
-       DOMAIN=releases.cilium.io ./contrib/release/uploadrev v1.0.3
+   Following the steps above, the release draft will already be prepared.
+   Preview the description and then publish the release.
 
+#. Prepare Helm changes for the release using the `Cilium Helm Charts Repository <https://github.com/cilium/charts/>`_
+   and push the changes into that repository (not the main cilium repository):
 
-   This step will print a markdown snippet which you will need when crafting
-   the GitHub release so make sure to keep it handy.
+   ::
+
+      ./prepare_artifacts.sh /path/to/cilium/repository/checked/out/to/release/commit
+      git push
 
-   .. note:
+#. Prepare Helm changes for the dev version of the branch using the `Cilium Helm Charts Repository <https://github.com/cilium/charts/>`_
+   for the vX.Y helm charts, and push the changes into that repository (not the main cilium repository):
 
-       This step requires valid AWS credentials to be available via the
-       environment variables ``AWS_ACCESS_KEY_ID`` and
-       ``AWS_SECRET_ACCESS_KEY``. Ping in the ``#development`` channel on Slack
-       if you have no access. It also requires the aws-cli tools to be installed.
+   In the ``cilium/cilium`` repository:
 
-#. Build the container images and push them
+   #. ``git checkout vx.y -b vx.z-dev``
+   #. Change the ``VERSION`` file to ``x.y-dev``
+   #. Run ``make -C install/kubernetes``
+
+   In the ``cilium/charts`` repository:
 
    ::
 
-      DOCKER_IMAGE_TAG=v1.0.3 make docker-image
-      docker push cilium/cilium:v1.0.3
+      ./prepare_artifacts.sh /path/to/cilium/repository/checked/out/to/release/commit
+      git push
 
-   .. note:
+   After pushing you can revert all the changes made in the local branch
+   ``x.y-dev`` from ``cilium/cilium``.
 
-      This step requires you to login with ``docker login`` first and it will
-      require your Docker hub ID to have access to the ``Cilium`` organization.
-      You can alternatively trigger a build on DockerHub directly if you have
-      credentials to do so.
 
-#. Push the git release tag
+#. Announce the release in the ``#general`` channel on Slack. Sample text:
 
    ::
 
-       git push --tags
+      :cilium-new: **Announcement:** Cilium vX.Y.Z has been released :tada:
+
+      <If security release or major bugfix, short summary of fix here>
 
-#. `Create a GitHub release <https://github.com/cilium/cilium/releases/new>`_:
+      For more details, see the release notes:
+      https://github.com/cilium/cilium/releases/tag/vX.Y.Z
 
-   #. Choose the correct target branch, e.g. ``v1.0``
-   #. Choose the correct target tag, e.g. ``v1.0.3``
-   #. Title: ``1.0.3``
-   #. Check the ``This is a pre-release`` box if you are releasing a release
-      candidate.
-   #. Fill in the release description with the output generated by github.com/cilium/release
+#. Create a new git branch based on the master branch to update ``README.rst``:
 
-#. Announce the release in the ``#general`` channel on Slack
+   ::
+
+      git checkout -b pr/bump-readme-vX.Y.Z origin/master
+      contrib/release/bump-readme.sh
+      # (Commit changes & submit PR)
 
 #. Bump the version of Cilium used in the Cilium upgrade tests to use the new release
 
    Please reach out on the ``#development`` channel on Slack for assistance with
    this task.
 
-#. Update the external tools and guides to point to the released Cilium version:
+#. Update the ``stable`` tags for ``cilium``, ``cilium-operator``,
+   ``cilium-operator-aws``, ``cilium-operator-azure``,
+   ``cilium-operator-generic``, ``cilium-docker-plugin`` and ``hubble-relay``
+   on DockerHub, for the latest version of Cilium. For example, if the latest
+   version is ``1.8``, then for all patch releases on the ``1.8`` line, this
+   step should be performed. Once ``1.9`` is out for example, then this is no
+   longer required for ``1.8``.
+
+   **Note**, the DockerHub UI will not allow you to modify the ``stable`` tag
+   directly. You will need to delete it, and then create a new, updated one.
+
+#. Update the following external tools and guides to point to the released
+   Cilium version. This step is only required on a new minor release like going
+   from ``1.8`` to ``1.9``.
 
-    * `kubeadm <https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/>`_
+    * `kubeadm <https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/>`_
     * `kops <https://github.com/kubernetes/kops/>`_
     * `kubespray <https://github.com/kubernetes-sigs/kubespray/>`_
 

contrib/release/prep-changelog.sh

@@ -0,0 +1,62 @@
+#!/bin/bash
+# SPDX-License-Identifier: Apache-2.0
+# Copyright 2020 Authors of Cilium
+
+DIR=$(dirname $(readlink -ne $BASH_SOURCE))
+source $DIR/lib/common.sh
+source $DIR/../backporting/common.sh
+
+RELEASE_TOOL_PATH="${RELEASE_TOOL_PATH:-$GOPATH/src/github.com/cilium/release}"
+RELNOTES="$RELEASE_TOOL_PATH/release"
+RELNOTESCACHE="release-state.json"
+
+usage() {
+    logecho "usage: $0 <OLD-VERSION> <NEW-VERSION>"
+    logecho "OLD-VERSION    Previous release version for comparison"
+    logecho "NEW-VERSION    Target release version"
+    logecho
+    logecho "--help     Print this help message"
+}
+
+handle_args() {
+    if ! common::argc_validate 2; then
+        usage 2>&1
+        common::exit 1
+    fi
+
+    if [[ "$1" = "--help" ]] || [[ "$1" = "-h" ]]; then
+        usage
+        common::exit 0
+    fi
+
+    if ! echo "$1" | grep -q "[0-9]\+\.[0-9]\+\.[0-9]\+"; then
+        usage 2>&1
+        common::exit 1 "Invalid OLD-VERSION ARG \"$1\"; Expected X.Y.Z"
+    fi
+
+    if ! echo "$2" | grep -q "[0-9]\+\.[0-9]\+\.[0-9]\+[-rc0-9]*"; then
+        usage 2>&1
+        common::exit 1 "Invalid NEW-VERSION ARG \"$1\"; Expected X.Y.Z[-rcW]"
+    fi
+}
+
+main() {
+    handle_args "$@"
+
+    local old_version="$(echo $1 | sed 's/^v//')"
+    local ersion="$(echo $2 | sed 's/^v//')"
+    local version="v$ersion"
+
+    logecho "Generating CHANGELOG.md"
+    rm -f $RELNOTESCACHE
+    echo -e "# Changelog\n\n## $version" > $version-changes.txt
+    $RELNOTES --base $old_version --head $(git rev-parse HEAD) >> $version-changes.txt
+    cp $version-changes.txt CHANGELOG-new.md
+    if [[ -e CHANGELOG.md ]]; then
+        tail -n+2 CHANGELOG.md >> CHANGELOG-new.md
+    fi
+    mv CHANGELOG-new.md CHANGELOG.md
+    logecho "Generated CHANGELOG.md"
+}
+
+main "$@"