-
Notifications
You must be signed in to change notification settings - Fork 87
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[build] Automate publication of tags to docker hub (#659)
* [build] Build and push to docker hub the dss image * [build] Change matching branch * [build] rename workflow * [build] change release tagging * [build] trigger on new tags * [build] Change login-action version * [build] fetch history and update docker_url * [build] documentation * Run the release pipeline only if it is on interuss/dss repository * Improve documentation * Add version script * Rename release.sh to tag.sh and update tag pattern * Finalize documentation and use short version for docker images * Final changes after cd test * Small additional improvements + lint * Force clean workspace as a prerequisite for releases
- Loading branch information
Showing
7 changed files
with
159 additions
and
21 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| # This workflow builds and pushes the dss docker image when a release tag is pushed. | ||
| # The following secrets must be provided: | ||
| # DOCKER_USERNAME: The username to log into the registry. | ||
| # DOCKER_PASSWORD: The access token of the user to log into the registry. | ||
| # DOCKER_URL: The base url of the repository. Example: `docker.io/interuss` or `interuss`. | ||
|
|
||
| name: Publish DSS image to Docker Registry (on new release tag) | ||
| on: | ||
| push: | ||
| tags: | ||
| # To modify to trigger the job for fork's releases | ||
| - "interuss/dss/v[0-9]+.[0-9]+.[0-9]+" | ||
| jobs: | ||
| docker-hub-push: | ||
| name: DSS Build and Push to Docker Hub | ||
| runs-on: ubuntu-latest | ||
| # To modify to enable the job for forked repository | ||
| if: github.repository == 'interuss/dss' | ||
| steps: | ||
| - name: Job information | ||
| run: | | ||
| echo "Job information" | ||
| echo "Trigger: ${{ github.event_name }}" | ||
| echo "Host: ${{ runner.os }}" | ||
| echo "Repository: ${{ github.repository }}" | ||
| echo "Branch: ${{ github.ref }}" | ||
| docker images | ||
| - name: Checkout | ||
| uses: actions/checkout@v2 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Log in to Docker Hub | ||
| uses: docker/login-action@v1 | ||
| with: | ||
| registry: docker.io | ||
| username: ${{ secrets.DOCKER_USERNAME }} | ||
| password: ${{ secrets.DOCKER_PASSWORD }} | ||
|
|
||
| - name: Build and push image | ||
| env: | ||
| DOCKER_URL: ${{ secrets.DOCKER_URL }} | ||
| run: | | ||
| build/build.sh |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,16 +1,23 @@ | ||
| # Release Management | ||
|
|
||
| Releases of the DSS are based on git tags in the format `v[0-9]+\.[0-9]+\.[0-9]+`. | ||
| When either an executable or image is built from a `git` checkout of the source, the latest tag up to the current commit satisfying the aforementioned format is used as the current version. If no such tags exists, the build system defaults to v0.0.0. | ||
| Releases of the DSS are based on git tags in the format `interuss/dss/v[0-9]+\.[0-9]+\.[0-9]+`. When either an executable or image is built from a `git` checkout of the source, the most recent tag is used as the version tag. If no such tag exists, the build system defaults to v0.0.0-[commit_hash]. If commits have been added to the tag, the commit hash is appended to the version. If the workspace is not clean, `-dirty` is appended to it. The version tag is computed by `scripts/git/version.sh`. | ||
|
|
||
| With that, releasing a DSS version and producing release artifacts requires the following steps: | ||
| * Create a tag `vX.Y.Z` on master and push it to the remote using `make release MAJOR=X MINOR=Y PATCH=Z` | ||
| * Optionally, build the main docker image, tag it with `vX.Y.Z` and push it out to an image registry of your choice. The official upstream image for a given release is made available at https://hub.docker.com/repository/docker/interuss/dss. | ||
| With that, releasing a DSS version in the canonical interuss fork requires the following steps: | ||
| - Create a release tag on master using `make tag MAJOR=X MINOR=Y PATCH=Z`. The script will push a tag (`release tag`) to the remote origin under the form of `[owner]/dss/vX.Y.Z` where `[owner]` is either the organization name or the username of the origin remote url. Official releases are `interuss/dss/v#.#.#`. | ||
| - The github workflow ([.github/workflows/dss-publish.yml](.github/workflows/dss-publish.yml)) is triggered for every new release tag. It builds and publishes the DSS image to the [official docker registry](https://hub.docker.com/repository/docker/interuss/dss). | ||
|
|
||
| To enable releases of DSS version in a fork, the following steps are required: | ||
| 1. Set the remote origin url of the repository of the target fork. (ie git@github.com:[owner]/dss.git) | ||
| 2. Edit in ([.github/workflows/dss-publish.yml](.github/workflows/dss-publish.yml)) the trigger to match the tags of the fork's owner as well as the job's entry condition to allow the forked repository. | ||
| 3. [Enable github actions in the forked project](https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/managing-github-actions-settings-for-a-repository#configuring-required-approval-for-workflows-from-public-forks). | ||
| 4. Configure the environment variables to setup the registry. (See instructions at the top of [.github/workflows/dss-publish.yml](.github/workflows/dss-publish.yml)) | ||
|
|
||
| Optionally, you can manually build the DSS docker image using [build/build.sh](build/build.sh), tag accordingly the image `interuss-local/dss` and push it out to an image registry of your choice. | ||
|
|
||
| # CockroachDB Version | ||
|
|
||
| When possible we try to use the latest version major and minor of CockroachDB (v 20.2.x). CRDB 20.2.0 comes with upgrades to the underlying storage of CRDB itself using a branched version of RocksDB called Pebble. | ||
|
|
||
| ## Backwards Compatibility | ||
|
|
||
| 20.2.x is backwards compatible with the files written by 20.1.x and upgrades are simple. However you CANNOT downgrade back to 20.1.x as the new version updates the metadata and prevents the older version of the CRDB to be started on same files. Although 20.2.x is compatible with 20.1.x in the same cluster; it was recommended that you quickly upgrade all the remianing nodes to 20.2.x and reduce the version drift as much as possible. The possible negative effect of running a mixed cluster is the difference in features available between the nodes, you must be sure that you are not leveraging the new or deprecated features as this could negatively affect your queries. | ||
| 20.2.x is backwards compatible with the files written by 20.1.x and upgrades are simple. However you CANNOT downgrade back to 20.1.x as the new version updates the metadata and prevents the older version of the CRDB to be started on same files. Although 20.2.x is compatible with 20.1.x in the same cluster; it was recommended that you quickly upgrade all the remianing nodes to 20.2.x and reduce the version drift as much as possible. The possible negative effect of running a mixed cluster is the difference in features available between the nodes, you must be sure that you are not leveraging the new or deprecated features as this could negatively affect your queries. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # This script prints the organization of the upstream repository using the remote origin url. | ||
| # It expects a github.com remote defined as origin and the following url formats: | ||
| # 1. git@github.com:interuss/dss.git | ||
| # 2. git@github.com/interuss/dss.git | ||
| # 3. https://github.com/interuss/dss.git | ||
|
|
||
| UPSTREAM_REPO=$(git remote get-url origin) | ||
| # Replace `:` by `/` to handle git@github.com:interuss/dss.git remote reference. | ||
| UPSTREAM_REPO=${UPSTREAM_REPO//:/\/} | ||
| # Remove hostname part | ||
| UPSTREAM_OWNER=$(dirname "${UPSTREAM_REPO#*github.com/*}") | ||
|
|
||
| echo "$UPSTREAM_OWNER" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| # This script prints the current version of a component in the repository based on the tags | ||
| # of the upstream repository (remote origin) matching the following convention: | ||
| # owner/component/version. Examples of values: | ||
| # - owner: interuss (automatically extracted from the remote origin url) | ||
| # - component: rid, scd, aux, uss_qualifier | ||
| # - version: v3.0.1[-hash][-dirty] | ||
| # - [-hash] (example: -8a493ef8 ) is added when commits have been added to the latest version tagged. | ||
| # - [-dirty] (example: -dirty) when the workspace is not clean. | ||
| # Only versions without [-hash] and without [-dirty] shall be released. | ||
|
|
||
| if [[ $# == 0 ]]; then | ||
| echo "Usage: $0 <COMPONENT> [--long]" | ||
| echo "Print the component's version number. (ie v0.0.1)" | ||
| echo "[--long]: Print the component's version using the long format including the upstream owner (ie interuss/scd/v0.0.1)." | ||
| exit 1 | ||
| fi | ||
|
|
||
| COMPONENT=${1} | ||
|
|
||
| RELEASE_FORMAT=false | ||
| if [[ $2 == "--long" ]]; then | ||
| RELEASE_FORMAT=true | ||
| fi | ||
|
|
||
| # Set working directory | ||
| cd "$(dirname "$0")" || exit 1 | ||
|
|
||
| UPSTREAM_ORG=$(./upstream_owner.sh) | ||
|
|
||
| # Look for the last tag of the component | ||
| LAST_VERSION_TAG=$(git describe --abbrev=1 --tags --match="${UPSTREAM_ORG}/${COMPONENT}/*" 2> /dev/null) | ||
| #echo "LAST_VERSION_TAG: $LAST_VERSION_TAG" | ||
|
|
||
| # Store in LAST_VERSION the version of the tag (ie v0.0.1) | ||
| LAST_VERSION=${LAST_VERSION_TAG##*/} | ||
| #echo "LAST_VERSION: $LAST_VERSION" | ||
|
|
||
| # If no version was found, use default v0.0.0. | ||
| if [[ -z "$LAST_VERSION" ]]; then | ||
| LAST_VERSION="v0.0.0" | ||
| fi | ||
|
|
||
| # Current commit | ||
| COMMIT=$(git rev-parse --short HEAD) | ||
|
|
||
| # Check if there are some commits on top of the tag by checking if an abbrev part is present. | ||
| if [[ "$LAST_VERSION" == *"-"* ]]; then | ||
| # Remove abbrev part | ||
| LAST_VERSION=${LAST_VERSION%%-*} | ||
| # Append the commit hash | ||
| LAST_VERSION=${LAST_VERSION}-${COMMIT} | ||
| fi | ||
|
|
||
| # Set the dirty flag if the workspace is not clean. | ||
| DIRTY="" | ||
| if test -n "$(git status -s)"; then | ||
| DIRTY="-dirty" | ||
| fi | ||
|
|
||
| if [[ "$RELEASE_FORMAT" == "true" ]]; then | ||
| echo "${UPSTREAM_ORG}"/"${COMPONENT}"/"${LAST_VERSION}""${DIRTY}" | ||
| else | ||
| echo "${LAST_VERSION}""${DIRTY}" | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters