Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kpt + Gcloud Image Definition #1553

Merged
merged 12 commits into from
Mar 18, 2021
72 changes: 70 additions & 2 deletions release/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ To cut a new kpt release perform the following:
- Within kustomize: [kyaml](https://github.com/kubernetes-sigs/kustomize/blob/master/releasing)
- Within kustomize: [cmd/config](https://github.com/kubernetes-sigs/kustomize/blob/master/releasing)
- Update `go.mod` file with correct versions of `cli-utils`, `kyaml`, and `cmd/config`
- Update the pinned base image versions in the kpt Dockerfiles
([Dockerfile](./Dockerfile) and [release/gcloud/Dockerfile](./release/gcloud/Dockerfile))
- Run `make all` (which should update `go.sum` and run `go mod tidy`)
- Create a `kpt` PR with previous `go.mod` and `go.sum` changes, and submit. [Example PR](https://github.com/GoogleContainerTools/kpt/pull/594)
- Fetch the latest master changes to a clean branch
Expand All @@ -35,13 +37,79 @@ also updated with the `latest` tag for tagged releases.
- `gs://kpt-dev/latest`
- `gs://kpt-dev/releases`

# Dry-Run Goreleaser
# Testing the Release Process

## Running Cloud Build Locally

You can use [`cloud-build-local`](https://github.com/GoogleCloudPlatform/cloud-build-local)
to run kpt's Cloud Build builds locally with custom parameters (`--substitutions`)
and dry-runs (`--dryrun`) to validate the builds syntax.

You will need to provide `--substitutions` for `TAG_NAME`, `_VERSION`,
`_GCS_BUCKET` and `_GITHUB_USER`. In a `--dryrun` these do not need to align
with existing resources. For example:

```sh
cloud-build-local --config=release/tag/cloudbuild.yaml --substitutions=TAG_NAME=test,_VERSION=test,_GCS_BUCKET=test,_GITHUB_USER=test --dryrun=true .
```

When running with `--dryrun=false` you will need to do some initial work to run
against your forked version of kpt:

> ⚠️ If your change depends on code changes (any file other than `cloudbuild.yaml`)
you will need to check in and tag those changes when following these steps.
Otherwise the first steps that checkout the code will not include your local
changes and all future steps will not include those changes. This includes any
modifications to `goreleaser.yaml`!

1. Setup your gcloud profile.
2. Create and enable the Google Cloud Secret Manager on the profile
3. Create a GitHub Personal Access Token and save it as `github-token` in the
Google Cloud Secrets Manager.

```sh
➜ gcloud secrets create github-token
```

4. Assign `--substitutions=_GITHUB_USER={your github username}`. Do **not** use
`GoogleContainerTools` (this would attempt to create a release for
`GoogleContainerTools/kpt`).
5. Create a tag on your fork. Assign that name to the `TAG_NAME`
substitution (multiple substitutions are comma delimated: FOO=a,BAR=b).

```sh
➜ git tag my-test-release
➜ git push origin my-test-release
```

6. Create a Cloud Storage bucket and assign its name to `_GCS_BUCKET` in a
substitution.
7. Assign a `_VERSION` substition. This will impact the results of
`kpt version`.
8. Run `cloud-build-local` with the updated substitutions.

> `cloud-build-local` will use `--dryrun=true` by default. This will allow you
to validate your substitutions and syntax. To perform an actual run (including
releasing and publishing images to your gcr/storage/github) set
`--dryrun=false`.

9. Grab a coffee ☕ or your favorite tea 🍵 this process is not quick.
10. Once this is done the tag you chose in step 5 should now be a release and
the kpt containers should be in your container registry.

## Dry-Run Goreleaser

To test local changes to the [`goreleaser.yaml`](./tag/goreleaser.yaml) config. You may
[install goreleaser](https://goreleaser.com/install/) locally and provide the
`--skip-verify --skip-publish` flags.

From the kpt directory you would run:
Set the GitHub User ENV to your username:

```sh
export _GITHUB_USER=YourUsername
```

From the kpt directory you can run the following to validate your release:

```sh
goreleaser release --skip-validate --skip-publish -f release/tag/goreleaser.yaml
Expand Down
34 changes: 34 additions & 0 deletions release/gcloud/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Copyright 2019 Google LLC
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, the existing top-level Dockerfile should be moved to release directory as well. This can be a separate PR

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack. Will open that PR after this is merged to avoid conflicts with the merge/squash of this change.

#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

FROM golang:1.13-stretch AS build
ENV CGO_ENABLED=0
ENV GO111MODULE=on

ARG KPT_VERSION=unknown

WORKDIR /go/src/
COPY go.mod .
COPY go.sum .
RUN go mod download
COPY . .
RUN go build -v -o /usr/local/bin/kpt -ldflags="-s -w -X github.com/GoogleContainerTools/kpt/run.version=$KPT_VERSION" ./

FROM gcr.io/google.com/cloudsdktool/cloud-sdk:332.0.0-alpine
RUN apk update && apk upgrade && \
apk add --no-cache git less diffutils bash openssh docker-cli && \
rm -rf /var/lib/apt/lists/* && \
rm /var/cache/apk/*
COPY --from=build /usr/local/bin/kpt /usr/local/bin/kpt
ENTRYPOINT ["kpt"]
20 changes: 18 additions & 2 deletions release/tag/cloudbuild.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,8 @@ steps:
dir: 'artifacts'
args: [ 'cp', '-a', 'public-read', 'kpt_windows_amd64.exe', 'gs://${_GCS_BUCKET}/latest/windows_amd64/kpt.exe' ]

# build docker image
# build docker images
## kpt image
- name: 'gcr.io/cloud-builders/docker'
dir: 'kpt'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/kpt:${TAG_NAME}', '.', '--build-arg', 'KPT_VERSION=${_VERSION}']
Expand All @@ -147,12 +148,27 @@ steps:
dir: 'kpt'
args: [ 'tag', 'gcr.io/$PROJECT_ID/kpt:${TAG_NAME}', 'gcr.io/$PROJECT_ID/kpt:latest' ]

# push the container image
## kpt + gcloud image
- name: 'gcr.io/cloud-builders/docker'
dir: 'kpt'
args: [ 'build', '-t', 'gcr.io/$PROJECT_ID/kpt-gcloud:${TAG_NAME}', '-f', 'release/gcloud/Dockerfile', '.', '--build-arg', 'KPT_VERSION=${_VERSION}']

- name: 'gcr.io/cloud-builders/docker'
dir: 'kpt'
args: [ 'tag', 'gcr.io/$PROJECT_ID/kpt-gcloud:${TAG_NAME}', 'gcr.io/$PROJECT_ID/kpt-gcloud:latest' ]

# push the container images
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/kpt:${TAG_NAME}']

- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/kpt-gcloud:${TAG_NAME}']

# update latest tag on release
- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/kpt:latest']

- name: 'gcr.io/cloud-builders/docker'
args: ['push', 'gcr.io/$PROJECT_ID/kpt-gcloud:latest']

timeout: '60m'
2 changes: 1 addition & 1 deletion release/tag/goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,5 +49,5 @@ changelog:
- Merge branch
release:
github:
owner: GoogleContainerTools
owner: "{{ .Env._GITHUB_USER }}"
name: kpt