-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add helper image for generating static deploy yamls for the operator. #254
Closed
Closed
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
## Preparing static yaml files for deployment on OpenShift and Kubernetes | ||
|
||
The Dockerfile in this directory can be used to automatically build static yaml files that can be applied to a cluster to deploy the DevWorkspace Operator without the need for kustomize. | ||
|
||
### Building the image | ||
```bash | ||
docker build -t <template-image> -f yaml-builder.Dockerfile . | ||
``` | ||
The docker build is self-contained; it clones the repo and builds directly from that codebase. Supported args are the same as those used in the Makefile, i.e. | ||
|
||
| Arg | Purpose | Default | | ||
| --- | ------- | ------- | | ||
| DEVWORKSPACE_BRANCH | Branch of this repo to clone & check out | master | ||
| NAMESPACE | Namespace used for deployment | devworkspace-controller | ||
| IMG | DevWorkspace Operator image to be deployed | quay.io/devfile/devworkspace-controller:next | ||
| PULL_POLICY | Sidecar PullPolicy used for workspaces | Always | ||
| DEFAULT_ROUTING | Default routingClass to use for DevWorkspaces that do not define one | basic | ||
| DEVWORKSPACE_API_VERSION | Commit hash of devfile/api dependency for DevWorkspace and DevWorkspaceTemplate CRDs | aeda60d4361911da85103f224644bfa792498499 | ||
|
||
### Using the image | ||
The container image above creates a tarball of the relevant deployment files, and can be extracted from the container: | ||
```bash | ||
docker create --name builder <template-image> | ||
docker cp builder:/devworkspace_operator_templates.tar.gz ./devworkspace_operator_templates.tar.gz | ||
docker rm builder | ||
tar -xzf devworkspace_operator_templates.tar.gz | ||
``` | ||
|
||
This will extract the files to the `deploy` directory, with subdirectories for the OpenShift and Kubernetes deployments of the operator (on OpenShift, the service-ca operator is used to provide certificates where necessary; on Kubernetes the deployment depends on the cert-manager operator and includes a Certificate object). | ||
|
||
Within each platform-dependent directory, the (large) file `combined.yaml` is a single file that can be applied to deploy the operator, and `objects/` contains each object in `combined.yaml` named according to `<resource-name>.<k8s-kind>.yaml` | ||
|
||
As the yaml generation happens statically, the configmap leaves the `devworkspace.routing.cluster_host_suffix` property unset; on Kubernetes a value must be provided here to correctly generate ingresses. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
#!/bin/bash | ||
|
||
set -e | ||
|
||
if [ -z "$DEVWORKSPACE_BRANCH" ]; then | ||
echo "Environment variable DEVWORKSPACE_BRANCH must be set" | ||
exit 1 | ||
fi | ||
|
||
BASE_DIR=${1:-/build} | ||
OUTPUT_DIR="${BASE_DIR}/deploy" | ||
TARBALL_PATH="${BASE_DIR}/devworkspace_operator_templates.tar.gz" | ||
|
||
# Clone repo | ||
git clone --depth=1 --branch "${DEVWORKSPACE_BRANCH}" https://github.com/devfile/devworkspace-operator.git | ||
cd devworkspace-operator | ||
|
||
# Grab devfile/api CRDs | ||
./update_devworkspace_crds.sh --init --api-version "$DEVWORKSPACE_API_VERSION" | ||
|
||
# Fill env in template files | ||
mv config/cert-manager/kustomization.yaml config/cert-manager/kustomization.yaml.bak | ||
mv config/service-ca/kustomization.yaml config/service-ca/kustomization.yaml.bak | ||
mv config/base/config.properties config/base/config.properties.bak | ||
mv config/base/manager_image_patch.yaml config/base/manager_image_patch.yaml.bak | ||
envsubst < config/cert-manager/kustomization.yaml.bak > config/cert-manager/kustomization.yaml | ||
envsubst < config/service-ca/kustomization.yaml.bak > config/service-ca/kustomization.yaml | ||
envsubst < config/base/config.properties.bak > config/base/config.properties | ||
envsubst < config/base/manager_image_patch.yaml.bak > config/base/manager_image_patch.yaml | ||
|
||
# Generate yaml files | ||
mkdir -p "${OUTPUT_DIR}"/{kubernetes,openshift} | ||
echo "===== Building yaml templates for Kubernetes =====" | ||
kustomize build config/cert-manager > "${OUTPUT_DIR}/kubernetes/combined.yaml" | ||
echo "===== Generated yaml templates for Kubernetes =====" | ||
echo "===== Building yaml templates for OpenShift =====" | ||
kustomize build config/service-ca > "${OUTPUT_DIR}/openshift/combined.yaml" | ||
echo "===== Generated yaml templates for OpenShift =====" | ||
|
||
# Take giant yaml file output by kustomize and separate it into a file per k8s object | ||
for dir in kubernetes openshift; do | ||
echo "===== Parsing files from ${OUTPUT_DIR}/${dir}/combined.yaml =====" | ||
pushd "${OUTPUT_DIR}/${dir}" &>/dev/null | ||
mkdir -p objects | ||
|
||
# Split combined.yaml into separate files for each record, with names temp01, temp02, etc. | ||
# Then rename each temp file according to the .metadata.name and .kind of the object | ||
csplit -s -f "temp" --suppress-matched "combined.yaml" '/^---$/' '{*}' | ||
for file in temp*; do | ||
name_kind=$(yq -r '"\(.metadata.name).\(.kind)"' "$file") | ||
mv "$file" "objects/${name_kind}.yaml" | ||
done | ||
|
||
popd &>/dev/null | ||
done | ||
|
||
# Compress files into a tarball | ||
cd "$BASE_DIR" | ||
tar -czvf "$TARBALL_PATH" deploy |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,36 @@ | ||
FROM alpine AS builder | ||
|
||
ARG DEVWORKSPACE_BRANCH=master | ||
ARG NAMESPACE=devworkspace-controller | ||
ARG IMG=quay.io/devfile/devworkspace-controller:next | ||
ARG PULL_POLICY=Always | ||
ARG DEFAULT_ROUTING=basic | ||
ARG DEVWORKSPACE_API_VERSION=aeda60d4361911da85103f224644bfa792498499 | ||
|
||
WORKDIR /build | ||
|
||
RUN apk add --no-cache \ | ||
bash \ | ||
coreutils \ | ||
curl \ | ||
gettext \ | ||
git \ | ||
jq \ | ||
python3 \ | ||
py-pip \ | ||
tar \ | ||
&& pip install yq \ | ||
&& curl -sL "https://github.com/kubernetes-sigs/kustomize/releases/download/kustomize%2Fv3.9.2/kustomize_v3.9.2_linux_amd64.tar.gz" -o kustomize.tar.gz \ | ||
&& tar -xvf kustomize.tar.gz --directory /usr/bin | ||
|
||
COPY ["prepare_templates.sh", "/"] | ||
RUN /prepare_templates.sh | ||
|
||
FROM alpine | ||
|
||
COPY --from=builder "/build/devworkspace_operator_templates.tar.gz" /devworkspace_operator_templates.tar.gz | ||
CMD echo "To extract yaml files from this repo, run:" &&\ | ||
echo " docker create --name builder <this-image>" &&\ | ||
echo " docker cp builder:/devworkspace_operator_templates.tar.gz ./devworkspace_operator_templates.tar.gz" &&\ | ||
echo " docker rm builder" &&\ | ||
echo " tar -xzf devworkspace_operator_templates.tar.gz" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That seems to duplicate stuff we have in Makefile, which means we have more to maintain and test.
According to discussion with @tolusha, it would simplify things for chectl if it can just grab static files from devworkspace-operator repo (even if then it will modify some pieces, like namespace, possible operator image, ...).
After go vendoring we're not afraid of redundant ~16k lines )
So, it make me thinking what if:
@amisevsk WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense; my main concern is the levels of indirection (pkg/api -> CRDs -> deployable yamls) but that can be addressed with PR checks I suppose. I'll close this one and port the changes over to a new one that includes the full yamls.