common automatic update#102
Merged
mbaldessari merged 63 commits intovalidatedpatterns:mainfrom Sep 6, 2023
Merged
Conversation
Contributor
mbaldessari
commented
Sep 6, 2023
mbaldessari
commented
- Add workflow to split helm charts into their own repo
- Small test for the workflow towards single chart repos
- Small test for the workflow towards single chart repos (part 2)
- Simplify split workflow
- Small test for the workflow towards single chart repos (part 3)
- Tiny change to trigger split workflow
- Add initial helm releasing workflow for acm chart
- Add helm repo updating workflow in the per-chart workflows folder
- Fix up CI superlinter on github actions
- Fix tests and make .disabled explicit
- Make sure we run the split workflow only when the changes land in validatedpatterns/common
- Update tests. We get an extra (non-impacting whitespace) with the new code
- re-add logic for extravaluefiles
- Add more tests for variable definedness/truth
- Switch helm to v3.12.3 in CI
- Unroll global.extraValueFiles in application-policies directly due to namespacing in the _helper.tpl
- Re-add code to operator-install to understand global.extraValueFiles
- Make sure to add dollar sign
- Add initial multi-source support
- Add changelog entry and add an explicit property entry to the schema
- Correct ifs and ranges in pattern, add comments
- Also quote name and value values
- Drop vault.ui.serviceType: "LoadBalancer"
- Release clustergroup chart version 0.0.2
- Update crd in common
- Add support for passing EXTRA_HELM_OPTS
- Disable kubeconform for the time being
- Add support for deploying multi source via CLI
- Upgrade ESO to v0.9.4
- Release 0.0.2 golang-external-secrets
- Simplify the passing of KUBECONFIG
- Update CRD for the operator
- Expose UUID
- Move to newly released checkout action version
- Update URLs to new github org
- Add ~/.config/validated-patterns in the secret search path
- Add support for ~/.config/validated-patterns/pattern-uuid
- Simplify the code around UUID variable definition
- Fix up common/ tests
- Move to new checkout action version
This will push any change done to a chart folder out into the separate repo corresponding to the chart that has been changed. This workflow needs a secret called CHARTS_REPOS_TOKEN that is a Personal Access Token with fine grained repo and workflow write access on the following repos: - acm-chart - hashicorp-vault-chart - golang-external-secrets-chart - clustergroup-chart - letsencrypt-chart
Add workflow to split helm charts into their own repo
Test workflow
Fix up CI superlinter on github actions
…idatedpatterns/common Otherwise a push to a private for to main would still invoke the split workflow, if a member of the vp org would do so. Since we do not want that let's make sure we limit this workflow to when the repository name is 'validatedpatterns/common'
Make sure we run the split workflow only when the changes land in validatedpatterns/common
Switch helm to v3.12.3 in CI
… namespacing in the _helper.tpl
Add extra value files and nulling logic to allow "disabling" applications
Re-add code to operator-install to understand global.extraValueFiles
This change adds initial multiSource support to patterns' applications.
The way this works is that nothing changes for applications defined in
values-*.yaml by default. So all patterns will work as usual.
What can change with this patch is that applications can slowly migrate
towards using multi source by changing an app definition from:
acm:
name: acm
path: common/acm
To:
acm:
name: acm
chart: acm
chartVersion: 0.0.*
So any time we have a `chart` field with a `chartVersion` and no
`repoURL` defined, the clustergroup chart will create a multisource
application with the values files taken from the patterns git repo
and the helm chart from https://charts.validatedpatterns.io/ using
the `chartVersion` defined in the application.
For example the above acm app would prodice the following:
project: hub
sources:
- ref: patternref
repoURL: https://github.com/mbaldessari/multicloud-gitops
targetRevision: multisource-test2
- chart: acm
helm:
ignoreMissingValueFiles: true
parameters:
- name: global.repoURL
value: $ARGOCD_APP_SOURCE_REPO_URL
- name: global.targetRevision
value: $ARGOCD_APP_SOURCE_TARGET_REVISION
- name: global.namespace
value: $ARGOCD_APP_NAMESPACE
- name: global.pattern
value: pattern-sample
- name: global.clusterDomain
value: mcg-hub.blueprints.rhecoeng.com
- name: global.clusterVersion
value: "4.13"
- name: global.clusterPlatform
value: AWS
- name: global.hubClusterDomain
value: apps.mcg-hub.blueprints.rhecoeng.com
- name: global.localClusterDomain
value: apps.mcg-hub.blueprints.rhecoeng.com
valueFiles:
- $patternref/values-global.yaml
- $patternref/values-hub.yaml
- $patternref/values-AWS.yaml
- $patternref/values-AWS-4.13.yaml
- $patternref/values-AWS-hub.yaml
- $patternref/values-4.13-hub.yaml
repoURL: https://charts.validatedpatterns.io/
targetRevision: 0.0.*
Note that this depends on the operator supporting multiSource
applications (version > 0.0.17).
Add initial multi-source support
Release clustergroup chart version 0.0.2
This is needed if we want to support multisource installations from the CLI
Update crd in common
Currently a user can set additional helm params via the
EXTRA_HELM_OPTS environment variable in order to tweak a value at `make
install` time.
This does not work correctly when we run things from our utility
container, that is because that variable is never passed from the host
to the container.
According to `man podman run` if we simply pass `-e EXTRA_HELM_OPTS` to
the podman invocation: "If an environment variable is spec‐
ified without a value, Podman checks the host environment for a value
and set the variable only if it is set on the host"
* Without setting EXTRA_HELM_OPTS:
unset EXTRA_HELM_OPTS; ./pattern.sh make install
make -f common/Makefile operator-deploy
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Checking prerequisites:
Check for 'git helm oc ansible': OK
Check for python-kubernetes: OK
Check for kubernetes.core collection: OK
Checking repository:
https://github.com/mbaldessari/multicloud-gitops.git - branch main: Running inside a container: Skipping git ssh checks
+ oc get crds patterns.gitops.hybrid-cloud-patterns.io
+ echo 'Running helm:'
Running helm:
+ helm upgrade --install multicloud-gitops common/operator-install/ -f values-global.yaml --set main.git.repoURL=https://github.com/mbaldessari/multicloud-gitops.git --set main.git.revision=main
* With EXTRA_HELM_OPTS set:
export EXTRA_HELM_OPTS="--set main.multiSourceConfig.enabled=true"; ./pattern.sh make install
make -f common/Makefile operator-deploy
make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops'
Checking prerequisites:
Check for 'git helm oc ansible': OK
Check for python-kubernetes: OK
Check for kubernetes.core collection: OK
Checking repository:
https://github.com/mbaldessari/multicloud-gitops.git - branch main: Running inside a container: Skipping git ssh checks
+ oc get crds patterns.gitops.hybrid-cloud-patterns.io
+ echo 'Running helm:'
Running helm:
+ helm upgrade --install multicloud-gitops common/operator-install/ -f values-global.yaml --set main.git.repoURL=https://github.com/mbaldessari/multicloud-gitops.git --set main.git.revision=main --set main.multiSourceConfig.enabled=true
(Briefly added set -x to see the exact commands during testing)
Add support for passing EXTRA_HELM_OPTS
Via: ``` export EXTRA_HELM_OPTS="--set main.multiSourceConfig.enabled=true" ./pattern.sh make install ``` one can now deploy a pattern with the experimental multisource support enabled. Tested with the above command and correctly deployed a multi-source based pattern.
Add support for deploying multi source via CLI
Upgrade ESO to v0.9.4
Release 0.0.2 golang-external-secrets
According to `man podman run` if we simply pass `-e KUBECONFIG` to the podman invocation: "If an environment variable is spec‐ ified without a value, Podman checks the host environment for a value and set the variable only if it is set on the host" So let's just do that and drop the current more complex fragile logic. Tested with: * No KUBECONFIG set unset KUBECONFIG; ./pattern.sh make install make -f common/Makefile operator-deploy make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops' Checking prerequisites: Check for 'git helm oc ansible': OK Check for python-kubernetes: OK Check for kubernetes.core collection: OK Checking repository: https://github.com/mbaldessari/multicloud-gitops.git - branch script-fix: Running inside a container: Skipping git ssh checks Running helm: Error: Kubernetes cluster unreachable: Get "https://localhos:6443/version" * With KUBECONFIG set export KUBECONFIG=~/sno1-kubeconfig ./pattern.sh make install make -f common/Makefile operator-deploy make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops' Checking prerequisites: Check for 'git helm oc ansible': OK Check for python-kubernetes: OK Check for kubernetes.core collection: OK Checking repository: https://github.com/mbaldessari/multicloud-gitops.git - branch script-fix: Running inside a container: Skipping git ssh checks Running helm: WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/michele/sno1-kubeconfig Release "multicloud-gitops" does not exist. Installing it now. NAME: multicloud-gitops LAST DEPLOYED: Mon Sep 4 07:04:16 2023 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: None make[1]: Leaving directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops' make load-secrets make[1]: Entering directory '/home/michele/Engineering/cloud-patterns/multicloud-gitops' make -f common/Makefile load-secrets ...
Small pattern.sh cleanup
Update CRD for the operator
Tested as:
$ helm template . > /tmp/a; helm template . --set 'main.analyticsUUID=foo' > /tmp/b; diff -u /tmp/a /tmp/b
--- /tmp/a 2023-09-04 15:49:51.160607725 +0200
+++ /tmp/b 2023-09-04 15:49:51.177607813 +0200
@@ -15,6 +15,7 @@
operatorSource: redhat-operators
multiSourceConfig:
enabled: false
+ analyticsUUID: foo
* Pristine environment: $ make show helm template common/operator-install/ --name-template common -f values-global.yaml --set main.git.repoURL="https://github.com/hybrid-cloud-patterns/common.git" --set main.git.revision=vp-paths --- apiVersion: gitops.hybrid-cloud-patterns.io/v1alpha1 kind: Pattern metadata: name: common namespace: openshift-operators spec: clusterGroupName: example gitSpec: targetRepo: https://github.com/hybrid-cloud-patterns/common.git targetRevision: vp-paths gitOpsSpec: operatorChannel: gitops-1.8 operatorSource: redhat-operators multiSourceConfig: enabled: false ... * Add UUID to the environment $ echo "vp-team-bandini" >> ~/.config/validated-patterns/pattern-uuid $ make show helm template common/operator-install/ --name-template common -f values-global.yaml --set main.git.repoURL="https://github.com/hybrid-cloud-patterns/common.git" --set main.git.revision=vp-paths --set main.analyticsUUID=vp-team-bandini --- apiVersion: gitops.hybrid-cloud-patterns.io/v1alpha1 kind: Pattern metadata: name: common namespace: openshift-operators spec: clusterGroupName: example gitSpec: targetRepo: https://github.com/hybrid-cloud-patterns/common.git targetRevision: vp-paths gitOpsSpec: operatorChannel: gitops-1.8 operatorSource: redhat-operators multiSourceConfig: enabled: false analyticsUUID: vp-team-bandini ...
Add support for ~/.config/validated-patterns/pattern-uuid
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.