From 262d3aec3af90b03527d5d4c3d45d9fc17816180 Mon Sep 17 00:00:00 2001 From: jonny <65790298+day0hero@users.noreply.github.com> Date: Tue, 1 Mar 2022 11:19:02 -0600 Subject: [PATCH] example template update (#1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Provide more subscription defaults * Review and simplify argo sync options * Ignore objects derived from our ACM policies * See if we can set SkipDryRunOnMissingResource att the application level * Revert "See if we can set SkipDryRunOnMissingResource att the application level" It was not possible This reverts commit 2ca42ed9d8c5f9673a8fe9617bd9ac1bf394bdd2. * Add more subscription defaults * Support to kustomize targets * Support applications using plugins * Cannot assume the subscription name and csv prefix match * Allow custom ignoreDifferences values * Helper for deploying the repo accurately * Include the site name in the initial app * Simple site to test argo plugins * Pass site value files when using the helm-with-kustomize plugin * Use the app name when rendering the helm template * Add missing file * kustomize debugging * The helm-with-kustomize example is useful to keep around * Additional help targets * Support loading just the site application on install * Make it easy to drive the bootstrap option from the helper * Make it easy to kustomize the secrets location from the helper * Limit helm's template name to under 53 chars * Create the application name ordered by significance * Site name is included in the site's .Release.Name * Try to stay under helm's name template max of 53 chars * Fix the application role binding to refer to the actual policy * Use the managed site name for the application and values file * Enforce quoting of helmOverrides for acm policy applications * Provide a default path to the site chart * Drop references to manuela in common * Allow namespaces without operatorgroups for charts that already include them * Add argocd secret handling * Add argosecret target * Remove unneeded util dir * Explain how to use common makefile * Allow namespaces without operatorgroups for charts that already include them * Add template toplevel makefile * Allow templates to know which namespace they're being deployed into * Refresh the kustomize-renderer example * Refresh the sample datacenter from manuela * Don't assume the install symlink is present * Attempt new template format * Ensure password has length before exiting wait loop * Replace makefile template and make embedded shell wait for password to have length as well as 0 exit * Add then * Make script explain what it's doing * make output clearer * Revert "Attempt new template format" This reverts commit c463cbc4ae387f14889c2b5a2f24a3f0134dfb05. * Try putting applications into a dedicated argo instance * All namespaces are created by the site chart * Fix default applicaiton namespace handling * Make the pattern name availble to argo applications * Give preference to whatever was specified in the secrets file * Strip off any auth tokens when calculating the target repo * Ensure there are no spaces in the namespace * Fix namespace for applications * Fix namespace for application acm policies * Include the pattern name to ensure uniqueness * Update repo name * Try putting applications into a dedicated argo instance * All namespaces are created by the site chart * Fix default applicaiton namespace handling * Make the pattern name availble to argo applications * Give preference to whatever was specified in the secrets file * Strip off any auth tokens when calculating the target repo * Ensure there are no spaces in the namespace * Fix namespace for applications * Fix namespace for application acm policies * Include the pattern name to ensure uniqueness * Fix the destination namespace for datacenter manifests * Try a simpler argo name * Use a shorter namespace * acm: Fix the target namespace for the site application policy * Fix application namespace * The application name is already unique to the namespace * Restore gitops to the name of the argocd CR * Match the service accounts to the argocd name * Document what argocd values need to be kept in sync * Updated note regarding argo name * Change the default 'name' label for namespaces * Update common Makefile to have more parameters for secret making * Re-factor install to not require .ONESHELL features as Mac doesn't support them out of the box * Update Makefile doc and SECRET_NAME parameter * Don't hardcode SECRET_NAME here * Move script to scripts dir and call it from there * New s3 secrets file for central-s3 support * Took care of merge conflicts with s3-secret.yaml * Adding functionalist to have a list of namespaces for a particular subscription * Enhance compatibility with older versions of git * Trim the example datacenter site * Support real helm charts too * Adds the if control to force booleans to be string type for argo on helm/vault overrides * Improved secrets handling in pipelines (#10) * Update common with new secret management * Add values-global to Makefile Co-authored-by: Wolfgang Kulhanek * Add note regarding tekton annotation * Add note regarding tekon annotation * Ensure updated secret template variables are defined * Missing template variable * Update values.yaml * Avoid assumptions about the site being called datacenter leaking into patterns * Add missing template variables * Standardize on Values.secrets for usernames as well as passwords * Sync the example global values file with the i-e pattern * Sync the plugin example application with the i-e pattern * Ensure helm options stay in sync and add a simple test * Make the test more stable and add missing values * Extend the unit tests to the acm and site charts * Ensure the global.repoURL variable is set consistently in tests * Add some elements to .gitignore * Fix whitespace in repoURL value in a POSIX-friendly way * Remove manuela-specific elements and secrets * Modify tests to match removal of secrets and namespace * Remove cruft from makefile * Loosen regex to extract target repo * Add structure for vault * Remove vault subdir to prep for alternate inclusion * Squashed 'vault/' content from commit 9fa25e9 git-subtree-dir: vault git-subtree-split: 9fa25e97c806073c7dd3274a851181cbb3d67868 * Change site to clustername to allow for multiple clusters in a config group * Remove staging and adjust tests to reflect that * Update examples for recent cleanups * Support ocp authentication for namespaced argos * Update examples for recent cleanups * Support ocp authentication for namespaced argos * Make sure that argo can manage cluster-wide resources from the relevant namespaces Otherwise we can error out with: Cluster level ClusterRoleBinding vault-server-binding cannot be managed when in namespaced mode By setting ARGOCD_CLUSTER_CONFIG_NAMESPACES to the namespaces where argocd instance is installed we allow it to create cluster wide resources for all namespaces. Tested this and now I am correctly able to invoke the vault helm chart from argo without the above error. References: - https://github.com/argoproj/argo-cd/issues/5886 - https://github.com/argoproj-labs/argocd-operator/issues/385 * Add some vault utilities and add a gitignore entry * Add Makefile target to init and unseal vault * Add an unseal target and provide it a default in the script * Initial import of chart from 9348383 of https://github.com/external-secrets/external-secrets * Remove vault and external secrets - we can install from helm directly * Protect ourselves from calling vault-init twice Without this we risk to easily overwrite the seal + token file and hence lose future access to the vault. * Add script for copying vault-token * Add Hub cluster domain as a global variable based on ingress config * Add code to extract root token * Add function to wrap an exec including the token * Add pki init to vault init makefile target * Expand the PKI domain (knock off the first two domains for the hub cluster, e.g. apps and the next one up to allow the PKI to be bigger * Correct pki role and domain creation * Add more functions for secrets management * pki init is done in vault_init, no need to have a separate makefile task * Fix the name of the function to initialize the kubernetes backend Otherwise we'll error out with: common/scripts/vault-utils.sh: line 85: vault_k8s_init: command not found * Add --interactive to the oc vault exec calls This allows us to read stdin and push a file via stdin. This is particularly useful when configuring the vault policy * Add a policy init function to setup initial policy for the vault * Add variable qualification to prevent helm template errors * Add vault-route to workaround hard-coding of passthrough mode in vault helm chart 0.18.0 * Correct route resource, remove namespace and spell variable correctly * Fix TTL lease typo in vault-init Current wrong command: bash-4.4$ vault secrets tune -max-lease=8760h pki flag provided but not defined: -max-lease Add -ttl at the end to fix it: bash-4.4$ vault secrets tune --max-lease-ttl=8760h pki Success! Tuned the secrets engine at: pki/ * Remove extra duplicate subcription YAML and force quoting in application install for consistency * Add local domain to ACM policy * Propogate localdomain setting to non-privileged argo * Fix some tests * Fix remaining tests * Remove manuela tag from clustergroup chart * Add extra framework options to level with clustergroup implementation * Remove vault-route application Now that vault-helm v0.19.0 has been released we can use it directly to create the vault route with tls.termination set to 'edge', hence this is not needed anymore. * Remove vault-route application Now that vault-helm v0.19.0 has been released we can use it directly to create the vault route with tls.termination set to 'edge', hence this is not needed anymore. * Supply hubClusterDomain for localHubCluster where we don't have ACM to populate the lookup * Don't conditionalize lookups when we know we need them * Remove bashism in vault_exec We currently uase "bash -c" inside vault_exec. This only works when using UBI-based images. Let's move to 'sh -c' to be a bit more robust in case we're in the presence of the upstream vault images which do not have bash installed. Tested a 'vault-init' on UBI images and it correctly worked with no errors whatsoever in the log. * Add namespace support to the regional gitops installations This allows argo on regional clusters to have more rights. Specifically this is needed to create the clusterbindingrole needed for the k8s-external-secret operator. As a first pass we'll use the '*' namespace. In a second iteration we'll need to look at restricting that to openshift-gitops and the namespace of the regional gitops instance. At this point of time such a change is too invasive and is at risk of breaking existing patterns. Tested this on multicloud gitops and I correctly get argo to create the clusterrolebinding in the k8s-external-secret: $ oc get clusterrolebinding | grep k8s-extern k8s-external-secrets-kubernetes-external-secrets ClusterRole/k8s-external-secrets-kubernetes-external-secrets k8s-external-secrets-kubernetes-external-secrets-auth ClusterRole/system:auth-delegator Previously this would fail with: Cluster level CustomResourceDefinition "externalsecrets.kubernetes-client.io" can not be managed when in namespaced mode * Add code to validate origin for install/upgrade * Add better domain alternation logic and Makefile validation * Stop using echo when returning a string in a function This is not portable [1] and in some shells and also on bash depending on the version [2] it may or may not automatically interpret switches (like -n). Let's switch to printf which is the POSIX-blessed way of doing things [3]. Tested this on my environment and was able to still do a vault-init without any errors. [1] https://wiki.bash-hackers.org/scripting/nonportable#echo_command [2] https://stackoverflow.com/questions/11193466/echo-n-prints-n [3] https://wiki.bash-hackers.org/commands/builtin/printf * Add support for pushing the kube-root-ca.crt from the HUB to the managed clusters By default this ACM templates is inactive and will only be activated if asked explicitely via the .pushHubCA parameter. It will pull the ca.crt field from the the kube-root-ca.crt ConfigMap on the hub into a secret on the managed cluster. This will then be used by the external-secrets pod so it can trust the https://vault-vault.apps.hub-domain... API endpoint of the vault. Tested with this change and once enabled via .pushHubCA the kubernetes-external-secrets pod could correctly connect to the vault running on the HUB (without this we'd get self-signed certs errors) * Fix the TARGET_REPO calculation * Fix common/ make test Currently we fail in a bunch of tests. Let's fix these up so we'll be able to introduce some unit testing CI soon. Tested with: $ make test |grep FAIL Testing install chart (naked) Testing clustergroup chart (naked) Testing acm chart (naked) Testing install chart (normal) Testing clustergroup chart (normal) Testing acm chart (normal) * Remove clusterselector for cases where we want the vault-ca installed on the hub cluster as well * Fix common/ make test Currently we fail in a bunch of tests. Let's fix these up so we'll be able to introduce some unit testing CI soon. Tested with: $ make test |grep FAIL Testing install chart (naked) Testing clustergroup chart (naked) Testing acm chart (naked) Testing install chart (normal) Testing clustergroup chart (normal) Testing acm chart (normal) * Replicate https://github.com/hybrid-cloud-patterns/multicloud-gitops/pull/36 * Remove policy to deploy vault CA as unnecessary * Changes to vault-utils to support vault/external-secrets combo * Rename the namespace and serviceaccounts to the name of the new golang-based external secrets operator We're moving to the newer golang-based external secrets operator at https://external-secrets.io/ To be more explicit about our intention we name namespaces and serviceaccounts golang-external-secrets. Let's rename it in the hub config as well. Tested with the other related golang changes and everything worked as expected. * Add golang-external-secrets chart * Add script to push vault secrets * Fix test error in clustergroup example This fixed the following: Testing clustergroup chart (naked) --- tests/clustergroup-naked.expected.yaml 2022-02-27 18:06:11.474410537 +0100 +++ tests/.clustergroup-naked.expected.yaml 2022-02-27 19:29:17.534554450 +0100 @@ -61,6 +61,7 @@ # Changing the name affects the ClusterRoleBinding, the generated secret, # route URL, and argocd.argoproj.io/managed-by annotations name: example-gitops + namespace: common-example annotations: argocd.argoproj.io/compare-options: IgnoreExtraneous spec: FAIL on clustergroup naked with opts [] make: *** [Makefile:34: test] Error 1 * Fix acm naked example test Currently errors out with: Testing acm chart (naked) --- tests/acm-naked.expected.yaml 2022-02-27 18:06:11.474410537 +0100 +++ tests/.acm-naked.expected.yaml 2022-02-27 19:38:40.898341311 +0100 @@ -2,13 +2,6 @@ # Source: acm/templates/policies/application-policies.yaml # TODO: Also create a GitOpsCluster.apps.open-cluster-management.io --- -# Source: acm/templates/policies/hub-certificate-authority.yaml -# We only push the hub CA to the regional clusters when the user explicitely tells us so -# This template fetches "ca.crt" from the "kube-root-ca.crt" configMap from the hub -# (this configmap is present in all namespaces) and puts it in the vault-ca secret inside -# the k8s-external-secrets namespace so the external-secrets pod knows how to trust -# the https://vault-vault.apps.hub-domain... endpoint ---- # Source: acm/templates/multiclusterhub.yaml apiVersion: operator.open-cluster-management.io/v1 kind: MultiClusterHub FAIL on acm naked with opts [] make: *** [Makefile:34: test] Error 1 This was just a leftover for when we removed the hub-ca app that pushed it around to managed clusters. * Fix tests/clustergroup-normal.expected.yaml test * Add initial checking github action on every pull/push At the moment this only runs "make test". We'll later expand this to do some additional linting. * Add a helmlint target that runs helm lint over the charts ❯ make helmlint ==> Linting install [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ==> Linting clustergroup [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ==> Linting acm [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed * Run make helmlint on every push/pull request * Add golang-external-secrets to the charts being tested * Add right params to helmlint ❯ make helmlint ==> Linting install [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ==> Linting clustergroup [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ==> Linting acm [INFO] Chart.yaml: icon is recommended 1 chart(s) linted, 0 chart(s) failed ==> Linting golang-external-secrets [INFO] Chart.yaml: icon is recommended [WARNING] /home/michele/Engineering/cloud-patterns/common/golang-external-secrets: chart directory is missing these dependencies: external-secrets 1 chart(s) linted, 0 chart(s) failed * Move make_common_subtree.sh to common from multicloud-gitops * Updated values-datacenter example values file with working content * adding working information to values-global * updated readme with useable Makefile * Removed previous version of common to convert to subtree from https://github.com/hybrid-cloud-patterns/common.git main * updating options in values-datacenter * generalizing README Co-authored-by: Andrew Beekhof Co-authored-by: Martin Jackson Co-authored-by: Martin Jackson Co-authored-by: Lester Claudio Co-authored-by: Wolfgang Kulhanek Co-authored-by: Michele Baldessari Co-authored-by: day0hero --- README.md | 40 + common | 1 - common/.github/linters/.markdown-lint.yml | 6 + common/.github/workflows/linter.yml | 56 + common/.gitignore | 7 + common/Makefile | 62 + common/Makefile.toplevel | 16 + common/acm/Chart.yaml | 6 + common/acm/templates/multiclusterhub.yaml | 8 + .../policies/application-policies.yaml | 108 + .../templates/policies/ocp-gitops-policy.yaml | 78 + common/acm/values.yaml | 9 + common/clustergroup/.helmignore | 1 + common/clustergroup/Chart.yaml | 6 + .../clustergroup/templates/applications.yaml | 61 + .../templates/argocd-super-role.yaml | 37 + common/clustergroup/templates/argocd.yaml | 98 + .../templates/gitops-namespace.yaml | 11 + common/clustergroup/templates/namespaces.yaml | 11 + .../clustergroup/templates/operatorgroup.yaml | 25 + common/clustergroup/templates/projects.yaml | 22 + .../clustergroup/templates/subscriptions.yaml | 40 + common/clustergroup/values.yaml | 50 + common/common | 1 + common/examples/kustomize-renderer/Chart.yaml | 6 + .../kustomize-renderer/environment.yaml | 34 + .../kustomize-renderer/kustomization.yaml | 5 + common/examples/kustomize-renderer/kustomize | 15 + .../templates/environment.yaml | 34 + .../examples/kustomize-renderer/values.yaml | 11 + common/examples/values-example.yaml | 60 + common/examples/values-secret.yaml | 22 + common/golang-external-secrets/Chart.yaml | 11 + ...ternal-secrets-hub-clusterrolebinding.yaml | 14 + ...lang-external-secrets-hub-secretstore.yaml | 24 + common/golang-external-secrets/values.yaml | 3 + common/install/.helmignore | 1 + common/install/Chart.yaml | 6 + .../crds/applications.argoproj.io.yaml | 2162 +++++++++++++++++ .../install/templates/argocd/application.yaml | 38 + .../install/templates/argocd/namespace.yaml | 8 + .../templates/argocd/subscription.yaml | 22 + common/install/values.yaml | 27 + common/reference-output.yaml | 119 + common/scripts/ansible-push-vault-secrets.sh | 120 + common/scripts/make_common_subtree.sh | 76 + common/scripts/secret.sh | 52 + common/scripts/test.sh | 39 + common/scripts/vault-token.sh | 64 + common/scripts/vault-utils.sh | 219 ++ common/tests/acm-naked.expected.yaml | 95 + common/tests/acm-normal.expected.yaml | 209 ++ common/tests/clustergroup-naked.expected.yaml | 152 ++ .../tests/clustergroup-normal.expected.yaml | 326 +++ common/tests/install-naked.expected.yaml | 63 + common/tests/install-normal.expected.yaml | 63 + common/values-global.yaml | 16 + scripts/make_common_subtree.sh | 77 + values-datacenter.yaml | 27 +- values-global.yaml | 7 +- 60 files changed, 4976 insertions(+), 11 deletions(-) delete mode 160000 common create mode 100644 common/.github/linters/.markdown-lint.yml create mode 100644 common/.github/workflows/linter.yml create mode 100644 common/.gitignore create mode 100644 common/Makefile create mode 100644 common/Makefile.toplevel create mode 100644 common/acm/Chart.yaml create mode 100644 common/acm/templates/multiclusterhub.yaml create mode 100644 common/acm/templates/policies/application-policies.yaml create mode 100644 common/acm/templates/policies/ocp-gitops-policy.yaml create mode 100644 common/acm/values.yaml create mode 100644 common/clustergroup/.helmignore create mode 100644 common/clustergroup/Chart.yaml create mode 100644 common/clustergroup/templates/applications.yaml create mode 100644 common/clustergroup/templates/argocd-super-role.yaml create mode 100644 common/clustergroup/templates/argocd.yaml create mode 100644 common/clustergroup/templates/gitops-namespace.yaml create mode 100644 common/clustergroup/templates/namespaces.yaml create mode 100644 common/clustergroup/templates/operatorgroup.yaml create mode 100644 common/clustergroup/templates/projects.yaml create mode 100644 common/clustergroup/templates/subscriptions.yaml create mode 100644 common/clustergroup/values.yaml create mode 120000 common/common create mode 100644 common/examples/kustomize-renderer/Chart.yaml create mode 100644 common/examples/kustomize-renderer/environment.yaml create mode 100644 common/examples/kustomize-renderer/kustomization.yaml create mode 100755 common/examples/kustomize-renderer/kustomize create mode 100644 common/examples/kustomize-renderer/templates/environment.yaml create mode 100644 common/examples/kustomize-renderer/values.yaml create mode 100644 common/examples/values-example.yaml create mode 100644 common/examples/values-secret.yaml create mode 100644 common/golang-external-secrets/Chart.yaml create mode 100644 common/golang-external-secrets/templates/golang-external-secrets-hub-clusterrolebinding.yaml create mode 100644 common/golang-external-secrets/templates/golang-external-secrets-hub-secretstore.yaml create mode 100644 common/golang-external-secrets/values.yaml create mode 100644 common/install/.helmignore create mode 100644 common/install/Chart.yaml create mode 100644 common/install/crds/applications.argoproj.io.yaml create mode 100644 common/install/templates/argocd/application.yaml create mode 100644 common/install/templates/argocd/namespace.yaml create mode 100644 common/install/templates/argocd/subscription.yaml create mode 100644 common/install/values.yaml create mode 100644 common/reference-output.yaml create mode 100755 common/scripts/ansible-push-vault-secrets.sh create mode 100755 common/scripts/make_common_subtree.sh create mode 100755 common/scripts/secret.sh create mode 100755 common/scripts/test.sh create mode 100755 common/scripts/vault-token.sh create mode 100755 common/scripts/vault-utils.sh create mode 100644 common/tests/acm-naked.expected.yaml create mode 100644 common/tests/acm-normal.expected.yaml create mode 100644 common/tests/clustergroup-naked.expected.yaml create mode 100644 common/tests/clustergroup-normal.expected.yaml create mode 100644 common/tests/install-naked.expected.yaml create mode 100644 common/tests/install-normal.expected.yaml create mode 100644 common/values-global.yaml create mode 100755 scripts/make_common_subtree.sh diff --git a/README.md b/README.md index fd6835c..4a267c7 100644 --- a/README.md +++ b/README.md @@ -1 +1,41 @@ # A template of a basic GitOps based pattern + +### Example Makefile +The below Makefile is an example that can be used to trigger your deployments: +```sh +BOOTSTRAP=1 +ARGO_TARGET_NAMESPACE=demo-namespace +PATTERN=vessel-id +COMPONENT=datacenter +SECRET_NAME="argocd-env" +TARGET_REPO=$(shell git remote show origin | grep Push | sed -e 's/.*URL://' -e 's%:[a-z].*@%@%' -e 's%:%/%' -e 's%git@%https://%' ) +CHART_OPTS=-f common/examples/values-secret.yaml -f values-global.yaml -f values-datacenter.yaml --set global.targetRevision=main --set global.valuesDirectoryURL="https://github.com/pattern-clone/pattern/raw/main/" --set global.pattern="dema-pattern" --set global.namespace="pattern-namespace" +NAME=$(shell basename `pwd`) + +.PHONY: default +default: show + +%: + echo "Delegating $* target" + make -f common/Makefile $* + +install: deploy +ifeq ($(BOOTSTRAP),1) + echo "Bootstrapping Demo Pattern" +endif + +predeploy: + ./scripts/precheck.sh + +update: upgrade +ifeq ($(BOOTSTRAP),1) + echo "Bootstrapping Demo Pattern" + make bootstrap +endif + +bootstrap: + +test: + make -f common/Makefile CHARTS="$(wildcard charts/datacenter/*)" PATTERN_OPTS="-f values-datacenter.yaml" test + make -f common/Makefile CHARTS="$(wildcard charts/factory/*)" PATTERN_OPTS="-f values-factory.yaml" test +``` diff --git a/common b/common deleted file mode 160000 index 2f3becc..0000000 --- a/common +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 2f3becce92d9030126d7a0fc6c30f63cf618435f diff --git a/common/.github/linters/.markdown-lint.yml b/common/.github/linters/.markdown-lint.yml new file mode 100644 index 0000000..a0bc47d --- /dev/null +++ b/common/.github/linters/.markdown-lint.yml @@ -0,0 +1,6 @@ +{ + "default": true, + "MD003": false, + "MD013": false, + "MD033": false +} \ No newline at end of file diff --git a/common/.github/workflows/linter.yml b/common/.github/workflows/linter.yml new file mode 100644 index 0000000..6f26172 --- /dev/null +++ b/common/.github/workflows/linter.yml @@ -0,0 +1,56 @@ +--- +name: Unit test common + +# +# Documentation: +# https://help.github.com/en/articles/workflow-syntax-for-github-actions +# + +############################# +# Start the job on all push # +############################# +on: [push, pull_request] + +############### +# Set the Job # +############### +jobs: + build: + # Name the Job + name: Unit common/ Code Base + # Set the agent to run on + runs-on: ubuntu-latest + + ################## + # Load all steps # + ################## + steps: + ########################## + # Checkout the code base # + ########################## + - name: Checkout Code + uses: actions/checkout@v2 + with: + # Full git history is needed to get a proper list of changed files within `super-linter` + fetch-depth: 0 + - name: Setup helm + uses: azure/setup-helm@v1 + # with: + # version: '' # default is latest stable + id: install + + ################################ + # Run Linter against code base # + ################################ + # - name: Lint Code Base + # uses: github/super-linter@v4 + # env: + # VALIDATE_ALL_CODEBASE: false + # DEFAULT_BRANCH: main + # GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + - name: Run make test + run: | + make test + - name: Run make helmlint + run: | + make helmlint diff --git a/common/.gitignore b/common/.gitignore new file mode 100644 index 0000000..765c0dd --- /dev/null +++ b/common/.gitignore @@ -0,0 +1,7 @@ +*~ +*.swp +*.swo +values-secret.yaml +.*.expected.yaml +pattern-vault.init +vault.init diff --git a/common/Makefile b/common/Makefile new file mode 100644 index 0000000..f88449e --- /dev/null +++ b/common/Makefile @@ -0,0 +1,62 @@ +BOOTSTRAP=1 +SECRETS=~/values-secret.yaml +NAME=$(shell basename `pwd`) +# This is to ensure that whether we start with a git@ or https:// URL, we end up with an https:// URL +# This is because we expect to use tokens for repo authentication as opposed to SSH keys +TARGET_REPO=$(shell git remote show origin | grep Push | sed -e 's/.*URL:[[:space:]]*//' -e 's%^git@%%' -e 's%^https://%%' -e 's%:%/%' -e 's%^%https://%') +# git branch --show-current is also available as of git 2.22, but we will use this for compatibility +TARGET_BRANCH=$(shell git rev-parse --abbrev-ref HEAD) +HUBCLUSTER_APPS_DOMAIN=$(shell oc get ingresses.config/cluster -o jsonpath={.spec.domain}) + +# --set values always take precedence over the contents of -f +HELM_OPTS=-f values-global.yaml -f $(SECRETS) --set main.git.repoURL="$(TARGET_REPO)" --set main.git.revision=$(TARGET_BRANCH) --set main.options.bootstrap=$(BOOTSTRAP) --set global.hubClusterDomain=$(HUBCLUSTER_APPS_DOMAIN) +TEST_OPTS= -f common/examples/values-secret.yaml -f values-global.yaml --set global.repoURL="https://github.com/pattern-clone/mypattern" --set main.git.repoURL="https://github.com/pattern-clone/mypattern" --set main.git.revision=main --set main.options.bootstrap=$(BOOTSTRAP) --set global.valuesDirectoryURL="https://github.com/pattern-clone/mypattern/raw/main" --set global.pattern="mypattern" --set global.namespace="pattern-namespace" --set global.hubClusterDomain=hub.example.com --set global.localClusterDomain=region.example.com +PATTERN_OPTS=-f common/examples/values-example.yaml + +# Makefiles that use this target must provide: +# PATTERN: The name of the pattern that is using it. This will be used programmatically for the source namespace +# TARGET_NAMESPACE: target namespace to install the secret into +# COMPONENT: The component of the target namespace. In industrial edge, factory or datacenter - and for the secret +# it needs to be datacenter because that's where the CI components run. +# SECRET_NAME: The name of the secret to manage +argosecret: + PATTERN="$(PATTERN)" TARGET_NAMESPACE="$(TARGET_NAMESPACE)" COMPONENT="$(COMPONENT)" SECRET_NAME="$(SECRET_NAME)" common/scripts/secret.sh + +# Makefiles in the individual patterns should call these targets explicitly +# e.g. from industrial-edge: make -f common/Makefile show +show: + helm template common/install/ --name-template $(NAME) $(HELM_OPTS) + +CHARTS=install clustergroup acm golang-external-secrets + +test: +# Test that all values used by the chart are in values.yaml with the same defaults as the pattern + @for t in $(CHARTS); do common/scripts/test.sh $$t naked ""; if [ $$? != 0 ]; then exit 1; fi; done +# Test the charts as the pattern would drive them + @for t in $(CHARTS); do common/scripts/test.sh $$t normal "$(TEST_OPTS) $(PATTERN_OPTS)"; if [ $$? != 0 ]; then exit 1; fi; done + +helmlint: + @for t in $(CHARTS); do helm lint $(TEST_OPTS) $(PATTERN_OPTS) $$t; if [ $$? != 0 ]; then exit 1; fi; done + +validate-origin: + git ls-remote $(TARGET_REPO) + +init: + git submodule update --init --recursive + +deploy: validate-origin + helm install $(NAME) common/install/ $(HELM_OPTS) + +upgrade: validate-origin + helm upgrade $(NAME) common/install/ $(HELM_OPTS) + +uninstall: + helm uninstall $(NAME) + +vault-init: + common/scripts/vault-utils.sh vault_init common/pattern-vault.init + +vault-unseal: + common/scripts/vault-utils.sh vault_unseal common/pattern-vault.init + +.phony: install test diff --git a/common/Makefile.toplevel b/common/Makefile.toplevel new file mode 100644 index 0000000..1317d8b --- /dev/null +++ b/common/Makefile.toplevel @@ -0,0 +1,16 @@ +BOOTSTRAP=1 +ARGO_TARGET_NAMESPACE=replaceme + +.PHONY: default +default: show + +%: + make -f common/Makefile $* + +install: deploy +ifeq ($(BOOTSTRAP),1) + make -f common/Makefile TARGET_NAMESPACE=$(ARGO_TARGET_NAMESPACE) argosecret +endif + +secret: + make -f common/Makefile TARGET_NAMESPACE=$(ARGO_TARGET_NAMESPACE) argosecret diff --git a/common/acm/Chart.yaml b/common/acm/Chart.yaml new file mode 100644 index 0000000..1c3db91 --- /dev/null +++ b/common/acm/Chart.yaml @@ -0,0 +1,6 @@ +apiVersion: v2 +description: A Helm chart to configure Advanced Cluster Manager for OpenShift +keywords: +- pattern +name: acm +version: 0.0.1 diff --git a/common/acm/templates/multiclusterhub.yaml b/common/acm/templates/multiclusterhub.yaml new file mode 100644 index 0000000..f925d5a --- /dev/null +++ b/common/acm/templates/multiclusterhub.yaml @@ -0,0 +1,8 @@ +apiVersion: operator.open-cluster-management.io/v1 +kind: MultiClusterHub +metadata: + name: multiclusterhub + namespace: open-cluster-management + annotations: + argocd.argoproj.io/sync-wave: "-1" +spec: {} diff --git a/common/acm/templates/policies/application-policies.yaml b/common/acm/templates/policies/application-policies.yaml new file mode 100644 index 0000000..5ab3d8b --- /dev/null +++ b/common/acm/templates/policies/application-policies.yaml @@ -0,0 +1,108 @@ +# TODO: Also create a GitOpsCluster.apps.open-cluster-management.io +{{- range .Values.clusterGroup.managedClusterGroups }} +apiVersion: policy.open-cluster-management.io/v1 +kind: Policy +metadata: + name: {{ .name }}-clustergroup-policy + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + remediationAction: enforce + disabled: false + policy-templates: + - objectDefinition: + apiVersion: policy.open-cluster-management.io/v1 + kind: ConfigurationPolicy + metadata: + name: {{ .name }}-clustergroup-config + spec: + remediationAction: enforce + severity: med + namespaceSelector: + exclude: + - kube-* + include: + - default + object-templates: + - complianceType: musthave + objectDefinition: + apiVersion: argoproj.io/v1alpha1 + kind: Application + metadata: + name: {{ $.Values.global.pattern }}-{{ .name }} + namespace: openshift-gitops + finalizers: + - argoproj.io/finalizer + spec: + project: default + source: + repoURL: {{ coalesce .repoURL $.Values.global.repoURL }} + targetRevision: {{ coalesce .targetRevision $.Values.global.targetRevision }} + path: {{ default "common/clustergroup" .path }} + helm: + valueFiles: + - "{{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }}/values-global.yaml" + - "{{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }}/values-{{ .name }}.yaml" + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.pattern + value: {{ $.Values.global.pattern }} + - name: global.valuesDirectoryURL + value: {{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }} + - name: global.hubClusterDomain + value: {{ $.Values.global.hubClusterDomain }} + - name: global.localClusterDomain + value: '{{ `{{ (lookup "config.openshift.io/v1" "Ingress" "" "cluster").spec.domain }}` }}' + {{- range .helmOverrides }} + - name: {{ .name }} + value: {{ .value | quote }} + {{- end }} + destination: + server: https://kubernetes.default.svc + namespace: {{ $.Values.global.pattern }}-{{ .name }} + syncPolicy: + automated: + prune: false + selfHeal: true + ignoreDifferences: + - group: apps + kind: Deployment + jsonPointers: + - /spec/replicas + - group: route.openshift.io + kind: Route + jsonPointers: + - /status +--- +apiVersion: policy.open-cluster-management.io/v1 +kind: PlacementBinding +metadata: + name: {{ .name }}-placement-binding + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +placementRef: + name: {{ .name }}-placement + kind: PlacementRule + apiGroup: apps.open-cluster-management.io +subjects: + - name: {{ .name }}-clustergroup-policy + kind: Policy + apiGroup: policy.open-cluster-management.io +--- +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: {{ .name }}-placement +spec: + clusterConditions: + - status: 'True' + type: ManagedClusterConditionAvailable + clusterSelector: {{ .clusterSelector | toPrettyJson }} +--- +{{- end }} diff --git a/common/acm/templates/policies/ocp-gitops-policy.yaml b/common/acm/templates/policies/ocp-gitops-policy.yaml new file mode 100644 index 0000000..e6f7013 --- /dev/null +++ b/common/acm/templates/policies/ocp-gitops-policy.yaml @@ -0,0 +1,78 @@ +apiVersion: policy.open-cluster-management.io/v1 +kind: Policy +metadata: + name: openshift-gitops-policy + annotations: + policy.open-cluster-management.io/standards: NIST-CSF + policy.open-cluster-management.io/categories: PR.DS Data Security + policy.open-cluster-management.io/controls: PR.DS-1 Data-at-rest + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + remediationAction: enforce + disabled: false + policy-templates: + - objectDefinition: + apiVersion: policy.open-cluster-management.io/v1 + kind: ConfigurationPolicy + metadata: + name: openshift-gitops-config + spec: + remediationAction: enforce + severity: med + namespaceSelector: + exclude: + - kube-* + include: + - default + object-templates: + - complianceType: musthave + objectDefinition: + # This is an auto-generated file. DO NOT EDIT + apiVersion: operators.coreos.com/v1alpha1 + kind: Subscription + metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: '' + spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: "*" +--- +apiVersion: policy.open-cluster-management.io/v1 +kind: PlacementBinding +metadata: + name: openshift-gitops-placement-binding + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +placementRef: + name: openshift-gitops-placement + kind: PlacementRule + apiGroup: apps.open-cluster-management.io +subjects: + - name: openshift-gitops-policy + kind: Policy + apiGroup: policy.open-cluster-management.io +--- +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: openshift-gitops-placement +spec: + clusterConditions: + - status: 'True' + type: ManagedClusterConditionAvailable + clusterSelector: + matchExpressions: + - key: vendor + operator: In + values: + - OpenShift diff --git a/common/acm/values.yaml b/common/acm/values.yaml new file mode 100644 index 0000000..d9c9458 --- /dev/null +++ b/common/acm/values.yaml @@ -0,0 +1,9 @@ +global: + pattern: none + valuesDirectoryURL: none + repoURL: none + targetRevision: main + + +clusterGroup: + managedClusterGroups: diff --git a/common/clustergroup/.helmignore b/common/clustergroup/.helmignore new file mode 100644 index 0000000..b25c15b --- /dev/null +++ b/common/clustergroup/.helmignore @@ -0,0 +1 @@ +*~ diff --git a/common/clustergroup/Chart.yaml b/common/clustergroup/Chart.yaml new file mode 100644 index 0000000..249163a --- /dev/null +++ b/common/clustergroup/Chart.yaml @@ -0,0 +1,6 @@ +apiVersion: v2 +description: A Helm chart to create per-clustergroup ArgoCD applications and any required namespaces or subscriptions +keywords: +- pattern +name: pattern-clustergroup +version: 0.0.1 diff --git a/common/clustergroup/templates/applications.yaml b/common/clustergroup/templates/applications.yaml new file mode 100644 index 0000000..aefe68d --- /dev/null +++ b/common/clustergroup/templates/applications.yaml @@ -0,0 +1,61 @@ +{{- $namespace := cat $.Values.global.pattern $.Values.clusterGroup.name | replace " " "-" }} +{{- range .Values.clusterGroup.applications }} +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: {{ .name }} + namespace: {{ $namespace }} +spec: + destination: + name: in-cluster + namespace: {{ default $namespace .namespace }} + project: {{ .project }} + source: + repoURL: {{ coalesce .repoURL $.Values.global.repoURL }} + targetRevision: {{ coalesce .targetRevision $.Values.global.targetRevision }} + {{- if .chart }} + chart: {{ .chart }} + {{- else }} + path: {{ .path }} + {{- end }} + {{- if .plugin }} + plugin: {{ .plugin | toPrettyJson }} + {{- else if not .kustomize }} + helm: + valueFiles: + - "{{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }}/values-global.yaml" + - "{{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }}/values-{{ $.Values.clusterGroup.name }}.yaml" + # Watch the progress of https://issues.redhat.com/browse/GITOPS-891 and update accordingly + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.pattern + value: {{ $.Values.global.pattern }} + - name: global.valuesDirectoryURL + value: {{ coalesce .valuesDirectoryURL $.Values.global.valuesDirectoryURL }} + - name: global.hubClusterDomain + value: {{ $.Values.global.hubClusterDomain }} + - name: global.localClusterDomain + value: {{ coalesce $.Values.global.localClusterDomain $.Values.global.hubClusterDomain }} + {{- range .overrides }} + - name: {{ .name }} + value: {{ .value | quote }} + {{- if .forceString }} + forceString: true + {{- end }} + {{- end }} + {{- end }} + {{- if .ignoreDifferences }} + ignoreDifferences: {{ .ignoreDifferences | toPrettyJson }} + {{- end }} +{{- if eq $.Values.global.options.syncPolicy "Automatic" }} + syncPolicy: + automated: {} + # selfHeal: true +{{- end }} +--- +{{- end }} diff --git a/common/clustergroup/templates/argocd-super-role.yaml b/common/clustergroup/templates/argocd-super-role.yaml new file mode 100644 index 0000000..f3c885f --- /dev/null +++ b/common/clustergroup/templates/argocd-super-role.yaml @@ -0,0 +1,37 @@ +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: openshift-gitops-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + name: openshift-gitops-argocd-application-controller + namespace: openshift-gitops + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + name: openshift-gitops-argocd-server + namespace: openshift-gitops +--- +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: {{ $.Values.global.pattern }}-{{ .Values.clusterGroup.name }}-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-application-controller + name: {{ .Values.clusterGroup.name }}-gitops-argocd-application-controller + namespace: {{ $.Values.global.pattern }}-{{ .Values.clusterGroup.name }} + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-server + name: {{ .Values.clusterGroup.name }}-gitops-argocd-server + namespace: {{ $.Values.global.pattern }}-{{ .Values.clusterGroup.name }} diff --git a/common/clustergroup/templates/argocd.yaml b/common/clustergroup/templates/argocd.yaml new file mode 100644 index 0000000..3423ecc --- /dev/null +++ b/common/clustergroup/templates/argocd.yaml @@ -0,0 +1,98 @@ +{{- $namespace := cat $.Values.global.pattern $.Values.clusterGroup.name | replace " " "-" }} +apiVersion: argoproj.io/v1alpha1 +kind: ArgoCD +metadata: + finalizers: + - argoproj.io/finalizer + # Changing the name affects the ClusterRoleBinding, the generated secret, + # route URL, and argocd.argoproj.io/managed-by annotations + name: {{ .Values.clusterGroup.name }}-gitops + namespace: {{ $namespace }} + annotations: + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + applicationInstanceLabelKey: argocd.argoproj.io/instance + # Not the greatest way to pass git/quay info to sub-applications, but it will do until + # we can support helmChart with kustomize + # The other option is to pass them in as environment variables eg. BLUEPRINT_VERSION + configManagementPlugins: "- name: kustomize-version\n generate:\n command: [sh, + -c]\n args: [\"kustomize version 1>&2 && exit 1\"]\n- name: kustomize-with-helm\n + \ generate:\n command: [\"kustomize\"]\n args: [\"build\", \"--enable-helm\"]\n- name: + helm-with-kustomize\n init:\n command: [\"/bin/sh\", \"-c\"]\n args: [\"helm + dependency build\"]\n generate:\n command: [/bin/bash, -c]\n args: [\"helm template . --name-template ${ARGOCD_APP_NAME:0:52} + -f {{ .Values.global.valuesDirectoryURL }}/values-global.yaml + -f {{ .Values.global.valuesDirectoryURL }}/values-{{ .Values.clusterGroup.name }}.yaml + --set global.repoURL=$ARGOCD_APP_SOURCE_REPO_URL + --set global.targetRevision=$ARGOCD_APP_SOURCE_TARGET_REVISION + --set global.namespace=$ARGOCD_APP_NAMESPACE + --set global.pattern={{ .Values.global.pattern }} + --set global.hubClusterDomain={{ .Values.global.hubClusterDomain }} + --set global.localClusterDomain={{ coalesce .Values.global.localClusterDomain .Values.global.hubClusterDomain }} + --set global.valuesDirectoryURL={{ .Values.global.valuesDirectoryURL }} + --post-renderer ./kustomize\"] + \ \n" + applicationSet: + resources: + limits: + cpu: "2" + memory: 1Gi + requests: + cpu: 250m + memory: 512Mi + controller: + processors: {} + resources: + limits: + cpu: "4" + memory: 4Gi + requests: + cpu: 500m + memory: 2Gi + dex: + openShiftOAuth: true + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 250m + memory: 128Mi + initialSSHKnownHosts: {} + rbac: + defaultPolicy: role:admin + repo: + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + resourceExclusions: | + - apiGroups: + - tekton.dev + kinds: + - TaskRun + - PipelineRun + server: + autoscale: + enabled: false + grpc: + ingress: + enabled: false + ingress: + enabled: false + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 125m + memory: 128Mi + route: + enabled: true + service: + type: "" + tls: + ca: {} +status: diff --git a/common/clustergroup/templates/gitops-namespace.yaml b/common/clustergroup/templates/gitops-namespace.yaml new file mode 100644 index 0000000..785ef75 --- /dev/null +++ b/common/clustergroup/templates/gitops-namespace.yaml @@ -0,0 +1,11 @@ +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: {{ $.Values.global.pattern }}-{{ .Values.clusterGroup.name }} + # The name here needs to be consistent with + # - acm/templates/policies/application-policies.yaml + # - clustergroup/templates/applications.yaml + # - any references to secrets and route URLs in documentation + name: {{ $.Values.global.pattern }}-{{ .Values.clusterGroup.name }} +spec: {} diff --git a/common/clustergroup/templates/namespaces.yaml b/common/clustergroup/templates/namespaces.yaml new file mode 100644 index 0000000..b3bc86a --- /dev/null +++ b/common/clustergroup/templates/namespaces.yaml @@ -0,0 +1,11 @@ +{{- range .Values.clusterGroup.namespaces }} +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: {{ default "pattern" $.Release.name }} + argocd.argoproj.io/managed-by: {{ $.Values.global.pattern }}-{{ $.Values.clusterGroup.name }} + name: {{ . }} +spec: +--- +{{- end }} diff --git a/common/clustergroup/templates/operatorgroup.yaml b/common/clustergroup/templates/operatorgroup.yaml new file mode 100644 index 0000000..0180a91 --- /dev/null +++ b/common/clustergroup/templates/operatorgroup.yaml @@ -0,0 +1,25 @@ +{{- range .Values.clusterGroup.namespaces }} + +{{- if empty $.Values.clusterGroup.operatorgroupExcludes }} +apiVersion: operators.coreos.com/v1 +kind: OperatorGroup +metadata: + name: {{ . }}-operator-group + namespace: {{ . }} +spec: + targetNamespaces: + - {{ . }} +--- +{{- else if not (has . $.Values.clusterGroup.operatorgroupExcludes) }} +apiVersion: operators.coreos.com/v1 +kind: OperatorGroup +metadata: + name: {{ . }}-operator-group + namespace: {{ . }} +spec: + targetNamespaces: + - {{ . }} +--- +{{- end }} + +{{- end }} diff --git a/common/clustergroup/templates/projects.yaml b/common/clustergroup/templates/projects.yaml new file mode 100644 index 0000000..d74e2cb --- /dev/null +++ b/common/clustergroup/templates/projects.yaml @@ -0,0 +1,22 @@ +{{- range .Values.clusterGroup.projects }} +apiVersion: argoproj.io/v1alpha1 +kind: AppProject +metadata: + name: {{ . }} + namespace: {{ $.Values.global.pattern }}-{{ $.Values.clusterGroup.name }} +spec: + description: "Pattern {{ . }}" + destinations: + - namespace: '*' + server: '*' + clusterResourceWhitelist: + - group: '*' + kind: '*' + namespaceResourceWhitelist: + - group: '*' + kind: '*' + sourceRepos: + - '*' +status: {} +--- +{{- end }} diff --git a/common/clustergroup/templates/subscriptions.yaml b/common/clustergroup/templates/subscriptions.yaml new file mode 100644 index 0000000..cf295bb --- /dev/null +++ b/common/clustergroup/templates/subscriptions.yaml @@ -0,0 +1,40 @@ +{{- range .Values.clusterGroup.subscriptions }} +{{- $subs := . }} +{{- $installPlanValue := .installPlanApproval }} + +{{- if $subs.namespaces }} +{{- range .namespaces }} +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: {{ $subs.name }} + namespace: {{ default "openshift-operators" . }} +spec: + name: {{ $subs.name }} + source: {{ default "redhat-operators" $subs.source }} + sourceNamespace: {{ default "openshift-marketplace" $subs.sourceNamespace }} + channel: {{ default "stable" $subs.channel }} + installPlanApproval: {{ coalesce $installPlanValue $.Values.global.options.installPlanApproval }} + {{- if $.Values.global.options.useCSV }} + startingCSV: {{ $subs.csv }} + {{- end }} +--- +{{- end }} +{{- else }} +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: {{ $subs.name }} + namespace: {{ default "openshift-operators" $subs.namespace }} +spec: + name: {{ $subs.name }} + source: {{ default "redhat-operators" $subs.source }} + sourceNamespace: {{ default "openshift-marketplace" $subs.sourceNamespace }} + channel: {{ default "stable" $subs.channel }} + installPlanApproval: {{ coalesce $installPlanValue $.Values.global.options.installPlanApproval }} + {{- if $.Values.global.options.useCSV }} + startingCSV: {{ $subs.csv }} + {{- end }} +--- +{{- end }} +{{- end }} diff --git a/common/clustergroup/values.yaml b/common/clustergroup/values.yaml new file mode 100644 index 0000000..aec0fcd --- /dev/null +++ b/common/clustergroup/values.yaml @@ -0,0 +1,50 @@ +global: + pattern: common + valuesDirectoryURL: https://github.com/pattern-clone/common/raw/main + options: + useCSV: True + syncPolicy: Automatic + installPlanApproval: Automatic + +clusterGroup: + name: example + + proposedOptions: + manageGitops: True + isHubCluster: True + +# managedClusterGroups: +# - name: factory +# # repoURL: https://github.com/dagger-refuse-cool/manuela-factory.git +# # Location of values-global.yaml, values-{name}.yaml, values-{app}.yaml +# # valuesDirectoryURL: https://github.com/dagger-refuse-cool/edge-gitops/raw/main/ +# targetRevision: main +# path: applications/factory +# helmOverrides: +# - name: clusterGroup.isHubCluster +# value: false +# clusterSelector: +# matchExpressions: +# - key: vendor +# operator: In +# values: +# - OpenShift +# +# namespaces: +# - open-cluster-management +# +# subscriptions: +# - name: advanced-cluster-management +# namespace: open-cluster-management +# source: redhat-operators +# channel: release-2.3 +# csv: v2.3.2 +# +# projects: +# - datacenter +# +# applications: +# - name: acm +# namespace: default +# project: datacenter +# path: applications/acm diff --git a/common/common b/common/common new file mode 120000 index 0000000..945c9b4 --- /dev/null +++ b/common/common @@ -0,0 +1 @@ +. \ No newline at end of file diff --git a/common/examples/kustomize-renderer/Chart.yaml b/common/examples/kustomize-renderer/Chart.yaml new file mode 100644 index 0000000..88a786c --- /dev/null +++ b/common/examples/kustomize-renderer/Chart.yaml @@ -0,0 +1,6 @@ +apiVersion: v2 +description: A Helm chart to demonstrate how to use with kustomize +keywords: +- pattern +name: example +version: 0.0.1 diff --git a/common/examples/kustomize-renderer/environment.yaml b/common/examples/kustomize-renderer/environment.yaml new file mode 100644 index 0000000..de4c48a --- /dev/null +++ b/common/examples/kustomize-renderer/environment.yaml @@ -0,0 +1,34 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: environment +data: + IMAGE_PROVIDER: {{ .Values.global.imageregistry.hostname }} + IMAGE_ACCOUNT: {{ .Values.global.imageregistry.account }} + GIT_EMAIL: {{ .Values.global.git.email }} + GIT_DEV_REPO_URL: https://{{ .Values.global.git.hostname }}/{{ .Values.global.git.account }}/manuela-dev.git + GIT_DEV_REPO_REVISION: {{ .Values.global.git.dev_revision }} + GIT_OPS_REPO_TEST_URL: {{ .Values.global.repoURL }} + GIT_OPS_REPO_TEST_REVISION: {{ .Values.global.targetRevision }} + GIT_OPS_REPO_PROD_URL: {{ .Values.global.repoURL }} + GIT_OPS_REPO_PROD_REVISION: {{ .Values.global.targetRevision }} + IOT_CONSUMER_IMAGE: iot-consumer + IOT_CONSUMER_YAML_PATH: images.(name==messaging).newTag + IOT_CONSUMER_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_CONSUMER_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/messaging/kustomization.yaml + IOT_CONSUMER_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/messaging/messaging-is.yaml + IOT_FRONTEND_IMAGE: iot-frontend + IOT_FRONTEND_YAML_PATH: images.(name==line-dashboard).newTag + IOT_FRONTEND_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_FRONTEND_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/line-dashboard/kustomization.yaml + IOT_FRONTEND_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/line-dashboard/line-dashboard-is.yaml + IOT_SWSENSOR_IMAGE: iot-software-sensor + IOT_SWSENSOR_YAML_PATH: images.(name==machine-sensor).newTag + IOT_SWSENSOR_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_SWSENSOR_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/machine-sensor/kustomization.yaml + IOT_SWSENSOR_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/machine-sensor/machine-sensor-is.yaml + IOT_ANOMALY_IMAGE: iot-anomaly-detection + IOT_ANOMALY_YAML_PATH: images.(name==anomaly-detection).newTag + IOT_ANOMALY_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_ANOMALY_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/anomaly-detection/kustomization.yaml + IOT_ANOMALY_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/anomaly-detection/anomaly-detection-is.yaml diff --git a/common/examples/kustomize-renderer/kustomization.yaml b/common/examples/kustomize-renderer/kustomization.yaml new file mode 100644 index 0000000..8d8bcd1 --- /dev/null +++ b/common/examples/kustomize-renderer/kustomization.yaml @@ -0,0 +1,5 @@ +resources: + - environment.yaml + +patches: +- helm.patch.yaml diff --git a/common/examples/kustomize-renderer/kustomize b/common/examples/kustomize-renderer/kustomize new file mode 100755 index 0000000..5f62b40 --- /dev/null +++ b/common/examples/kustomize-renderer/kustomize @@ -0,0 +1,15 @@ +#!/bin/bash -x + +BASE=`dirname $0` +if [ $BASE = $PWD ]; then + BASE=./ +fi + +cat <&0 > "$BASE/helm.patch.yaml" + +# Including at least one log to stderr allows us to see the full -x output +echo $HOME $PWD 1>&2 +ls -al 1>&2 + +kubectl kustomize "$BASE" && rm "$BASE/helm.patch.yaml" +#kubectl kustomize "$BASE" > "$BASE/result.yaml" diff --git a/common/examples/kustomize-renderer/templates/environment.yaml b/common/examples/kustomize-renderer/templates/environment.yaml new file mode 100644 index 0000000..de4c48a --- /dev/null +++ b/common/examples/kustomize-renderer/templates/environment.yaml @@ -0,0 +1,34 @@ +apiVersion: v1 +kind: ConfigMap +metadata: + name: environment +data: + IMAGE_PROVIDER: {{ .Values.global.imageregistry.hostname }} + IMAGE_ACCOUNT: {{ .Values.global.imageregistry.account }} + GIT_EMAIL: {{ .Values.global.git.email }} + GIT_DEV_REPO_URL: https://{{ .Values.global.git.hostname }}/{{ .Values.global.git.account }}/manuela-dev.git + GIT_DEV_REPO_REVISION: {{ .Values.global.git.dev_revision }} + GIT_OPS_REPO_TEST_URL: {{ .Values.global.repoURL }} + GIT_OPS_REPO_TEST_REVISION: {{ .Values.global.targetRevision }} + GIT_OPS_REPO_PROD_URL: {{ .Values.global.repoURL }} + GIT_OPS_REPO_PROD_REVISION: {{ .Values.global.targetRevision }} + IOT_CONSUMER_IMAGE: iot-consumer + IOT_CONSUMER_YAML_PATH: images.(name==messaging).newTag + IOT_CONSUMER_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_CONSUMER_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/messaging/kustomization.yaml + IOT_CONSUMER_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/messaging/messaging-is.yaml + IOT_FRONTEND_IMAGE: iot-frontend + IOT_FRONTEND_YAML_PATH: images.(name==line-dashboard).newTag + IOT_FRONTEND_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_FRONTEND_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/line-dashboard/kustomization.yaml + IOT_FRONTEND_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/line-dashboard/line-dashboard-is.yaml + IOT_SWSENSOR_IMAGE: iot-software-sensor + IOT_SWSENSOR_YAML_PATH: images.(name==machine-sensor).newTag + IOT_SWSENSOR_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_SWSENSOR_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/machine-sensor/kustomization.yaml + IOT_SWSENSOR_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/machine-sensor/machine-sensor-is.yaml + IOT_ANOMALY_IMAGE: iot-anomaly-detection + IOT_ANOMALY_YAML_PATH: images.(name==anomaly-detection).newTag + IOT_ANOMALY_TEST_KUSTOMIZATION_PATH: charts/datacenter/manuela-tst/kustomization.yaml + IOT_ANOMALY_PROD_KUSTOMIZATION_PATH: charts/factory/manuela-stormshift/anomaly-detection/kustomization.yaml + IOT_ANOMALY_PROD_IMAGESTREAM_PATH: charts/factory/manuela-stormshift/anomaly-detection/anomaly-detection-is.yaml diff --git a/common/examples/kustomize-renderer/values.yaml b/common/examples/kustomize-renderer/values.yaml new file mode 100644 index 0000000..e713c39 --- /dev/null +++ b/common/examples/kustomize-renderer/values.yaml @@ -0,0 +1,11 @@ +global: + git: + provider: github.com + account: PLAINTEXT + username: PLAINTEXT + email: SOMEWHERE@EXAMPLE.COM + dev_revision: main + + quay: + provider: quay.io + account: PLAINTEXT diff --git a/common/examples/values-example.yaml b/common/examples/values-example.yaml new file mode 100644 index 0000000..47d79e3 --- /dev/null +++ b/common/examples/values-example.yaml @@ -0,0 +1,60 @@ +global: + options: + useCSV: False + syncPolicy: Automatic + installPlanApproval: Automatic + +clusterGroup: + name: example + + namespaces: + - open-cluster-management + - application-ci + + subscriptions: + - name: advanced-cluster-management + namespace: open-cluster-management + channel: release-2.3 + csv: advanced-cluster-management.v2.3.2 + + - name: openshift-pipelines-operator-rh + csv: redhat-openshift-pipelines.v1.5.1 + + projects: + - datacenter + + applications: + - name: acm + namespace: open-cluster-management + project: datacenter + path: common/acm + ignoreDifferences: + - group: internal.open-cluster-management.io + kind: ManagedClusterInfo + jsonPointers: + - /spec/loggingCA + + - name: pipelines + namespace: application-ci + project: datacenter + path: charts/datacenter/pipelines + + managedClusterGroups: + - name: edge + # Optional - Point to a different repo + # repoURL: https://github.com/dagger-refuse-cool/mySite.git + # Must contain values-{clustergroupname}.yaml at the top level + targetRevision: main + helmOverrides: + # Values must be strings! + - name: clusterGroup.isHubCluster + value: "false" + clusterSelector: +# matchLabels: +# clusterGroup: factory + matchExpressions: + - key: vendor + operator: In + values: + - OpenShift + diff --git a/common/examples/values-secret.yaml b/common/examples/values-secret.yaml new file mode 100644 index 0000000..89904c6 --- /dev/null +++ b/common/examples/values-secret.yaml @@ -0,0 +1,22 @@ +main: + git: + repoURL: https://github.com/example/common + +secrets: + # NEVER COMMIT THESE VALUES TO GIT + enabled: true + imageregistry: + # Quay -> Robot Accounts -> Robot Login + account: test-account + token: test-quay-token + + git: + # Go to: https://github.com/settings/tokens + username: test-user + token: test-git-token + + aws: + s3Secret: test-secret + + + diff --git a/common/golang-external-secrets/Chart.yaml b/common/golang-external-secrets/Chart.yaml new file mode 100644 index 0000000..141bf91 --- /dev/null +++ b/common/golang-external-secrets/Chart.yaml @@ -0,0 +1,11 @@ +apiVersion: v2 +description: A Helm chart to configure the golang-based external-secrets +keywords: +- pattern +name: golang-external-secrets +version: 0.0.1 +dependencies: + - name: external-secrets + version: "0.4.2" + repository: "https://charts.external-secrets.io" + #"https://external-secrets.github.io/kubernetes-external-secrets" diff --git a/common/golang-external-secrets/templates/golang-external-secrets-hub-clusterrolebinding.yaml b/common/golang-external-secrets/templates/golang-external-secrets-hub-clusterrolebinding.yaml new file mode 100644 index 0000000..17fd483 --- /dev/null +++ b/common/golang-external-secrets/templates/golang-external-secrets-hub-clusterrolebinding.yaml @@ -0,0 +1,14 @@ +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: role-tokenreview-binding + namespace: default +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: system:auth-delegator +subjects: +- kind: ServiceAccount + name: golang-external-secrets + namespace: golang-external-secrets diff --git a/common/golang-external-secrets/templates/golang-external-secrets-hub-secretstore.yaml b/common/golang-external-secrets/templates/golang-external-secrets-hub-secretstore.yaml new file mode 100644 index 0000000..9980ca9 --- /dev/null +++ b/common/golang-external-secrets/templates/golang-external-secrets-hub-secretstore.yaml @@ -0,0 +1,24 @@ +apiVersion: external-secrets.io/v1alpha1 +kind: ClusterSecretStore +metadata: + name: vault-backend + namespace: golang-external-secrets +spec: + provider: + vault: + server: https://vault-vault.{{ .Values.global.hubClusterDomain }} + path: secret + # Version of KV backend + version: v2 + caProvider: + type: ConfigMap + name: kube-root-ca.crt + key: ca.crt + namespace: golang-external-secrets + auth: + kubernetes: + mountPath: {{ .Values.mountPath }} + role: {{ .Values.mountRole }} + serviceAccountRef: + name: golang-external-secrets + namespace: golang-external-secrets diff --git a/common/golang-external-secrets/values.yaml b/common/golang-external-secrets/values.yaml new file mode 100644 index 0000000..7724e9f --- /dev/null +++ b/common/golang-external-secrets/values.yaml @@ -0,0 +1,3 @@ +--- +mountPath: "hub" +mountRole: "hub-role" diff --git a/common/install/.helmignore b/common/install/.helmignore new file mode 100644 index 0000000..b25c15b --- /dev/null +++ b/common/install/.helmignore @@ -0,0 +1 @@ +*~ diff --git a/common/install/Chart.yaml b/common/install/Chart.yaml new file mode 100644 index 0000000..d1c80ab --- /dev/null +++ b/common/install/Chart.yaml @@ -0,0 +1,6 @@ +apiVersion: v2 +description: A Helm chart to build and deploy a Cloud Pattern +keywords: +- pattern +name: pattern-install +version: 0.0.1 diff --git a/common/install/crds/applications.argoproj.io.yaml b/common/install/crds/applications.argoproj.io.yaml new file mode 100644 index 0000000..094bf05 --- /dev/null +++ b/common/install/crds/applications.argoproj.io.yaml @@ -0,0 +1,2162 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + generation: 1 + labels: + app.kubernetes.io/name: applications.argoproj.io + app.kubernetes.io/part-of: argocd + operators.coreos.com/openshift-gitops-operator.openshift-operators: "" + name: applications.argoproj.io +spec: + conversion: + strategy: None + group: argoproj.io + names: + kind: Application + listKind: ApplicationList + plural: applications + shortNames: + - app + - apps + singular: application + scope: Namespaced + versions: + - additionalPrinterColumns: + - jsonPath: .status.sync.status + name: Sync Status + type: string + - jsonPath: .status.health.status + name: Health Status + type: string + - jsonPath: .status.sync.revision + name: Revision + priority: 10 + type: string + name: v1alpha1 + schema: + openAPIV3Schema: + description: Application is a definition of Application resource. + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds' + type: string + metadata: + type: object + operation: + description: Operation contains information about a requested or running + operation + properties: + info: + description: Info is a list of informational items for this operation + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + initiatedBy: + description: InitiatedBy contains information about who initiated + the operations + properties: + automated: + description: Automated is set to true if operation was initiated + automatically by the application controller. + type: boolean + username: + description: Username contains the name of a user who started + operation + type: string + type: object + retry: + description: Retry controls the strategy to apply if a sync fails + properties: + backoff: + description: Backoff controls how to backoff on subsequent retries + of failed syncs + properties: + duration: + description: Duration is the amount to back off. Default unit + is seconds, but could also be a duration (e.g. "2m", "1h") + type: string + factor: + description: Factor is a factor to multiply the base duration + after each failed retry + format: int64 + type: integer + maxDuration: + description: MaxDuration is the maximum amount of time allowed + for the backoff strategy + type: string + type: object + limit: + description: Limit is the maximum number of attempts for retrying + a failed sync. If set to 0, no retries will be performed. + format: int64 + type: integer + type: object + sync: + description: Sync contains parameters for the operation + properties: + dryRun: + description: DryRun specifies to perform a `kubectl apply --dry-run` + without actually performing the sync + type: boolean + manifests: + description: Manifests is an optional field that overrides sync + source with a local directory for development + items: + type: string + type: array + prune: + description: Prune specifies to delete resources from the cluster + that are no longer tracked in git + type: boolean + resources: + description: Resources describes which resources shall be part + of the sync + items: + description: SyncOperationResource contains resources to sync. + properties: + group: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + type: array + revision: + description: Revision is the revision (Git) or chart version (Helm) + which to sync the application to If omitted, will use the revision + specified in app spec. + type: string + source: + description: Source overrides the source definition set in the + application. This is typically set in a Rollback operation and + is nil during a Sync operation + properties: + chart: + description: Chart is a Helm chart name, and must be specified + for applications sourced from a Helm repo. + type: string + directory: + description: Directory holds path/directory specific options + properties: + exclude: + description: Exclude contains a glob pattern to match + paths against that should be explicitly excluded from + being used during manifest generation + type: string + include: + description: Include contains a glob pattern to match + paths against that should be explicitly included during + manifest generation + type: string + jsonnet: + description: Jsonnet holds options specific to Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet External + Variables + items: + description: JsonnetVar represents a variable to + be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level Arguments + items: + description: JsonnetVar represents a variable to + be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan a directory + recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters to the + helm template + items: + description: HelmFileParameter is a file parameter that's + passed to helm template during manifest generation + properties: + name: + description: Name is the name of the Helm parameter + type: string + path: + description: Path is the path to the file containing + the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters which + are passed to the helm template command upon manifest + generation + items: + description: HelmParameter is a parameter that's passed + to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether to tell + Helm to interpret booleans and numbers as strings + type: boolean + name: + description: Name is the name of the Helm parameter + type: string + value: + description: Value is the value for the Helm parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name to use. + If omitted it will use the application name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value files + to use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be passed + to helm template, typically defined as a block + type: string + version: + description: Version is the Helm version to use for templating + (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application environment + name + type: string + parameters: + description: Parameters are a list of ksonnet component + parameter override values + items: + description: KsonnetParameter is a ksonnet component + parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional + annotations to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional labels + to add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image override + specifications + items: + description: KustomizeImage represents a Kustomize image + definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to resources + for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to resources + for Kustomize apps + type: string + version: + description: Version controls which version of Kustomize + to use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git repository, + and is only valid for applications sourced from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management + plugin specific options + properties: + env: + description: Env is a list of environment variable entries + items: + description: EnvEntry represents an entry in the application's + environment + properties: + name: + description: Name is the name of the variable, usually + expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository (Git or + Helm) that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of the source + to sync the application to. In case of Git, this can be + commit, tag, or branch. If omitted, will equal to HEAD. + In case of Helm, this is a semver tag for the Chart's version. + type: string + required: + - repoURL + type: object + syncOptions: + description: SyncOptions provide per-sync sync-options, e.g. Validate=false + items: + type: string + type: array + syncStrategy: + description: SyncStrategy describes how to perform the sync + properties: + apply: + description: Apply will perform a `kubectl apply` to perform + the sync. + properties: + force: + description: Force indicates whether or not to supply + the --force flag to `kubectl apply`. The --force flag + deletes and re-create the resource, when PATCH encounters + conflict and has retried for 5 times. + type: boolean + type: object + hook: + description: Hook will submit any referenced resources to + perform the sync. This is the default strategy + properties: + force: + description: Force indicates whether or not to supply + the --force flag to `kubectl apply`. The --force flag + deletes and re-create the resource, when PATCH encounters + conflict and has retried for 5 times. + type: boolean + type: object + type: object + type: object + type: object + spec: + description: ApplicationSpec represents desired application state. Contains + link to repository with application definition and additional parameters + link definition revision. + properties: + destination: + description: Destination is a reference to the target Kubernetes server + and namespace + properties: + name: + description: Name is an alternate way of specifying the target + cluster by its symbolic name + type: string + namespace: + description: Namespace specifies the target namespace for the + application's resources. The namespace will only be set for + namespace-scoped resources that have not set a value for .metadata.namespace + type: string + server: + description: Server specifies the URL of the target cluster and + must be set to the Kubernetes control plane API + type: string + type: object + ignoreDifferences: + description: IgnoreDifferences is a list of resources and their fields + which should be ignored during comparison + items: + description: ResourceIgnoreDifferences contains resource filter + and list of json paths which should be ignored during comparison + with live state. + properties: + group: + type: string + jsonPointers: + items: + type: string + type: array + kind: + type: string + name: + type: string + namespace: + type: string + required: + - jsonPointers + - kind + type: object + type: array + info: + description: Info contains a list of information (URLs, email addresses, + and plain text) that relates to the application + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + project: + description: Project is a reference to the project this application + belongs to. The empty string means that application belongs to the + 'default' project. + type: string + revisionHistoryLimit: + description: RevisionHistoryLimit limits the number of items kept + in the application's revision history, which is used for informational + purposes as well as for rollbacks to previous versions. This should + only be changed in exceptional circumstances. Setting to zero will + store no history. This will reduce storage used. Increasing will + increase the space used to store the history, so we do not recommend + increasing it. Default is 10. + format: int64 + type: integer + source: + description: Source is a reference to the location of the application's + manifests or chart + properties: + chart: + description: Chart is a Helm chart name, and must be specified + for applications sourced from a Helm repo. + type: string + directory: + description: Directory holds path/directory specific options + properties: + exclude: + description: Exclude contains a glob pattern to match paths + against that should be explicitly excluded from being used + during manifest generation + type: string + include: + description: Include contains a glob pattern to match paths + against that should be explicitly included during manifest + generation + type: string + jsonnet: + description: Jsonnet holds options specific to Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet External Variables + items: + description: JsonnetVar represents a variable to be + passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level Arguments + items: + description: JsonnetVar represents a variable to be + passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan a directory + recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters to the helm + template + items: + description: HelmFileParameter is a file parameter that's + passed to helm template during manifest generation + properties: + name: + description: Name is the name of the Helm parameter + type: string + path: + description: Path is the path to the file containing + the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters which + are passed to the helm template command upon manifest generation + items: + description: HelmParameter is a parameter that's passed + to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether to tell + Helm to interpret booleans and numbers as strings + type: boolean + name: + description: Name is the name of the Helm parameter + type: string + value: + description: Value is the value for the Helm parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name to use. + If omitted it will use the application name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value files to + use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be passed to + helm template, typically defined as a block + type: string + version: + description: Version is the Helm version to use for templating + (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application environment + name + type: string + parameters: + description: Parameters are a list of ksonnet component parameter + override values + items: + description: KsonnetParameter is a ksonnet component parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional annotations + to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional labels to + add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image override + specifications + items: + description: KustomizeImage represents a Kustomize image + definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to resources + for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to resources + for Kustomize apps + type: string + version: + description: Version controls which version of Kustomize to + use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git repository, + and is only valid for applications sourced from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management plugin + specific options + properties: + env: + description: Env is a list of environment variable entries + items: + description: EnvEntry represents an entry in the application's + environment + properties: + name: + description: Name is the name of the variable, usually + expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository (Git or Helm) + that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of the source + to sync the application to. In case of Git, this can be commit, + tag, or branch. If omitted, will equal to HEAD. In case of Helm, + this is a semver tag for the Chart's version. + type: string + required: + - repoURL + type: object + syncPolicy: + description: SyncPolicy controls when and how a sync will be performed + properties: + automated: + description: Automated will keep an application synced to the + target revision + properties: + allowEmpty: + description: 'AllowEmpty allows apps have zero live resources + (default: false)' + type: boolean + prune: + description: 'Prune specifies whether to delete resources + from the cluster that are not found in the sources anymore + as part of automated sync (default: false)' + type: boolean + selfHeal: + description: 'SelfHeal specifes whether to revert resources + back to their desired state upon modification in the cluster + (default: false)' + type: boolean + type: object + retry: + description: Retry controls failed sync retry behavior + properties: + backoff: + description: Backoff controls how to backoff on subsequent + retries of failed syncs + properties: + duration: + description: Duration is the amount to back off. Default + unit is seconds, but could also be a duration (e.g. + "2m", "1h") + type: string + factor: + description: Factor is a factor to multiply the base duration + after each failed retry + format: int64 + type: integer + maxDuration: + description: MaxDuration is the maximum amount of time + allowed for the backoff strategy + type: string + type: object + limit: + description: Limit is the maximum number of attempts for retrying + a failed sync. If set to 0, no retries will be performed. + format: int64 + type: integer + type: object + syncOptions: + description: Options allow you to specify whole app sync-options + items: + type: string + type: array + type: object + required: + - destination + - project + - source + type: object + status: + description: ApplicationStatus contains status information for the application + properties: + conditions: + description: Conditions is a list of currently observed application + conditions + items: + description: ApplicationCondition contains details about an application + condition, which is usally an error or warning + properties: + lastTransitionTime: + description: LastTransitionTime is the time the condition was + last observed + format: date-time + type: string + message: + description: Message contains human-readable message indicating + details about condition + type: string + type: + description: Type is an application condition type + type: string + required: + - message + - type + type: object + type: array + health: + description: Health contains information about the application's current + health status + properties: + message: + description: Message is a human-readable informational message + describing the health status + type: string + status: + description: Status holds the status code of the application or + resource + type: string + type: object + history: + description: History contains information about the application's + sync history + items: + description: RevisionHistory contains history information about + a previous sync + properties: + deployStartedAt: + description: DeployStartedAt holds the time the sync operation + started + format: date-time + type: string + deployedAt: + description: DeployedAt holds the time the sync operation completed + format: date-time + type: string + id: + description: ID is an auto incrementing identifier of the RevisionHistory + format: int64 + type: integer + revision: + description: Revision holds the revision the sync was performed + against + type: string + source: + description: Source is a reference to the application source + used for the sync operation + properties: + chart: + description: Chart is a Helm chart name, and must be specified + for applications sourced from a Helm repo. + type: string + directory: + description: Directory holds path/directory specific options + properties: + exclude: + description: Exclude contains a glob pattern to match + paths against that should be explicitly excluded from + being used during manifest generation + type: string + include: + description: Include contains a glob pattern to match + paths against that should be explicitly included during + manifest generation + type: string + jsonnet: + description: Jsonnet holds options specific to Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet External + Variables + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level + Arguments + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan a directory + recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters to the + helm template + items: + description: HelmFileParameter is a file parameter + that's passed to helm template during manifest generation + properties: + name: + description: Name is the name of the Helm parameter + type: string + path: + description: Path is the path to the file containing + the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters + which are passed to the helm template command upon + manifest generation + items: + description: HelmParameter is a parameter that's passed + to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether to + tell Helm to interpret booleans and numbers + as strings + type: boolean + name: + description: Name is the name of the Helm parameter + type: string + value: + description: Value is the value for the Helm parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name to + use. If omitted it will use the application name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value files + to use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be passed + to helm template, typically defined as a block + type: string + version: + description: Version is the Helm version to use for + templating (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application environment + name + type: string + parameters: + description: Parameters are a list of ksonnet component + parameter override values + items: + description: KsonnetParameter is a ksonnet component + parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional + annotations to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional labels + to add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image override + specifications + items: + description: KustomizeImage represents a Kustomize + image definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to resources + for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to resources + for Kustomize apps + type: string + version: + description: Version controls which version of Kustomize + to use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git repository, + and is only valid for applications sourced from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management + plugin specific options + properties: + env: + description: Env is a list of environment variable entries + items: + description: EnvEntry represents an entry in the application's + environment + properties: + name: + description: Name is the name of the variable, + usually expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository (Git or + Helm) that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of the + source to sync the application to. In case of Git, this + can be commit, tag, or branch. If omitted, will equal + to HEAD. In case of Helm, this is a semver tag for the + Chart's version. + type: string + required: + - repoURL + type: object + required: + - deployedAt + - id + - revision + type: object + type: array + observedAt: + description: 'ObservedAt indicates when the application state was + updated without querying latest git state Deprecated: controller + no longer updates ObservedAt field' + format: date-time + type: string + operationState: + description: OperationState contains information about any ongoing + operations, such as a sync + properties: + finishedAt: + description: FinishedAt contains time of operation completion + format: date-time + type: string + message: + description: Message holds any pertinent messages when attempting + to perform operation (typically errors). + type: string + operation: + description: Operation is the original requested operation + properties: + info: + description: Info is a list of informational items for this + operation + items: + properties: + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + initiatedBy: + description: InitiatedBy contains information about who initiated + the operations + properties: + automated: + description: Automated is set to true if operation was + initiated automatically by the application controller. + type: boolean + username: + description: Username contains the name of a user who + started operation + type: string + type: object + retry: + description: Retry controls the strategy to apply if a sync + fails + properties: + backoff: + description: Backoff controls how to backoff on subsequent + retries of failed syncs + properties: + duration: + description: Duration is the amount to back off. Default + unit is seconds, but could also be a duration (e.g. + "2m", "1h") + type: string + factor: + description: Factor is a factor to multiply the base + duration after each failed retry + format: int64 + type: integer + maxDuration: + description: MaxDuration is the maximum amount of + time allowed for the backoff strategy + type: string + type: object + limit: + description: Limit is the maximum number of attempts for + retrying a failed sync. If set to 0, no retries will + be performed. + format: int64 + type: integer + type: object + sync: + description: Sync contains parameters for the operation + properties: + dryRun: + description: DryRun specifies to perform a `kubectl apply + --dry-run` without actually performing the sync + type: boolean + manifests: + description: Manifests is an optional field that overrides + sync source with a local directory for development + items: + type: string + type: array + prune: + description: Prune specifies to delete resources from + the cluster that are no longer tracked in git + type: boolean + resources: + description: Resources describes which resources shall + be part of the sync + items: + description: SyncOperationResource contains resources + to sync. + properties: + group: + type: string + kind: + type: string + name: + type: string + namespace: + type: string + required: + - kind + - name + type: object + type: array + revision: + description: Revision is the revision (Git) or chart version + (Helm) which to sync the application to If omitted, + will use the revision specified in app spec. + type: string + source: + description: Source overrides the source definition set + in the application. This is typically set in a Rollback + operation and is nil during a Sync operation + properties: + chart: + description: Chart is a Helm chart name, and must + be specified for applications sourced from a Helm + repo. + type: string + directory: + description: Directory holds path/directory specific + options + properties: + exclude: + description: Exclude contains a glob pattern to + match paths against that should be explicitly + excluded from being used during manifest generation + type: string + include: + description: Include contains a glob pattern to + match paths against that should be explicitly + included during manifest generation + type: string + jsonnet: + description: Jsonnet holds options specific to + Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet + External Variables + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest + generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level + Arguments + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest + generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan + a directory recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters + to the helm template + items: + description: HelmFileParameter is a file parameter + that's passed to helm template during manifest + generation + properties: + name: + description: Name is the name of the Helm + parameter + type: string + path: + description: Path is the path to the file + containing the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters + which are passed to the helm template command + upon manifest generation + items: + description: HelmParameter is a parameter that's + passed to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether + to tell Helm to interpret booleans and + numbers as strings + type: boolean + name: + description: Name is the name of the Helm + parameter + type: string + value: + description: Value is the value for the + Helm parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name + to use. If omitted it will use the application + name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value + files to use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be + passed to helm template, typically defined as + a block + type: string + version: + description: Version is the Helm version to use + for templating (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application + environment name + type: string + parameters: + description: Parameters are a list of ksonnet + component parameter override values + items: + description: KsonnetParameter is a ksonnet component + parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional + annotations to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional + labels to add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image + override specifications + items: + description: KustomizeImage represents a Kustomize + image definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to + resources for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to + resources for Kustomize apps + type: string + version: + description: Version controls which version of + Kustomize to use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git + repository, and is only valid for applications sourced + from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management + plugin specific options + properties: + env: + description: Env is a list of environment variable + entries + items: + description: EnvEntry represents an entry in + the application's environment + properties: + name: + description: Name is the name of the variable, + usually expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository + (Git or Helm) that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of + the source to sync the application to. In case of + Git, this can be commit, tag, or branch. If omitted, + will equal to HEAD. In case of Helm, this is a semver + tag for the Chart's version. + type: string + required: + - repoURL + type: object + syncOptions: + description: SyncOptions provide per-sync sync-options, + e.g. Validate=false + items: + type: string + type: array + syncStrategy: + description: SyncStrategy describes how to perform the + sync + properties: + apply: + description: Apply will perform a `kubectl apply` + to perform the sync. + properties: + force: + description: Force indicates whether or not to + supply the --force flag to `kubectl apply`. + The --force flag deletes and re-create the resource, + when PATCH encounters conflict and has retried + for 5 times. + type: boolean + type: object + hook: + description: Hook will submit any referenced resources + to perform the sync. This is the default strategy + properties: + force: + description: Force indicates whether or not to + supply the --force flag to `kubectl apply`. + The --force flag deletes and re-create the resource, + when PATCH encounters conflict and has retried + for 5 times. + type: boolean + type: object + type: object + type: object + type: object + phase: + description: Phase is the current phase of the operation + type: string + retryCount: + description: RetryCount contains time of operation retries + format: int64 + type: integer + startedAt: + description: StartedAt contains time of operation start + format: date-time + type: string + syncResult: + description: SyncResult is the result of a Sync operation + properties: + resources: + description: Resources contains a list of sync result items + for each individual resource in a sync operation + items: + description: ResourceResult holds the operation result details + of a specific resource + properties: + group: + description: Group specifies the API group of the resource + type: string + hookPhase: + description: HookPhase contains the state of any operation + associated with this resource OR hook This can also + contain values for non-hook resources. + type: string + hookType: + description: HookType specifies the type of the hook. + Empty for non-hook resources + type: string + kind: + description: Kind specifies the API kind of the resource + type: string + message: + description: Message contains an informational or error + message for the last sync OR operation + type: string + name: + description: Name specifies the name of the resource + type: string + namespace: + description: Namespace specifies the target namespace + of the resource + type: string + status: + description: Status holds the final result of the sync. + Will be empty if the resources is yet to be applied/pruned + and is always zero-value for hooks + type: string + syncPhase: + description: SyncPhase indicates the particular phase + of the sync that this result was acquired in + type: string + version: + description: Version specifies the API version of the + resource + type: string + required: + - group + - kind + - name + - namespace + - version + type: object + type: array + revision: + description: Revision holds the revision this sync operation + was performed to + type: string + source: + description: Source records the application source information + of the sync, used for comparing auto-sync + properties: + chart: + description: Chart is a Helm chart name, and must be specified + for applications sourced from a Helm repo. + type: string + directory: + description: Directory holds path/directory specific options + properties: + exclude: + description: Exclude contains a glob pattern to match + paths against that should be explicitly excluded + from being used during manifest generation + type: string + include: + description: Include contains a glob pattern to match + paths against that should be explicitly included + during manifest generation + type: string + jsonnet: + description: Jsonnet holds options specific to Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet External + Variables + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level + Arguments + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan a directory + recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters to + the helm template + items: + description: HelmFileParameter is a file parameter + that's passed to helm template during manifest + generation + properties: + name: + description: Name is the name of the Helm parameter + type: string + path: + description: Path is the path to the file containing + the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters + which are passed to the helm template command upon + manifest generation + items: + description: HelmParameter is a parameter that's + passed to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether + to tell Helm to interpret booleans and numbers + as strings + type: boolean + name: + description: Name is the name of the Helm parameter + type: string + value: + description: Value is the value for the Helm + parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name + to use. If omitted it will use the application name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value files + to use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be passed + to helm template, typically defined as a block + type: string + version: + description: Version is the Helm version to use for + templating (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application + environment name + type: string + parameters: + description: Parameters are a list of ksonnet component + parameter override values + items: + description: KsonnetParameter is a ksonnet component + parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional + annotations to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional + labels to add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image override + specifications + items: + description: KustomizeImage represents a Kustomize + image definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to resources + for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to resources + for Kustomize apps + type: string + version: + description: Version controls which version of Kustomize + to use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git repository, + and is only valid for applications sourced from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management + plugin specific options + properties: + env: + description: Env is a list of environment variable + entries + items: + description: EnvEntry represents an entry in the + application's environment + properties: + name: + description: Name is the name of the variable, + usually expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository (Git + or Helm) that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of the + source to sync the application to. In case of Git, this + can be commit, tag, or branch. If omitted, will equal + to HEAD. In case of Helm, this is a semver tag for the + Chart's version. + type: string + required: + - repoURL + type: object + required: + - revision + type: object + required: + - operation + - phase + - startedAt + type: object + reconciledAt: + description: ReconciledAt indicates when the application state was + reconciled using the latest git version + format: date-time + type: string + resources: + description: Resources is a list of Kubernetes resources managed by + this application + items: + description: 'ResourceStatus holds the current sync and health status + of a resource TODO: describe members of this type' + properties: + group: + type: string + health: + description: HealthStatus contains information about the currently + observed health state of an application or resource + properties: + message: + description: Message is a human-readable informational message + describing the health status + type: string + status: + description: Status holds the status code of the application + or resource + type: string + type: object + hook: + type: boolean + kind: + type: string + name: + type: string + namespace: + type: string + requiresPruning: + type: boolean + status: + description: SyncStatusCode is a type which represents possible + comparison results + type: string + version: + type: string + type: object + type: array + sourceType: + description: SourceType specifies the type of this application + type: string + summary: + description: Summary contains a list of URLs and container images + used by this application + properties: + externalURLs: + description: ExternalURLs holds all external URLs of application + child resources. + items: + type: string + type: array + images: + description: Images holds all images of application child resources. + items: + type: string + type: array + type: object + sync: + description: Sync contains information about the application's current + sync status + properties: + comparedTo: + description: ComparedTo contains information about what has been + compared + properties: + destination: + description: Destination is a reference to the application's + destination used for comparison + properties: + name: + description: Name is an alternate way of specifying the + target cluster by its symbolic name + type: string + namespace: + description: Namespace specifies the target namespace + for the application's resources. The namespace will + only be set for namespace-scoped resources that have + not set a value for .metadata.namespace + type: string + server: + description: Server specifies the URL of the target cluster + and must be set to the Kubernetes control plane API + type: string + type: object + source: + description: Source is a reference to the application's source + used for comparison + properties: + chart: + description: Chart is a Helm chart name, and must be specified + for applications sourced from a Helm repo. + type: string + directory: + description: Directory holds path/directory specific options + properties: + exclude: + description: Exclude contains a glob pattern to match + paths against that should be explicitly excluded + from being used during manifest generation + type: string + include: + description: Include contains a glob pattern to match + paths against that should be explicitly included + during manifest generation + type: string + jsonnet: + description: Jsonnet holds options specific to Jsonnet + properties: + extVars: + description: ExtVars is a list of Jsonnet External + Variables + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + libs: + description: Additional library search dirs + items: + type: string + type: array + tlas: + description: TLAS is a list of Jsonnet Top-level + Arguments + items: + description: JsonnetVar represents a variable + to be passed to jsonnet during manifest generation + properties: + code: + type: boolean + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + recurse: + description: Recurse specifies whether to scan a directory + recursively for manifests + type: boolean + type: object + helm: + description: Helm holds helm specific options + properties: + fileParameters: + description: FileParameters are file parameters to + the helm template + items: + description: HelmFileParameter is a file parameter + that's passed to helm template during manifest + generation + properties: + name: + description: Name is the name of the Helm parameter + type: string + path: + description: Path is the path to the file containing + the values for the Helm parameter + type: string + type: object + type: array + parameters: + description: Parameters is a list of Helm parameters + which are passed to the helm template command upon + manifest generation + items: + description: HelmParameter is a parameter that's + passed to helm template during manifest generation + properties: + forceString: + description: ForceString determines whether + to tell Helm to interpret booleans and numbers + as strings + type: boolean + name: + description: Name is the name of the Helm parameter + type: string + value: + description: Value is the value for the Helm + parameter + type: string + type: object + type: array + releaseName: + description: ReleaseName is the Helm release name + to use. If omitted it will use the application name + type: string + valueFiles: + description: ValuesFiles is a list of Helm value files + to use when generating a template + items: + type: string + type: array + values: + description: Values specifies Helm values to be passed + to helm template, typically defined as a block + type: string + version: + description: Version is the Helm version to use for + templating (either "2" or "3") + type: string + type: object + ksonnet: + description: Ksonnet holds ksonnet specific options + properties: + environment: + description: Environment is a ksonnet application + environment name + type: string + parameters: + description: Parameters are a list of ksonnet component + parameter override values + items: + description: KsonnetParameter is a ksonnet component + parameter + properties: + component: + type: string + name: + type: string + value: + type: string + required: + - name + - value + type: object + type: array + type: object + kustomize: + description: Kustomize holds kustomize specific options + properties: + commonAnnotations: + additionalProperties: + type: string + description: CommonAnnotations is a list of additional + annotations to add to rendered manifests + type: object + commonLabels: + additionalProperties: + type: string + description: CommonLabels is a list of additional + labels to add to rendered manifests + type: object + images: + description: Images is a list of Kustomize image override + specifications + items: + description: KustomizeImage represents a Kustomize + image definition in the format [old_image_name=]: + type: string + type: array + namePrefix: + description: NamePrefix is a prefix appended to resources + for Kustomize apps + type: string + nameSuffix: + description: NameSuffix is a suffix appended to resources + for Kustomize apps + type: string + version: + description: Version controls which version of Kustomize + to use for rendering manifests + type: string + type: object + path: + description: Path is a directory path within the Git repository, + and is only valid for applications sourced from Git. + type: string + plugin: + description: ConfigManagementPlugin holds config management + plugin specific options + properties: + env: + description: Env is a list of environment variable + entries + items: + description: EnvEntry represents an entry in the + application's environment + properties: + name: + description: Name is the name of the variable, + usually expressed in uppercase + type: string + value: + description: Value is the value of the variable + type: string + required: + - name + - value + type: object + type: array + name: + type: string + type: object + repoURL: + description: RepoURL is the URL to the repository (Git + or Helm) that contains the application manifests + type: string + targetRevision: + description: TargetRevision defines the revision of the + source to sync the application to. In case of Git, this + can be commit, tag, or branch. If omitted, will equal + to HEAD. In case of Helm, this is a semver tag for the + Chart's version. + type: string + required: + - repoURL + type: object + required: + - destination + - source + type: object + revision: + description: Revision contains information about the revision + the comparison has been performed to + type: string + status: + description: Status is the sync state of the comparison + type: string + required: + - status + type: object + type: object + required: + - metadata + - spec + type: object + served: true + storage: true + subresources: {} diff --git a/common/install/templates/argocd/application.yaml b/common/install/templates/argocd/application.yaml new file mode 100644 index 0000000..a462ddd --- /dev/null +++ b/common/install/templates/argocd/application.yaml @@ -0,0 +1,38 @@ +{{- $valuesDirectoryURL := cat .Values.main.git.repoURL "/raw/" .Values.main.git.revision -}} +{{- $valuesDirectoryURLFixed := $valuesDirectoryURL | replace " " "" | replace ".git" "" }} +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: {{ .Release.Name }}-{{ .Values.main.clusterGroupName }} + namespace: openshift-gitops +spec: + destination: + name: in-cluster + namespace: {{ .Release.Name }}-{{ .Values.main.clusterGroupName }} + project: default + source: + repoURL: {{ .Values.main.git.repoURL }} + targetRevision: {{ .Values.main.git.revision }} + path: common/clustergroup + helm: + valueFiles: + - "{{ coalesce .Values.main.git.valuesDirectoryURL $valuesDirectoryURLFixed }}/values-global.yaml" + - "{{ coalesce .Values.main.git.valuesDirectoryURL $valuesDirectoryURLFixed }}/values-{{ .Values.main.clusterGroupName }}.yaml" + # Track the progress of https://github.com/argoproj/argo-cd/pull/6280 + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.valuesDirectoryURL + value: {{ coalesce .Values.main.git.valuesDirectoryURL $valuesDirectoryURLFixed }} + - name: global.pattern + value: {{ .Release.Name }} + - name: global.hubClusterDomain + value: {{ .Values.global.hubClusterDomain }} +{{- if eq .Values.main.options.syncPolicy "Automatic" }} + syncPolicy: + automated: {} +{{- end }} diff --git a/common/install/templates/argocd/namespace.yaml b/common/install/templates/argocd/namespace.yaml new file mode 100644 index 0000000..f852fe1 --- /dev/null +++ b/common/install/templates/argocd/namespace.yaml @@ -0,0 +1,8 @@ +# Pre-create so we can create our argo app for keeping subscriptions in sync +# Do it here so that we don't try to sync it in the future +{{- if .Values.main.options.bootstrap }} +apiVersion: v1 +kind: Namespace +metadata: + name: openshift-gitops +{{- end }} diff --git a/common/install/templates/argocd/subscription.yaml b/common/install/templates/argocd/subscription.yaml new file mode 100644 index 0000000..c1fb9a4 --- /dev/null +++ b/common/install/templates/argocd/subscription.yaml @@ -0,0 +1,22 @@ +{{- if .Values.main.options.bootstrap }} +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: "" +spec: + channel: {{ .Values.main.gitops.channel }} + installPlanApproval: {{ .Values.main.options.installPlanApproval }} + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: {{ .Release.Name }}-{{ .Values.main.clusterGroupName }},openshift-gitops +{{- if .Values.main.options.useCSV }} + startingCSV: openshift-gitops-operator.{{ .Values.main.gitops.csv }} +{{- end }} +{{- end }} diff --git a/common/install/values.yaml b/common/install/values.yaml new file mode 100644 index 0000000..05d5746 --- /dev/null +++ b/common/install/values.yaml @@ -0,0 +1,27 @@ +main: + git: + repoURL: https://github.com/pattern-clone/mypattern + revision: main + #valuesDirectoryURL: https://github.com/hybrid-cloud-patterns/industrial-edge/raw/main/ + + options: + syncPolicy: Automatic + installPlanApproval: Automatic + useCSV: False + bootstrap: True + + gitops: + channel: stable + source: redhat-operators + csv: v1.3.0 + + clusterGroupName: default + +global: + imageregistry: + type: quay + + git: + hostname: github.com + # Account is the user or organization under which the pattern repo lives + account: hybrid-cloud-patterns diff --git a/common/reference-output.yaml b/common/reference-output.yaml new file mode 100644 index 0000000..aa0e27e --- /dev/null +++ b/common/reference-output.yaml @@ -0,0 +1,119 @@ +--- +# Source: pattern-install/templates/argocd/namespace.yaml +# Pre-create so we can create our argo app for keeping subscriptions in sync +# Do it here so that we don't try to sync it in the future +apiVersion: v1 +kind: Namespace +metadata: + name: openshift-gitops +--- +# Source: pattern-install/templates/namespace.yaml +apiVersion: v1 +kind: Namespace +metadata: + name: manuela-ci + labels: + manuela-role: pipeline + app.kubernetes.io/instance: manuela +--- +# Source: pattern-install/templates/pipeline/serviceaccount.yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: pipeline + namespace: manuela-ci +secrets: +- name: git-repo-credentials +- name: image-registry-credentials +--- +# Source: pattern-install/templates/secrets/s3-secret.yaml +kind: Secret +apiVersion: v1 +metadata: + name: s3-secret +type: Opaque +data: + # Pre-create as part of the initial 'helm install' chart + # Create a file with the following: + # s3.accessKey: KEY + # s3.secretKey: secret key + #application.properties: base64 encrypted value of the above file + # This should live in the values-secret.yaml file + application.properties: BASE64STRING +--- +# Source: pattern-install/templates/secrets/secret-git-repo-credentials.yaml +apiVersion: v1 +kind: Secret +metadata: + name: git-repo-credentials + namespace: manuela-ci + annotations: + # Tekton magic, see https://tekton.dev/vault/pipelines-v0.15.2/auth/ + tekton.dev/git-0: https://github.com/hybrid-cloud-patterns +type: kubernetes.io/basic-auth +stringData: + username: STRING + password: STRING +--- +# Source: pattern-install/templates/secrets/secret-image-registry-credentials.yaml +apiVersion: v1 +kind: Secret +metadata: + name: openshift-registry-credentials + namespace: manuela-ci + annotations: + # Tekton magic, see https://tekton.dev/vault/pipelines-v0.15.2/auth/ + tekton.dev/docker-0: "https://" +type: kubernetes.io/basic-auth +stringData: + username: STRING + password: STRING +--- +# Source: pattern-install/templates/argocd/application.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: common-example + namespace: openshift-gitops +spec: + destination: + name: in-cluster + namespace: common-example + project: default + source: + repoURL: https://github.com/beekhof/common.git + targetRevision: main + path: common/clustergroup + helm: + valueFiles: + - "https://github.com/beekhof/patterns/raw/main/values-global.yaml" + - "https://github.com/beekhof/patterns/raw/main/values-example.yaml" + # Track the progress of https://github.com/argoproj/argo-cd/pull/6280 + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.valuesDirectoryURL + value: https://github.com/beekhof/patterns/raw/main + - name: global.pattern + value: common + syncPolicy: + automated: {} +--- +# Source: pattern-install/templates/argocd/subscription.yaml +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: "" +spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace diff --git a/common/scripts/ansible-push-vault-secrets.sh b/common/scripts/ansible-push-vault-secrets.sh new file mode 100755 index 0000000..e75d651 --- /dev/null +++ b/common/scripts/ansible-push-vault-secrets.sh @@ -0,0 +1,120 @@ +#!/usr/bin/env ansible-playbook +- name: Secret injection of validated-patterns + hosts: localhost + connection: local + gather_facts: no + vars: + values_secret: "{{ lookup('env', 'HOME') }}/values-secret.yaml" + kubeconfig: "{{ lookup('env', 'KUBECONFIG') }}" + vault_ns: "vault" + vault_pod: "vault-0" + vault_path: "secret/hub" + debug: False + + tasks: + - name: Check for existence of "{{ values_secret }}" + ansible.builtin.stat: + path: "{{ values_secret }}" + register: result + failed_when: not result.stat.exists + + - name: Check that KUBECONFIG is correctly set + fail: + msg: "KUBECONFIG is not set. Please set it so we can inject the secrets into the cluster's vault" + failed_when: kubeconfig is not defined or kubeconfig | length == 0 + when: not debug | bool + + - name: Parse "{{ values_secret }}" + ansible.builtin.set_fact: + all_values: "{{ lookup('file', values_secret) | from_yaml }}" + + - name: Set secrets fact + ansible.builtin.set_fact: + secrets: "{{ all_values['secrets'] }}" + + - name: Verify we have any secrets at all + ansible.builtin.fail: + msg: "Was not able to parse any secrets from file {{ values_secret }}: {{ all_values }}" + failed_when: + secrets is not defined or secrets | length == 0 + + # Detect here if we have only the following two keys under a password + # s3.accessKey: + # s3.secretKey: + # If we do, then detect it and calculate the b64 s3Secret token + # Note: the vars: line is due to https://github.com/ansible/ansible/issues/40239 + - name: Check if any of the passwords has only s3.[accessKey,secretKey] and generate the combined s3Secret in that case + ansible.builtin.set_fact: + s3keys: "{{ s3keys | default({}) | combine({ item.key: {'s3Secret': s3secret | b64encode } }) }}" + vars: + s3secret: "{{ 's3.accessKey: ' + item.value['s3.accessKey'] + '\ns3.secretKey: ' + item.value['s3.secretKey'] }}" + when: + - '"s3.accessKey" in item.value.keys()' + - '"s3.secretKey" in item.value.keys()' + - '"s3Secret" not in item.value.keys()' + loop: + "{{ secrets | dict2items }}" + loop_control: + label: "{{ item.key }}" + + - name: Merge any s3Secret into the secrets dictionary if we have any + ansible.builtin.set_fact: + secrets: "{{ secrets | combine(s3keys) }}" + when: + s3keys is defined and s3keys | length > 0 + + - name: Check for vault namespace + kubernetes.core.k8s_info: + kind: Namespace + name: "{{ vault_ns }}" + register: vault_ns_rc + failed_when: vault_ns_rc.resources | length == 0 + when: not debug | bool + + - name: Check if the vault pod is present + kubernetes.core.k8s_info: + kind: Pod + namespace: "{{ vault_ns }}" + name: "{{ vault_pod }}" + register: vault_pod_rc + failed_when: vault_pod_rc.resources | length == 0 + when: not debug | bool + + # vault status returns 1 on error and 2 on sealed + # so we can bail out when sealed + - name: Check if the vault is unsealed + kubernetes.core.k8s_exec: + namespace: "{{ vault_ns }}" + pod: "{{ vault_pod }}" + command: vault status + register: vault_status + failed_when: vault_status.rc|int == 1 + when: not debug | bool + + - name: Check vault status return + ansible.builtin.fail: + msg: The vault is still sealed. Please run "make vault-init" first with KUBECONFIG pointing to the HUB cluster + when: + - not debug | bool + - vault_status.rc | int > 0 + + - name: Debug + debug: + msg: "vault kv put {{ vault_path }}/{{ item.key }} -> {{ item.value.keys() | zip(item.value.values()) | map('join', '=') | list | join(' ')}}" + loop: + "{{ secrets | dict2items }}" + loop_control: + label: "{{ item.key }}" + when: debug | bool + + - name: Add the actual secrets to the vault + kubernetes.core.k8s_exec: + namespace: "{{ vault_ns }}" + pod: "{{ vault_pod }}" + command: | + sh -c "vault kv put {{ vault_path }}/{{ item.key }} {{ item.value.keys() | zip(item.value.values()) | map('join', '=') | list | join(' ')}}" + loop: + "{{ secrets | dict2items }}" + loop_control: + label: "{{ item.key }}" + when: not debug | bool diff --git a/common/scripts/make_common_subtree.sh b/common/scripts/make_common_subtree.sh new file mode 100755 index 0000000..a5e406d --- /dev/null +++ b/common/scripts/make_common_subtree.sh @@ -0,0 +1,76 @@ +#!/bin/sh + +if [ "$1" = "-h" ]; then + echo "This script will convert common into a subtree and add a remote to help manage it." + echo "The script takes three positional arguments, as follows:" + echo + echo "$0 " + echo + echo "Run without arguments, the script would run as if these arguments had been passed:" + echo "$0 https://github.com/hybrid-cloud-patterns/common.git main common-subtree" + echo + echo "Please ensure the git subtree command is available. On RHEL/Fedora, the git subtree command" + echo "is in a separate package called git-subtree" + exit 1 +fi + +if [ -f '/etc/redhat-release' ]; then + rpm -qa | grep git-subtree 2>&1 + if [ ! $? = 0 ]; then + echo "you need to install git-subtree" + echo "would you like to install it now?" + select ANS in yes no + do + case $ANS in + yes) + sudo dnf install git-subtree -y + break + ;; + no) + exit + break + ;; + *) + echo "You must enter yes or no" + ;; + esac + done + fi +fi + +if [ "$1" ]; then + subtree_repo=$1 +else + subtree_repo=https://github.com/hybrid-cloud-patterns/common.git +fi + +if [ "$2" ]; then + subtree_branch=$2 +else + subtree_branch=main +fi + +if [ "$3" ]; then + subtree_remote=$3 +else + subtree_remote=common-subtree +fi + +git diff --quiet || (echo "This script must be run on a clean working tree" && exit 1) + +echo "Changing directory to project root" +cd `git rev-parse --show-toplevel` + +echo "Removing existing common and replacing it with subtree from $subtree_repo $subtree_remote" +rm -rf common + +echo "Committing removal of common" +(git add -A :/ && git commit -m "Removed previous version of common to convert to subtree from $subtree_repo $subtree_branch") || exit 1 + +echo "Adding (possibly replacing) subtree remote $subtree_remote" +git remote rm "$subtree_remote" +git remote add -f "$subtree_remote" "$subtree_repo" || exit 1 +git subtree add --prefix=common "$subtree_remote" "$subtree_branch" || exit 1 + +echo "Complete. You may now push these results if you are satisfied" +exit 0 diff --git a/common/scripts/secret.sh b/common/scripts/secret.sh new file mode 100755 index 0000000..fc9bde8 --- /dev/null +++ b/common/scripts/secret.sh @@ -0,0 +1,52 @@ +#!/bin/sh + +#This script must receive arguments for the following parameters; it is easiest to pass them as environment variables from the command line. +#Example values from the first use for it, in industrial-edge +# +#This script is written to be POSIX-compliant, as not all sh are created equal - many are bash but +#Debian-derivatives use dash which doesn't support some bash syntax +# +#PATTERN=industrial-edge +#TARGET_NAMESPACE=manuela-ci +#COMPONENT=datacenter +#SECRET_NAME='argocd-env' + +passwd_resource="secrets/${COMPONENT}-gitops-cluster" +src_ns="${PATTERN}-${COMPONENT}" + +ns=0 +gitops=0 + +# Check for Namespaces and Secrets to be ready (it takes the cluster a few minutes to deploy them) +while [ 1 ] ; do + echo -n "Checking for namespace $TARGET_NAMESPACE to exist..." + if [ oc get namespace $TARGET_NAMESPACE >/dev/null 2>/dev/null ]; then + echo "not yet" + ns=0 + sleep 2 + continue + else + echo "OK" + ns=1 + fi + + echo -n "Checking for $passwd_resource to be populated in $src_ns..." + pw=`oc -n $src_ns extract $passwd_resource --to=- 2>/dev/null` + if [ "$?" = 0 ] && [ -n "$pw" ]; then + echo "OK" + gitops=1 + else + echo "not yet" + gitops=0 + sleep 2 + continue + fi + + echo "Conditions met, managing secret $SECRET_NAME in $TARGET_NAMESPACE" + break +done + +user=$(echo admin | base64) +password=$(echo $pw | base64) + +echo "{ \"apiVersion\": \"v1\", \"kind\": \"Secret\", \"metadata\": { \"name\": \"$SECRET_NAME\", \"namespace\": \"$TARGET_NAMESPACE\" }, \"data\": { \"ARGOCD_PASSWORD\": \"$password\", \"ARGOCD_USERNAME\": \"$user\" }, \"type\": \"Opaque\" }" | oc apply -f- diff --git a/common/scripts/test.sh b/common/scripts/test.sh new file mode 100755 index 0000000..6e400fc --- /dev/null +++ b/common/scripts/test.sh @@ -0,0 +1,39 @@ +#!/bin/bash +target=$1 +name=$(echo $1 | sed -e s@/@-@g -e s@charts-@@) +TEST_VARIANT="$2" +CHART_OPTS="$3" + +TESTDIR=tests +REFERENCE=${TESTDIR}/${name}-${TEST_VARIANT}.expected.yaml +OUTPUT=${TESTDIR}/.${name}-${TEST_VARIANT}.expected.yaml +#REFERENCE=${TESTDIR}/${name}.expected.yaml +#OUTPUT=${TESTDIR}/.${name}.expected.yaml + +echo "Testing $1 chart (${TEST_VARIANT})" >&2 +helm template $target --name-template $name ${CHART_OPTS} > ${OUTPUT} +#cp ${OUTPUT} ${REFERENCE} +if [ ! -e ${REFERENCE} ]; then + touch ${REFERENCE} +fi +diff -u ${REFERENCE} ${OUTPUT} +rc=$? +if [ $rc = 0 ]; then + rm -f ${OUTPUT} +fi + +if [ $TEST_VARIANT = normal -a $rc = 0 ]; then + # Another method of finding variables missing from values.yaml, eg. + # - name: -datacenter + # + name: pattern-name-datacenter + # Alas we can't make it fatal because there *should* be some differences + diff -u ${TESTDIR}/${name}-naked.expected.yaml ${TESTDIR}/${name}-normal.expected.yaml +fi + +if [ $rc = 0 ]; then + echo "PASS on $target $TEST_VARIANT with opts [$CHART_OPTS]" +else + echo "FAIL on $target $TEST_VARIANT with opts [$CHART_OPTS]" +fi + +exit $rc diff --git a/common/scripts/vault-token.sh b/common/scripts/vault-token.sh new file mode 100755 index 0000000..8a141fd --- /dev/null +++ b/common/scripts/vault-token.sh @@ -0,0 +1,64 @@ +#!/bin/sh + +#This script is written to be POSIX-compliant, as not all sh are created equal - many are bash but +#Debian-derivatives use dash which doesn't support some bash syntax +# +#TARGET_NAMESPACE=manuela-ci + +TARGET_NAMESPACE="external-secrets-deployable" +token_resource="vault" +src_ns="vault" + +ns=0 +ok=0 + +# Check for Namespaces and Secrets to be ready (it takes the cluster a few minutes to deploy them) +while [ 1 ] ; do + echo -n "Checking for namespace $TARGET_NAMESPACE to exist..." + if [ oc get namespace $TARGET_NAMESPACE >/dev/null 2>/dev/null ]; then + echo "not yet" + ns=0 + sleep 2 + continue + else + echo "OK" + ns=1 + fi + + echo -n "Checking for $token_resource to be populated in $src_ns..." + tok=`oc sa get-token -n $src_ns $token_resource 2>/dev/null` + if [ "$?" = 0 ] && [ -n "$tok" ]; then + echo "OK" + ok=1 + else + echo "not yet" + ok=0 + sleep 2 + continue + fi + + echo "Conditions met, managing sa token $token_resource in $TARGET_NAMESPACE" + break +done + +TOKEN=$(echo $tok | base64 -w 0) +cat << HERE | oc apply -f- +apiVersion: apps.open-cluster-management.io/v1 +kind: Deployable +metadata: + name: custom-kubernetes-token + namespace: external-secrets-deployable +spec: + channels: + - external-secrets-ns + template: + apiVersion: v1 + kind: Secret + metadata: + name: custom-kubernetes-token + data: + token: ${TOKEN} + type: Opaque +HERE + +#echo "{ \"apiVersion\": \"apps.open-cluster-management.io/v1\", \"kind\": \"Deployable\", \"metadata\": { \"name\": \"custom-kubernetes-token\", \"namespace\": \"external-secrets-deployable\" }, { spec: { \"channels\": \[\"data\": { \"ARGOCD_PASSWORD\": \"$password\", \"ARGOCD_USERNAME\": \"$user\" }, \"type\": \"Opaque\" }" | oc apply -f- diff --git a/common/scripts/vault-utils.sh b/common/scripts/vault-utils.sh new file mode 100755 index 0000000..df4980c --- /dev/null +++ b/common/scripts/vault-utils.sh @@ -0,0 +1,219 @@ +#!/bin/sh + +# Assumptions - vault in the demo will be running in non-HA mode so there will only be a vault-0 +# vault will be running in the "vault" namespace + +vault_delete() +{ + oc -n vault delete pod vault-0 & + oc -n vault delete pvc data-vault-0 & +} + +vault_ready_check() +{ +#NAME READY STATUS RESTARTS AGE +#vault-0 1/1 Running 0 44m + oc get po -n vault | grep vault-0 | awk '{ print $2, $3 }' 2>/dev/null +} + +get_vault_ready() +{ + rdy_output=`vault_ready_check` + + # Things we may have to wait for: + # being assigned to a host + if [ "$?" ]; then + sleep 5 + until [ "$rdy_output" ] + do + sleep 5 + rdy_output=`vault_ready_check` + done + fi + + printf "%s" "$rdy_output" +} + +vault_unseal() +{ + # Argument is expected to be the text output of the vault operator init command which includes Unseal Keys + # (5 by default) and a root token. + if [ -n "$1" ]; then + file=$1 + else + file=common/vault.init + fi + + for unseal in `cat $file | grep "Unseal Key" | awk '{ print $4 }'` + do + oc -n vault exec vault-0 -- vault operator unseal $unseal + done + + vault_login $file +} + +vault_init() +{ + # Argument is expected to be the text output of the vault operator init command which includes Unseal Keys + # (5 by default) and a root token. + if [ -n "$1" ]; then + file=$1 + else + file=common/vault.init + fi + + if [ -f "$file" ] && grep -q -e '^Unseal' "$file"; then + echo "$file already exists and contains seal secrets. If this is what you really wanted, move that file away first or call vault_delete" + exit 1 + fi + + # The vault is ready to be initialized when it is "Running" but not "ready". Unsealing it makes it ready + rdy_check=`get_vault_ready` + + if [ "$rdy_check" = "1/1 Running" ]; then + echo "Vault is already ready, exiting" + exit 0 + fi + + until [ "$rdy_check" = "0/1 Running" ] + do + echo $rdy_check + rdy_check=`get_vault_ready` + done + + oc -n vault exec vault-0 -- vault operator init | tee $file + + vault_unseal $file + vault_login $file + + vault_secrets_init $file + vault_kubernetes_init $file + vault_policy_init $file + + # Do not need pki init by default + # But this is how you could call it if you need it + #vault_pki_init $file +} + +# Retrieves the root token specified in the file $1 +vault_get_root_token() +{ + # Argument is expected to be the text output of the vault operator init command which includes Unseal Keys + # (5 by default) and a root token. + if [ -n "$1" ]; then + file=$1 + else + file=common/vault.init + fi + + token=`grep "Initial Root Token" $file | awk '{ print $4 }'` + printf "%s" "$token" +} + +# Exec a vault command wrapped with the vault root token specified in the file +# $1 +vault_token_exec() +{ + file="$1" + token=`vault_get_root_token $file` + shift + cmd="$@" + + vault_exec $file "VAULT_TOKEN=$token $cmd" +} + +vault_exec() +{ + file="$1" + token=`vault_get_root_token $file` + shift + cmd="$@" + + oc -n vault exec -i vault-0 -- sh -c "$cmd" +} + +vault_login() +{ + file="$1" + token=`vault_get_root_token $file` + shift + cmd="$@" + + vault_exec $file "vault login $token" +} + +oc_get_domain() +{ + oc get ingresses.config/cluster -o jsonpath={.spec.domain} +} + +oc_get_pki_domain() +{ + printf "%s" `oc_get_domain | cut -d. -f3-` +} + +oc_get_pki_role() +{ + pkidomain=`oc_get_pki_domain` + certrole=`printf "%s" "$pkidomain" | sed 's|\.|_|g'` + printf "%s" "$certrole" +} + +vault_pki_init() +{ + file="$1" + + pkidomain=`oc_get_pki_domain` + pkirole=`oc_get_pki_role` + + vault_exec $file "vault secrets enable pki" + vault_exec $file "vault secrets tune --max-lease-ttl=8760h pki" + vault_exec $file "vault write pki/root/generate/internal common_name=$pkidomain ttl=8760h" + vault_exec $file 'vault write pki/config/urls issuing_certificates="http://127.0.0.1:8200/v1/pki/ca" crl_distribution_points="http://127.0.0.1:8200/v1/pki/crl"' + vault_exec $file "vault write pki/roles/$pkirole allowed_domains=$pkidomain allow_subdomains=true max_ttl=8760h" +} + +vault_kubernetes_init() +{ + file="$1" + + vault_exec $file "vault auth enable --path=hub kubernetes" +} + +vault_policy_init() +{ + file="$1" + + k8s_host='https://$KUBERNETES_PORT_443_TCP_ADDR:443' + secret_name="$(oc get -n golang-external-secrets serviceaccount golang-external-secrets -o jsonpath='{.secrets}' | jq -r '.[] | select(.name | test ("golang-external-secrets-token-")).name')" + sa_token="$(oc get secret -n golang-external-secrets ${secret_name} -o go-template='{{ .data.token | base64decode }}')" + + vault_exec $file "vault write auth/hub/config token_reviewer_jwt=$sa_token kubernetes_host=$k8s_host kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt issuer=https://kubernetes.default.svc" + vault_exec $file 'vault policy write hub-secret - << EOF +path "secret/data/hub/*" + { capabilities = ["create", "read", "update", "delete", "list"] +} +EOF +' + vault_exec $file 'vault write auth/hub/role/hub-role bound_service_account_names="golang-external-secrets" bound_service_account_namespaces="golang-external-secrets" policies="default,hub-secret" ttl="15m"' +} + +vault_secrets_init() +{ + file="$1" + + vault_exec $file "vault secrets enable -path=secret kv-v2" +} + +vault_create_secret() +{ + file="$1" + shift + secret_path="$1" + shift + secret="$@" + + vault_exec $file "vault kv put $secret_path $secret" +} + +$@ diff --git a/common/tests/acm-naked.expected.yaml b/common/tests/acm-naked.expected.yaml new file mode 100644 index 0000000..2cd4858 --- /dev/null +++ b/common/tests/acm-naked.expected.yaml @@ -0,0 +1,95 @@ +--- +# Source: acm/templates/policies/application-policies.yaml +# TODO: Also create a GitOpsCluster.apps.open-cluster-management.io +--- +# Source: acm/templates/multiclusterhub.yaml +apiVersion: operator.open-cluster-management.io/v1 +kind: MultiClusterHub +metadata: + name: multiclusterhub + namespace: open-cluster-management + annotations: + argocd.argoproj.io/sync-wave: "-1" +spec: {} +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: policy.open-cluster-management.io/v1 +kind: PlacementBinding +metadata: + name: openshift-gitops-placement-binding + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +placementRef: + name: openshift-gitops-placement + kind: PlacementRule + apiGroup: apps.open-cluster-management.io +subjects: + - name: openshift-gitops-policy + kind: Policy + apiGroup: policy.open-cluster-management.io +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: openshift-gitops-placement +spec: + clusterConditions: + - status: 'True' + type: ManagedClusterConditionAvailable + clusterSelector: + matchExpressions: + - key: vendor + operator: In + values: + - OpenShift +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: policy.open-cluster-management.io/v1 +kind: Policy +metadata: + name: openshift-gitops-policy + annotations: + policy.open-cluster-management.io/standards: NIST-CSF + policy.open-cluster-management.io/categories: PR.DS Data Security + policy.open-cluster-management.io/controls: PR.DS-1 Data-at-rest + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + remediationAction: enforce + disabled: false + policy-templates: + - objectDefinition: + apiVersion: policy.open-cluster-management.io/v1 + kind: ConfigurationPolicy + metadata: + name: openshift-gitops-config + spec: + remediationAction: enforce + severity: med + namespaceSelector: + exclude: + - kube-* + include: + - default + object-templates: + - complianceType: musthave + objectDefinition: + # This is an auto-generated file. DO NOT EDIT + apiVersion: operators.coreos.com/v1alpha1 + kind: Subscription + metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: '' + spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: "*" diff --git a/common/tests/acm-normal.expected.yaml b/common/tests/acm-normal.expected.yaml new file mode 100644 index 0000000..0bbe76b --- /dev/null +++ b/common/tests/acm-normal.expected.yaml @@ -0,0 +1,209 @@ +--- +# Source: acm/templates/multiclusterhub.yaml +apiVersion: operator.open-cluster-management.io/v1 +kind: MultiClusterHub +metadata: + name: multiclusterhub + namespace: open-cluster-management + annotations: + argocd.argoproj.io/sync-wave: "-1" +spec: {} +--- +# Source: acm/templates/policies/application-policies.yaml +apiVersion: policy.open-cluster-management.io/v1 +kind: PlacementBinding +metadata: + name: edge-placement-binding + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +placementRef: + name: edge-placement + kind: PlacementRule + apiGroup: apps.open-cluster-management.io +subjects: + - name: edge-clustergroup-policy + kind: Policy + apiGroup: policy.open-cluster-management.io +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: policy.open-cluster-management.io/v1 +kind: PlacementBinding +metadata: + name: openshift-gitops-placement-binding + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true +placementRef: + name: openshift-gitops-placement + kind: PlacementRule + apiGroup: apps.open-cluster-management.io +subjects: + - name: openshift-gitops-policy + kind: Policy + apiGroup: policy.open-cluster-management.io +--- +# Source: acm/templates/policies/application-policies.yaml +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: edge-placement +spec: + clusterConditions: + - status: 'True' + type: ManagedClusterConditionAvailable + clusterSelector: { + "matchExpressions": [ + { + "key": "vendor", + "operator": "In", + "values": [ + "OpenShift" + ] + } + ] +} +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: openshift-gitops-placement +spec: + clusterConditions: + - status: 'True' + type: ManagedClusterConditionAvailable + clusterSelector: + matchExpressions: + - key: vendor + operator: In + values: + - OpenShift +--- +# Source: acm/templates/policies/application-policies.yaml +# TODO: Also create a GitOpsCluster.apps.open-cluster-management.io +apiVersion: policy.open-cluster-management.io/v1 +kind: Policy +metadata: + name: edge-clustergroup-policy + annotations: + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + remediationAction: enforce + disabled: false + policy-templates: + - objectDefinition: + apiVersion: policy.open-cluster-management.io/v1 + kind: ConfigurationPolicy + metadata: + name: edge-clustergroup-config + spec: + remediationAction: enforce + severity: med + namespaceSelector: + exclude: + - kube-* + include: + - default + object-templates: + - complianceType: musthave + objectDefinition: + apiVersion: argoproj.io/v1alpha1 + kind: Application + metadata: + name: mypattern-edge + namespace: openshift-gitops + finalizers: + - argoproj.io/finalizer + spec: + project: default + source: + repoURL: https://github.com/pattern-clone/mypattern + targetRevision: main + path: common/clustergroup + helm: + valueFiles: + - "https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml" + - "https://github.com/pattern-clone/mypattern/raw/main/values-edge.yaml" + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.pattern + value: mypattern + - name: global.valuesDirectoryURL + value: https://github.com/pattern-clone/mypattern/raw/main + - name: global.hubClusterDomain + value: hub.example.com + - name: global.localClusterDomain + value: '{{ (lookup "config.openshift.io/v1" "Ingress" "" "cluster").spec.domain }}' + - name: clusterGroup.isHubCluster + value: "false" + destination: + server: https://kubernetes.default.svc + namespace: mypattern-edge + syncPolicy: + automated: + prune: false + selfHeal: true + ignoreDifferences: + - group: apps + kind: Deployment + jsonPointers: + - /spec/replicas + - group: route.openshift.io + kind: Route + jsonPointers: + - /status +--- +# Source: acm/templates/policies/ocp-gitops-policy.yaml +apiVersion: policy.open-cluster-management.io/v1 +kind: Policy +metadata: + name: openshift-gitops-policy + annotations: + policy.open-cluster-management.io/standards: NIST-CSF + policy.open-cluster-management.io/categories: PR.DS Data Security + policy.open-cluster-management.io/controls: PR.DS-1 Data-at-rest + argocd.argoproj.io/sync-options: SkipDryRunOnMissingResource=true + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + remediationAction: enforce + disabled: false + policy-templates: + - objectDefinition: + apiVersion: policy.open-cluster-management.io/v1 + kind: ConfigurationPolicy + metadata: + name: openshift-gitops-config + spec: + remediationAction: enforce + severity: med + namespaceSelector: + exclude: + - kube-* + include: + - default + object-templates: + - complianceType: musthave + objectDefinition: + # This is an auto-generated file. DO NOT EDIT + apiVersion: operators.coreos.com/v1alpha1 + kind: Subscription + metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: '' + spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: "*" diff --git a/common/tests/clustergroup-naked.expected.yaml b/common/tests/clustergroup-naked.expected.yaml new file mode 100644 index 0000000..6245b70 --- /dev/null +++ b/common/tests/clustergroup-naked.expected.yaml @@ -0,0 +1,152 @@ +--- +# Source: pattern-clustergroup/templates/gitops-namespace.yaml +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: common-example + # The name here needs to be consistent with + # - acm/templates/policies/application-policies.yaml + # - clustergroup/templates/applications.yaml + # - any references to secrets and route URLs in documentation + name: common-example +spec: {} +--- +# Source: pattern-clustergroup/templates/argocd-super-role.yaml +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: openshift-gitops-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + name: openshift-gitops-argocd-application-controller + namespace: openshift-gitops + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + name: openshift-gitops-argocd-server + namespace: openshift-gitops +--- +# Source: pattern-clustergroup/templates/argocd-super-role.yaml +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: common-example-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-application-controller + name: example-gitops-argocd-application-controller + namespace: common-example + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-server + name: example-gitops-argocd-server + namespace: common-example +--- +# Source: pattern-clustergroup/templates/argocd.yaml +apiVersion: argoproj.io/v1alpha1 +kind: ArgoCD +metadata: + finalizers: + - argoproj.io/finalizer + # Changing the name affects the ClusterRoleBinding, the generated secret, + # route URL, and argocd.argoproj.io/managed-by annotations + name: example-gitops + namespace: common-example + annotations: + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + applicationInstanceLabelKey: argocd.argoproj.io/instance + # Not the greatest way to pass git/quay info to sub-applications, but it will do until + # we can support helmChart with kustomize + # The other option is to pass them in as environment variables eg. BLUEPRINT_VERSION + configManagementPlugins: "- name: kustomize-version\n generate:\n command: [sh, + -c]\n args: [\"kustomize version 1>&2 && exit 1\"]\n- name: kustomize-with-helm\n + \ generate:\n command: [\"kustomize\"]\n args: [\"build\", \"--enable-helm\"]\n- name: + helm-with-kustomize\n init:\n command: [\"/bin/sh\", \"-c\"]\n args: [\"helm + dependency build\"]\n generate:\n command: [/bin/bash, -c]\n args: [\"helm template . --name-template ${ARGOCD_APP_NAME:0:52} + -f https://github.com/pattern-clone/common/raw/main/values-global.yaml + -f https://github.com/pattern-clone/common/raw/main/values-example.yaml + --set global.repoURL=$ARGOCD_APP_SOURCE_REPO_URL + --set global.targetRevision=$ARGOCD_APP_SOURCE_TARGET_REVISION + --set global.namespace=$ARGOCD_APP_NAMESPACE + --set global.pattern=common + --set global.hubClusterDomain= + --set global.localClusterDomain= + --set global.valuesDirectoryURL=https://github.com/pattern-clone/common/raw/main + --post-renderer ./kustomize\"] + \ \n" + applicationSet: + resources: + limits: + cpu: "2" + memory: 1Gi + requests: + cpu: 250m + memory: 512Mi + controller: + processors: {} + resources: + limits: + cpu: "4" + memory: 4Gi + requests: + cpu: 500m + memory: 2Gi + dex: + openShiftOAuth: true + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 250m + memory: 128Mi + initialSSHKnownHosts: {} + rbac: + defaultPolicy: role:admin + repo: + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + resourceExclusions: | + - apiGroups: + - tekton.dev + kinds: + - TaskRun + - PipelineRun + server: + autoscale: + enabled: false + grpc: + ingress: + enabled: false + ingress: + enabled: false + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 125m + memory: 128Mi + route: + enabled: true + service: + type: "" + tls: + ca: {} +status: diff --git a/common/tests/clustergroup-normal.expected.yaml b/common/tests/clustergroup-normal.expected.yaml new file mode 100644 index 0000000..25983b1 --- /dev/null +++ b/common/tests/clustergroup-normal.expected.yaml @@ -0,0 +1,326 @@ +--- +# Source: pattern-clustergroup/templates/gitops-namespace.yaml +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: mypattern-example + # The name here needs to be consistent with + # - acm/templates/policies/application-policies.yaml + # - clustergroup/templates/applications.yaml + # - any references to secrets and route URLs in documentation + name: mypattern-example +spec: {} +--- +# Source: pattern-clustergroup/templates/namespaces.yaml +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: pattern + argocd.argoproj.io/managed-by: mypattern-example + name: open-cluster-management +spec: +--- +# Source: pattern-clustergroup/templates/namespaces.yaml +apiVersion: v1 +kind: Namespace +metadata: + labels: + name: pattern + argocd.argoproj.io/managed-by: mypattern-example + name: application-ci +spec: +--- +# Source: pattern-clustergroup/templates/argocd-super-role.yaml +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: openshift-gitops-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + name: openshift-gitops-argocd-application-controller + namespace: openshift-gitops + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + name: openshift-gitops-argocd-server + namespace: openshift-gitops +--- +# Source: pattern-clustergroup/templates/argocd-super-role.yaml +# WARNING: ONLY USE THIS FOR MANAGING CLUSTERS NOT FOR REGULAR USERS +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: mypattern-example-cluster-admin-rolebinding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-application-controller + name: example-gitops-argocd-application-controller + namespace: mypattern-example + # NOTE: THIS MUST BE FIXED FOR MULTITENANT SETUP + - kind: ServiceAccount + # This is the {ArgoCD.name}-argocd-server + name: example-gitops-argocd-server + namespace: mypattern-example +--- +# Source: pattern-clustergroup/templates/projects.yaml +apiVersion: argoproj.io/v1alpha1 +kind: AppProject +metadata: + name: datacenter + namespace: mypattern-example +spec: + description: "Pattern datacenter" + destinations: + - namespace: '*' + server: '*' + clusterResourceWhitelist: + - group: '*' + kind: '*' + namespaceResourceWhitelist: + - group: '*' + kind: '*' + sourceRepos: + - '*' +status: {} +--- +# Source: pattern-clustergroup/templates/applications.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: acm + namespace: mypattern-example +spec: + destination: + name: in-cluster + namespace: open-cluster-management + project: datacenter + source: + repoURL: https://github.com/pattern-clone/mypattern + targetRevision: + path: common/acm + helm: + valueFiles: + - "https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml" + - "https://github.com/pattern-clone/mypattern/raw/main/values-example.yaml" + # Watch the progress of https://issues.redhat.com/browse/GITOPS-891 and update accordingly + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.pattern + value: mypattern + - name: global.valuesDirectoryURL + value: https://github.com/pattern-clone/mypattern/raw/main + - name: global.hubClusterDomain + value: hub.example.com + - name: global.localClusterDomain + value: region.example.com + ignoreDifferences: [ + { + "group": "internal.open-cluster-management.io", + "jsonPointers": [ + "/spec/loggingCA" + ], + "kind": "ManagedClusterInfo" + } +] + syncPolicy: + automated: {} + # selfHeal: true +--- +# Source: pattern-clustergroup/templates/applications.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: pipelines + namespace: mypattern-example +spec: + destination: + name: in-cluster + namespace: application-ci + project: datacenter + source: + repoURL: https://github.com/pattern-clone/mypattern + targetRevision: + path: charts/datacenter/pipelines + helm: + valueFiles: + - "https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml" + - "https://github.com/pattern-clone/mypattern/raw/main/values-example.yaml" + # Watch the progress of https://issues.redhat.com/browse/GITOPS-891 and update accordingly + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.pattern + value: mypattern + - name: global.valuesDirectoryURL + value: https://github.com/pattern-clone/mypattern/raw/main + - name: global.hubClusterDomain + value: hub.example.com + - name: global.localClusterDomain + value: region.example.com + syncPolicy: + automated: {} + # selfHeal: true +--- +# Source: pattern-clustergroup/templates/argocd.yaml +apiVersion: argoproj.io/v1alpha1 +kind: ArgoCD +metadata: + finalizers: + - argoproj.io/finalizer + # Changing the name affects the ClusterRoleBinding, the generated secret, + # route URL, and argocd.argoproj.io/managed-by annotations + name: example-gitops + namespace: mypattern-example + annotations: + argocd.argoproj.io/compare-options: IgnoreExtraneous +spec: + applicationInstanceLabelKey: argocd.argoproj.io/instance + # Not the greatest way to pass git/quay info to sub-applications, but it will do until + # we can support helmChart with kustomize + # The other option is to pass them in as environment variables eg. BLUEPRINT_VERSION + configManagementPlugins: "- name: kustomize-version\n generate:\n command: [sh, + -c]\n args: [\"kustomize version 1>&2 && exit 1\"]\n- name: kustomize-with-helm\n + \ generate:\n command: [\"kustomize\"]\n args: [\"build\", \"--enable-helm\"]\n- name: + helm-with-kustomize\n init:\n command: [\"/bin/sh\", \"-c\"]\n args: [\"helm + dependency build\"]\n generate:\n command: [/bin/bash, -c]\n args: [\"helm template . --name-template ${ARGOCD_APP_NAME:0:52} + -f https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml + -f https://github.com/pattern-clone/mypattern/raw/main/values-example.yaml + --set global.repoURL=$ARGOCD_APP_SOURCE_REPO_URL + --set global.targetRevision=$ARGOCD_APP_SOURCE_TARGET_REVISION + --set global.namespace=$ARGOCD_APP_NAMESPACE + --set global.pattern=mypattern + --set global.hubClusterDomain=hub.example.com + --set global.localClusterDomain=region.example.com + --set global.valuesDirectoryURL=https://github.com/pattern-clone/mypattern/raw/main + --post-renderer ./kustomize\"] + \ \n" + applicationSet: + resources: + limits: + cpu: "2" + memory: 1Gi + requests: + cpu: 250m + memory: 512Mi + controller: + processors: {} + resources: + limits: + cpu: "4" + memory: 4Gi + requests: + cpu: 500m + memory: 2Gi + dex: + openShiftOAuth: true + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 250m + memory: 128Mi + initialSSHKnownHosts: {} + rbac: + defaultPolicy: role:admin + repo: + resources: + limits: + cpu: "1" + memory: 512Mi + requests: + cpu: 250m + memory: 256Mi + resourceExclusions: | + - apiGroups: + - tekton.dev + kinds: + - TaskRun + - PipelineRun + server: + autoscale: + enabled: false + grpc: + ingress: + enabled: false + ingress: + enabled: false + resources: + limits: + cpu: 500m + memory: 256Mi + requests: + cpu: 125m + memory: 128Mi + route: + enabled: true + service: + type: "" + tls: + ca: {} +status: +--- +# Source: pattern-clustergroup/templates/operatorgroup.yaml +apiVersion: operators.coreos.com/v1 +kind: OperatorGroup +metadata: + name: open-cluster-management-operator-group + namespace: open-cluster-management +spec: + targetNamespaces: + - open-cluster-management +--- +# Source: pattern-clustergroup/templates/operatorgroup.yaml +apiVersion: operators.coreos.com/v1 +kind: OperatorGroup +metadata: + name: application-ci-operator-group + namespace: application-ci +spec: + targetNamespaces: + - application-ci +--- +# Source: pattern-clustergroup/templates/subscriptions.yaml +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: advanced-cluster-management + namespace: open-cluster-management +spec: + name: advanced-cluster-management + source: redhat-operators + sourceNamespace: openshift-marketplace + channel: release-2.3 + installPlanApproval: Automatic +--- +# Source: pattern-clustergroup/templates/subscriptions.yaml +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openshift-pipelines-operator-rh + namespace: openshift-operators +spec: + name: openshift-pipelines-operator-rh + source: redhat-operators + sourceNamespace: openshift-marketplace + channel: stable + installPlanApproval: Automatic diff --git a/common/tests/install-naked.expected.yaml b/common/tests/install-naked.expected.yaml new file mode 100644 index 0000000..4ed9b44 --- /dev/null +++ b/common/tests/install-naked.expected.yaml @@ -0,0 +1,63 @@ +--- +# Source: pattern-install/templates/argocd/namespace.yaml +# Pre-create so we can create our argo app for keeping subscriptions in sync +# Do it here so that we don't try to sync it in the future +apiVersion: v1 +kind: Namespace +metadata: + name: openshift-gitops +--- +# Source: pattern-install/templates/argocd/application.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: install-default + namespace: openshift-gitops +spec: + destination: + name: in-cluster + namespace: install-default + project: default + source: + repoURL: https://github.com/pattern-clone/mypattern + targetRevision: main + path: common/clustergroup + helm: + valueFiles: + - "https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml" + - "https://github.com/pattern-clone/mypattern/raw/main/values-default.yaml" + # Track the progress of https://github.com/argoproj/argo-cd/pull/6280 + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.valuesDirectoryURL + value: https://github.com/pattern-clone/mypattern/raw/main + - name: global.pattern + value: install + - name: global.hubClusterDomain + value: + syncPolicy: + automated: {} +--- +# Source: pattern-install/templates/argocd/subscription.yaml +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: "" +spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: install-default,openshift-gitops diff --git a/common/tests/install-normal.expected.yaml b/common/tests/install-normal.expected.yaml new file mode 100644 index 0000000..42c6541 --- /dev/null +++ b/common/tests/install-normal.expected.yaml @@ -0,0 +1,63 @@ +--- +# Source: pattern-install/templates/argocd/namespace.yaml +# Pre-create so we can create our argo app for keeping subscriptions in sync +# Do it here so that we don't try to sync it in the future +apiVersion: v1 +kind: Namespace +metadata: + name: openshift-gitops +--- +# Source: pattern-install/templates/argocd/application.yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: install-example + namespace: openshift-gitops +spec: + destination: + name: in-cluster + namespace: install-example + project: default + source: + repoURL: https://github.com/pattern-clone/mypattern + targetRevision: main + path: common/clustergroup + helm: + valueFiles: + - "https://github.com/pattern-clone/mypattern/raw/main/values-global.yaml" + - "https://github.com/pattern-clone/mypattern/raw/main/values-example.yaml" + # Track the progress of https://github.com/argoproj/argo-cd/pull/6280 + parameters: + - name: global.repoURL + value: $ARGOCD_APP_SOURCE_REPO_URL + - name: global.targetRevision + value: $ARGOCD_APP_SOURCE_TARGET_REVISION + - name: global.namespace + value: $ARGOCD_APP_NAMESPACE + - name: global.valuesDirectoryURL + value: https://github.com/pattern-clone/mypattern/raw/main + - name: global.pattern + value: install + - name: global.hubClusterDomain + value: hub.example.com + syncPolicy: + automated: {} +--- +# Source: pattern-install/templates/argocd/subscription.yaml +apiVersion: operators.coreos.com/v1alpha1 +kind: Subscription +metadata: + name: openshift-gitops-operator + namespace: openshift-operators + labels: + operators.coreos.com/openshift-gitops-operator.openshift-operators: "" +spec: + channel: stable + installPlanApproval: Automatic + name: openshift-gitops-operator + source: redhat-operators + sourceNamespace: openshift-marketplace + config: + env: + - name: ARGOCD_CLUSTER_CONFIG_NAMESPACES + value: install-example,openshift-gitops diff --git a/common/values-global.yaml b/common/values-global.yaml new file mode 100644 index 0000000..dcda060 --- /dev/null +++ b/common/values-global.yaml @@ -0,0 +1,16 @@ +global: + options: + useCSV: True + syncPolicy: Manual + installPlanApproval: Automatic + + git: + hostname: github.com + # Account is the user or organization under which the pattern repos lives + account: hybrid-cloud-patterns + email: someone@somewhere.com + dev_revision: main + +main: + clusterGroupName: example + diff --git a/scripts/make_common_subtree.sh b/scripts/make_common_subtree.sh new file mode 100755 index 0000000..4a12558 --- /dev/null +++ b/scripts/make_common_subtree.sh @@ -0,0 +1,77 @@ +#!/bin/sh + +if [ "$1" = "-h" ]; then + echo "This script will convert common into a subtree and add a remote to help manage it." + echo "The script takes three positional arguments, as follows:" + echo + echo "$0 " + echo + echo "Run without arguments, the script would run as if these arguments had been passed:" + echo "$0 https://github.com/hybrid-cloud-patterns/common.git main common-subtree" + echo + echo "Please ensure the git subtree command is available. On RHEL/Fedora, the git subtree command" + echo "is in a separate package called git-subtree" + exit 1 +fi + +OS=$(uname -o) +if [ -f '/etc/redhat-release' ]; then + rpm -qa | grep git-subtree 2>&1 + if [ ! $? = 0 ]; then + echo "you need to install git-subtree" + echo "would you like to install it now?" + select ANS in yes no + do + case $ANS in + yes) + sudo dnf install git-subtree -y + break + ;; + no) + exit + break + ;; + *) + echo "You must enter yes or no" + ;; + esac + done + fi +fi + +if [ "$1" ]; then + subtree_repo=$1 +else + subtree_repo=https://github.com/hybrid-cloud-patterns/common.git +fi + +if [ "$2" ]; then + subtree_branch=$2 +else + subtree_branch=main +fi + +if [ "$3" ]; then + subtree_remote=$3 +else + subtree_remote=common-subtree +fi + +git diff --quiet || (echo "This script must be run on a clean working tree" && exit 1) + +echo "Changing directory to project root" +cd `git rev-parse --show-toplevel` + +echo "Removing existing common and replacing it with subtree from $subtree_repo $subtree_remote" +rm -rf common + +echo "Committing removal of common" +(git add -A :/ && git commit -m "Removed previous version of common to convert to subtree from $subtree_repo $subtree_branch") || exit 1 + +echo "Adding (possibly replacing) subtree remote $subtree_remote" +git remote rm "$subtree_remote" +git remote add -f "$subtree_remote" "$subtree_repo" || exit 1 +git subtree add --prefix=common "$subtree_remote" "$subtree_branch" || exit 1 + +echo "Complete. You may not push these results if you are satisfied" +exit 0 diff --git a/values-datacenter.yaml b/values-datacenter.yaml index 28da54a..ca30316 100644 --- a/values-datacenter.yaml +++ b/values-datacenter.yaml @@ -1,17 +1,20 @@ -global: - valuesDirectoryURL: https://github.com/dagger-refuse-cool/edge-gitops/raw/main/ - options: - useCSV: False - syncPolicy: Automatic - installPlanApproval: Automatic - -site: +clusterGroup: name: datacenter + isHubCluster: true + + proposedOptions: + manageGitops: True + isHubCluster: True +#Provide a list of namespaces to be created by the gitops operator. namespaces: - open-cluster-management - application-ci +#Define the operator subscriptions - if a specific channel is required, then include it; else +#create stanza without it to pull the default channel. For operators that are coming from community +#or certified operators you must add a source declaration. See opendatahub example below. + subscriptions: - name: advanced-cluster-management namespace: open-cluster-management @@ -21,9 +24,17 @@ site: - name: openshift-pipelines-operator-rh csv: redhat-openshift-pipelines.v1.5.1 + - name: opendatahub-operator + csv: opendatahub-operator.v1.1.2 + source: community-operators + +#Define the argocd project projects: - datacenter +#Define the applications to be manage by argocd - each defined application below is presented as +#an individual application within the argocd user interface. + applications: - name: acm namespace: open-cluster-management diff --git a/values-global.yaml b/values-global.yaml index faa2d90..e8d1d07 100644 --- a/values-global.yaml +++ b/values-global.yaml @@ -1,6 +1,9 @@ global: - valuesDirectoryURL: https://github.com/dagger-refuse-cool/edge-gitops/raw/main/ options: - useCSV: True + pattern: PROVIDE_PATTERN_NAME + useCSV: False syncPolicy: Manual installPlanApproval: Automatic + +main: + clusterGroupName: datacenter