diff --git a/applicationset/examples/cluster/cluster-example.yaml b/applicationset/examples/cluster/cluster-example.yaml new file mode 100644 index 0000000000000..497e7657de68c --- /dev/null +++ b/applicationset/examples/cluster/cluster-example.yaml @@ -0,0 +1,19 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - clusters: {} + template: + metadata: + name: '{{name}}-guestbook' + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + targetRevision: HEAD + path: guestbook + destination: + server: '{{server}}' + namespace: guestbook diff --git a/applicationset/examples/clusterDecisionResource/README.md b/applicationset/examples/clusterDecisionResource/README.md new file mode 100644 index 0000000000000..045cf8202e5ec --- /dev/null +++ b/applicationset/examples/clusterDecisionResource/README.md @@ -0,0 +1,57 @@ +# How the Cluster Decision Resource generator works for clusterDecisionResource +1. The Cluster Decision Resource generator reads a configurable status format: +```yaml +status: + clusters: + - name: cluster-01 + - name: cluster-02 +``` +This is a common status format. Another format that could be read looks like this: +```yaml +status: + decisions: + - clusterName: cluster-01 + namespace: cluster-01 + - clusterName: cluster-02 + namespace: cluster-02 +``` +2. Any resource that has a list of key / value pairs, where the value matches ArgoCD cluster names can be used. +3. The key / value pairs found in each element of the list will be available to the template. As well, `name` and `server` will still be available to the template. +4. The Service Account used by the ApplicationSet controller must have access to `Get` the resource you want to retrieve the duck type definition from +5. A configMap is used to identify the resource to read status of generated ArgoCD clusters from. You can use multiple resources by creating a ConfigMap for each one in the ArgoCD namespace. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: my-configmap +data: + apiVersion: group.io/v1 + kind: mykinds + statusListKey: clusters + matchKey: name +``` + * `apiVersion` - This is the apiVersion of your resource + * `kind` - This is the plural kind of your resource + * `statusListKey` - Default is 'clusters', this is the key found in your resource's status that is a list of ArgoCD clusters. + * `matchKey` - Is the key name found in the cluster list, `name` and `clusterName` are the keys in the examples above. + +# Applying the example +1. Connect to a cluster with the ApplicationSet controller running +2. Edit the Role for the ApplicationSet service account, and grant it permission to `list` the `placementdecisions` resources, from apiGroups `cluster.open-cluster-management.io/v1alpha1` +```yaml +- apiGroups: + - "cluster.open-cluster-management.io/v1alpha1" + resources: + - placementdecisions + verbs: + - list +``` +3. Apply the following controller and associated ManagedCluster CRD's: +https://github.com/open-cluster-management/placement +4. Now apply the PlacementDecision and an ApplicationSet: +```bash +kubectl apply -f ./placementdecision.yaml +kubectl apply -f ./configMap.yaml +kubectl apply -f ./ducktype-example.yaml +``` +5. For now this won't do anything until you create a controller that populates the `Status.Decisions` array. \ No newline at end of file diff --git a/applicationset/examples/clusterDecisionResource/configMap.yaml b/applicationset/examples/clusterDecisionResource/configMap.yaml new file mode 100644 index 0000000000000..69ddc81fc3fd9 --- /dev/null +++ b/applicationset/examples/clusterDecisionResource/configMap.yaml @@ -0,0 +1,11 @@ +# To generate a Status.Decisions from this CRD, requires https://github.com/open-cluster-management/multicloud-operators-placementrule be deployed +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: ocm-placement +data: + apiVersion: apps.open-cluster-management.io/v1 + kind: placementrules + statusListKey: decisions + matchKey: clusterName diff --git a/applicationset/examples/clusterDecisionResource/ducktype-example.yaml b/applicationset/examples/clusterDecisionResource/ducktype-example.yaml new file mode 100644 index 0000000000000..1663bbb06e483 --- /dev/null +++ b/applicationset/examples/clusterDecisionResource/ducktype-example.yaml @@ -0,0 +1,27 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: book-import +spec: + generators: + - clusterDecisionResource: + configMapRef: ocm-placement + name: test-placement + requeueAfterSeconds: 30 + template: + metadata: + name: '{{clusterName}}-book-import' + spec: + project: "default" + source: + repoURL: https://github.com/open-cluster-management/application-samples.git + targetRevision: HEAD + path: book-import + destination: + name: '{{clusterName}}' + namespace: bookimport + syncPolicy: + automated: + prune: true + syncOptions: + - CreateNamespace=true diff --git a/applicationset/examples/clusterDecisionResource/placementdecision.yaml b/applicationset/examples/clusterDecisionResource/placementdecision.yaml new file mode 100644 index 0000000000000..666c2acabeaa8 --- /dev/null +++ b/applicationset/examples/clusterDecisionResource/placementdecision.yaml @@ -0,0 +1,18 @@ +--- +apiVersion: apps.open-cluster-management.io/v1 +kind: PlacementRule +metadata: + name: test-placement +spec: + clusterReplicas: 1 # Availability choice, maximum number of clusters to provision at once + clusterSelector: + matchLabels: + 'usage': 'development' + clusterConditions: + - type: ManagedClusterConditionAvailable + status: "True" +# Below is sample output the generator can consume. +status: + decisions: + - clusterName: cluster-01 + - clusterName: cluster-02 \ No newline at end of file diff --git a/applicationset/examples/design-doc/applicationset.yaml b/applicationset/examples/design-doc/applicationset.yaml new file mode 100644 index 0000000000000..8249b727d2dc9 --- /dev/null +++ b/applicationset/examples/design-doc/applicationset.yaml @@ -0,0 +1,22 @@ +# This is an example of a typical ApplicationSet which uses the cluster generator. +# An ApplicationSet is comprised with two stanzas: +# - spec.generator - producer of a list of values supplied as arguments to an app template +# - spec.template - an application template, which has been parameterized +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - clusters: {} + template: + metadata: + name: '{{name}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + chart: guestbook + destination: + server: '{{server}}' + namespace: guestbook diff --git a/applicationset/examples/design-doc/clusters.yaml b/applicationset/examples/design-doc/clusters.yaml new file mode 100644 index 0000000000000..966455a67981c --- /dev/null +++ b/applicationset/examples/design-doc/clusters.yaml @@ -0,0 +1,33 @@ +# The cluster generator produces an items list from all clusters registered to Argo CD. +# It automatically provides the following fields as values to the app template: +# - name +# - server +# - metadata.labels. +# - metadata.annotations. +# - values. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - clusters: + selector: + matchLabels: + argocd.argoproj.io/secret-type: cluster + values: + project: default + template: + metadata: + name: '{{name}}-guestbook' + labels: + environment: '{{metadata.labels.environment}}' + spec: + project: '{{values.project}}' + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + chart: guestbook + destination: + server: '{{server}}' + namespace: guestbook diff --git a/applicationset/examples/design-doc/git-directory-discovery.yaml b/applicationset/examples/design-doc/git-directory-discovery.yaml new file mode 100644 index 0000000000000..7ff9bb3c053e5 --- /dev/null +++ b/applicationset/examples/design-doc/git-directory-discovery.yaml @@ -0,0 +1,44 @@ +# This example demonstrates the git directory generator, which produces an items list +# based on discovery of directories in a git repo matching a specified pattern. +# Git generators automatically provide {{path}} and {{path.basename}} as available +# variables to the app template. +# +# Suppose the following git directory structure (note the use of different config tools): +# +# cluster-deployments +# └── add-ons +# ├── argo-rollouts +# │   ├── all.yaml +# │   └── kustomization.yaml +# ├── argo-workflows +# │   └── install.yaml +# ├── grafana +# │   ├── Chart.yaml +# │   └── values.yaml +# └── prometheus-operator +# ├── Chart.yaml +# └── values.yaml +# +# The following ApplicationSet would produce four applications (in different namespaces), +# using the directory basename as both the namespace and application name. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-addons +spec: + generators: + - git: + repoURL: https://github.com/infra-team/cluster-deployments.git + directories: + - path: add-ons/* + template: + metadata: + name: '{{path.basename}}' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: http://kubernetes.default.svc + namespace: '{{path.basename}}' diff --git a/applicationset/examples/design-doc/git-files-discovery.yaml b/applicationset/examples/design-doc/git-files-discovery.yaml new file mode 100644 index 0000000000000..7cb717d31d49a --- /dev/null +++ b/applicationset/examples/design-doc/git-files-discovery.yaml @@ -0,0 +1,55 @@ +# This example demonstrates a git file generator which traverses the directory structure of a git +# repository to discover items based on a filename convention. For each file discovered, the +# contents of the discovered files themselves, act as the set of inputs to the app template. +# +# Suppose the following git directory structure: +# +# cluster-deployments +# ├── apps +# │ └── guestbook +# │ └── install.yaml +# └── cluster-config +# ├── engineering +# │ ├── dev +# │ │ └── config.json +# │ └── prod +# │ └── config.json +# └── finance +# ├── dev +# │ └── config.json +# └── prod +# └── config.json +# +# The discovered files (e.g. config.json) files can be any structured data supplied to the +# generated application. e.g.: +# { +# "aws_account": "123456", +# "asset_id": "11223344" +# "cluster": { +# "owner": "Jesse_Suen@intuit.com", +# "name": "engineering-dev", +# "address": "http://1.2.3.4" +# } +# } +# +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - git: + repoURL: https://github.com/infra-team/cluster-deployments.git + files: + - path: "**/config.json" + template: + metadata: + name: '{{cluster.name}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: apps/guestbook + destination: + server: '{{cluster.address}}' + namespace: guestbook diff --git a/applicationset/examples/design-doc/git-files-literal.yaml b/applicationset/examples/design-doc/git-files-literal.yaml new file mode 100644 index 0000000000000..f2c52b0c220f7 --- /dev/null +++ b/applicationset/examples/design-doc/git-files-literal.yaml @@ -0,0 +1,68 @@ +# This example demonstrates a git file generator which produces its items based on one or +# more files referenced in a git repo. The referenced files would contain a json/yaml list of +# arbitrary structured objects. Each item of the list would become a set of parameters to a +# generated application. +# +# Suppose the following git directory structure: +# +# cluster-deployments +# ├── apps +# │ └── guestbook +# │ ├── v1.0 +# │ │ └── install.yaml +# │ └── v2.0 +# │ └── install.yaml +# └── config +# └── clusters.json +# +# In this example, the `clusters.json` file is json list of structured data: +# [ +# { +# "account": "123456", +# "asset_id": "11223344", +# "cluster": { +# "owner": "Jesse_Suen@intuit.com", +# "name": "engineering-dev", +# "address": "http://1.2.3.4" +# }, +# "appVersions": { +# "prometheus-operator": "v0.38", +# "guestbook": "v2.0" +# } +# }, +# { +# "account": "456789", +# "asset_id": "55667788", +# "cluster": { +# "owner": "Alexander_Matyushentsev@intuit.com", +# "name": "engineering-prod", +# "address": "http://2.4.6.8" +# }, +# "appVersions": { +# "prometheus-operator": "v0.38", +# "guestbook": "v1.0" +# } +# } +# ] +# +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - git: + repoURL: https://github.com/infra-team/cluster-deployments.git + files: + - path: config/clusters.json + template: + metadata: + name: '{{cluster.name}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: apps/guestbook/{{appVersions.guestbook}} + destination: + server: http://kubernetes.default.svc + namespace: guestbook diff --git a/applicationset/examples/design-doc/list.yaml b/applicationset/examples/design-doc/list.yaml new file mode 100644 index 0000000000000..7cdbc5552442a --- /dev/null +++ b/applicationset/examples/design-doc/list.yaml @@ -0,0 +1,33 @@ +# The list generator specifies a literal list of argument values to the app spec template. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://1.2.3.4 + values: + project: dev + - cluster: engineering-prod + url: https://2.4.6.8 + values: + project: prod + - cluster: finance-preprod + url: https://9.8.7.6 + values: + project: preprod + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: '{{values.project}}' + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook diff --git a/applicationset/examples/design-doc/proposal/README.md b/applicationset/examples/design-doc/proposal/README.md new file mode 100644 index 0000000000000..5839139f928f8 --- /dev/null +++ b/applicationset/examples/design-doc/proposal/README.md @@ -0,0 +1,3 @@ +# Proposal Examples +This directory contains examples that are not yet implemented. +They are part of the project to indicate future progress, and we are welcome any contribution that will add an implementation diff --git a/applicationset/examples/design-doc/proposal/filters.yaml b/applicationset/examples/design-doc/proposal/filters.yaml new file mode 100644 index 0000000000000..295d2dce975da --- /dev/null +++ b/applicationset/examples/design-doc/proposal/filters.yaml @@ -0,0 +1,48 @@ +# For all generators, filters can be applied to reduce the generated items to a smaller subset. +# A powerful set of filter expressions are supported using syntax provided by the +# https://github.com/antonmedv/expr library. Examples expressions are demonstrated below +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + # Match all clusters who meet ALL of the following conditions: + # 1. name matches the regex `sales-.*` + # 2. environment label is either 'staging' or 'prod' + - clusters: + filters: + - expr: '{{name}} matches "sales-.*"' + - expr: '{{metadata.labels.environment}} in [staging, prod]' + values: + version: '2.0.0' + # Filter items from `config/clusters.json` in the `cluster-deployments` git repo, + # to only those having the `cluster.enabled == true` property. e.g.: + # { + # ... + # "cluster": { + # "enabled": true, + # ... + # } + # } + - git: + repoURL: https://github.com/infra-team/cluster-deployments.git + files: + - path: config/clusters.json + filters: + - expr: '{{cluster.enabled}} == true' + template: + metadata: + name: '{{name}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: "{{values.version}}" + chart: guestbook + helm: + parameters: + - name: foo + value: "{{metadata.annotations.foo}}" + destination: + server: '{{server}}' + namespace: guestbook diff --git a/applicationset/examples/design-doc/template-override.yaml b/applicationset/examples/design-doc/template-override.yaml new file mode 100644 index 0000000000000..9eade199f9762 --- /dev/null +++ b/applicationset/examples/design-doc/template-override.yaml @@ -0,0 +1,48 @@ +# App templates can also be defined as part of the generator's template stanza. Sometimes it is +# useful to do this in order to override the spec.template stanza, and when simple string +# parameterization are insufficient. In the below examples, the generators[].XXX.template is +# a partial definition, which overrides/patch the default template. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://1.2.3.4 + template: + metadata: {} + spec: + project: "project" + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + path: '{{cluster}}-override' + destination: {} + + - list: + elements: + - cluster: engineering-prod + url: https://1.2.3.4 + template: + metadata: {} + spec: + project: "project2" + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + path: '{{cluster}}-override2' + destination: {} + + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: "project" + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook diff --git a/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/kustomization.yaml b/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/kustomization.yaml new file mode 100644 index 0000000000000..68cd552162014 --- /dev/null +++ b/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/kustomization.yaml @@ -0,0 +1,6 @@ +#namePrefix: kustomize- + +resources: +- namespace-install.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/namespace-install.yaml b/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/namespace-install.yaml new file mode 100644 index 0000000000000..c140cd6297f98 --- /dev/null +++ b/applicationset/examples/git-generator-directory/cluster-addons/argo-workflows/namespace-install.yaml @@ -0,0 +1,417 @@ +# This is an auto-generated file. DO NOT EDIT +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: clusterworkflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: ClusterWorkflowTemplate + listKind: ClusterWorkflowTemplateList + plural: clusterworkflowtemplates + shortNames: + - clusterwftmpl + - cwft + singular: clusterworkflowtemplate + scope: Cluster + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cronworkflows.argoproj.io +spec: + group: argoproj.io + names: + kind: CronWorkflow + listKind: CronWorkflowList + plural: cronworkflows + shortNames: + - cwf + - cronwf + singular: cronworkflow + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workfloweventbindings.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowEventBinding + listKind: WorkflowEventBindingList + plural: workfloweventbindings + shortNames: + - wfeb + singular: workfloweventbinding + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflows.argoproj.io +spec: + additionalPrinterColumns: + - JSONPath: .status.phase + description: Status of the workflow + name: Status + type: string + - JSONPath: .status.startedAt + description: When the workflow was started + format: date-time + name: Age + type: date + group: argoproj.io + names: + kind: Workflow + listKind: WorkflowList + plural: workflows + shortNames: + - wf + singular: workflow + scope: Namespaced + subresources: {} + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowTemplate + listKind: WorkflowTemplateList + plural: workflowtemplates + shortNames: + - wftmpl + singular: workflowtemplate + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo-server +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-role +rules: +- apiGroups: + - "" + resources: + - pods + - pods/exec + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - persistentvolumeclaims + verbs: + - create + - delete + - get +- apiGroups: + - argoproj.io + resources: + - workflows + - workflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete + - create +- apiGroups: + - argoproj.io + resources: + - workflowtemplates + - workflowtemplates/finalizers + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get +- apiGroups: + - argoproj.io + resources: + - cronworkflows + - cronworkflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - create + - get + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-server-role +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get + - create +- apiGroups: + - "" + resources: + - pods + - pods/exec + - pods/log + verbs: + - get + - list + - watch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - watch + - create + - patch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - argoproj.io + resources: + - workflows + - workfloweventbindings + - workflowtemplates + - cronworkflows + - cronworkflows/finalizers + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-role +subjects: +- kind: ServiceAccount + name: argo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-server-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-server-role +subjects: +- kind: ServiceAccount + name: argo-server +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: workflow-controller-configmap +--- +apiVersion: v1 +kind: Service +metadata: + name: argo-server +spec: + ports: + - name: web + port: 2746 + targetPort: 2746 + selector: + app: argo-server +--- +apiVersion: v1 +kind: Service +metadata: + name: workflow-controller-metrics +spec: + ports: + - name: metrics + port: 9090 + protocol: TCP + targetPort: 9090 + selector: + app: workflow-controller +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: argo-server +spec: + selector: + matchLabels: + app: argo-server + template: + metadata: + labels: + app: argo-server + spec: + containers: + - args: + - server + - --namespaced + image: argoproj/argocli:v2.12.5 + name: argo-server + ports: + - containerPort: 2746 + name: web + readinessProbe: + httpGet: + path: / + port: 2746 + scheme: HTTP + initialDelaySeconds: 10 + periodSeconds: 20 + volumeMounts: + - mountPath: /tmp + name: tmp + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo-server + volumes: + - emptyDir: {} + name: tmp +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: workflow-controller +spec: + selector: + matchLabels: + app: workflow-controller + template: + metadata: + labels: + app: workflow-controller + spec: + containers: + - args: + - --configmap + - workflow-controller-configmap + - --executor-image + - argoproj/argoexec:v2.12.5 + - --namespaced + command: + - workflow-controller + image: argoproj/workflow-controller:v2.12.5 + livenessProbe: + httpGet: + path: /metrics + port: metrics + initialDelaySeconds: 30 + periodSeconds: 30 + name: workflow-controller + ports: + - containerPort: 9090 + name: metrics + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo diff --git a/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/Chart.yaml b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/Chart.yaml new file mode 100644 index 0000000000000..8247e4204049a --- /dev/null +++ b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/Chart.yaml @@ -0,0 +1,14 @@ +apiVersion: v2 +name: helm-prometheus-operator + +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +appVersion: "1.0" \ No newline at end of file diff --git a/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/requirements.yaml b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/requirements.yaml new file mode 100644 index 0000000000000..2e34cfccee057 --- /dev/null +++ b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/requirements.yaml @@ -0,0 +1,4 @@ +dependencies: +- name: kube-prometheus-stack + version: 9.4.10 + repository: https://prometheus-community.github.io/helm-charts diff --git a/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/values.yaml b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/values.yaml new file mode 100644 index 0000000000000..f09e9043f5db6 --- /dev/null +++ b/applicationset/examples/git-generator-directory/cluster-addons/prometheus-operator/values.yaml @@ -0,0 +1 @@ +# Blank values.yaml diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/kustomization.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/kustomization.yaml new file mode 100644 index 0000000000000..68cd552162014 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/kustomization.yaml @@ -0,0 +1,6 @@ +#namePrefix: kustomize- + +resources: +- namespace-install.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/namespace-install.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/namespace-install.yaml new file mode 100644 index 0000000000000..c140cd6297f98 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/argo-workflows/namespace-install.yaml @@ -0,0 +1,417 @@ +# This is an auto-generated file. DO NOT EDIT +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: clusterworkflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: ClusterWorkflowTemplate + listKind: ClusterWorkflowTemplateList + plural: clusterworkflowtemplates + shortNames: + - clusterwftmpl + - cwft + singular: clusterworkflowtemplate + scope: Cluster + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cronworkflows.argoproj.io +spec: + group: argoproj.io + names: + kind: CronWorkflow + listKind: CronWorkflowList + plural: cronworkflows + shortNames: + - cwf + - cronwf + singular: cronworkflow + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workfloweventbindings.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowEventBinding + listKind: WorkflowEventBindingList + plural: workfloweventbindings + shortNames: + - wfeb + singular: workfloweventbinding + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflows.argoproj.io +spec: + additionalPrinterColumns: + - JSONPath: .status.phase + description: Status of the workflow + name: Status + type: string + - JSONPath: .status.startedAt + description: When the workflow was started + format: date-time + name: Age + type: date + group: argoproj.io + names: + kind: Workflow + listKind: WorkflowList + plural: workflows + shortNames: + - wf + singular: workflow + scope: Namespaced + subresources: {} + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowTemplate + listKind: WorkflowTemplateList + plural: workflowtemplates + shortNames: + - wftmpl + singular: workflowtemplate + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo-server +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-role +rules: +- apiGroups: + - "" + resources: + - pods + - pods/exec + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - persistentvolumeclaims + verbs: + - create + - delete + - get +- apiGroups: + - argoproj.io + resources: + - workflows + - workflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete + - create +- apiGroups: + - argoproj.io + resources: + - workflowtemplates + - workflowtemplates/finalizers + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get +- apiGroups: + - argoproj.io + resources: + - cronworkflows + - cronworkflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - create + - get + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-server-role +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get + - create +- apiGroups: + - "" + resources: + - pods + - pods/exec + - pods/log + verbs: + - get + - list + - watch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - watch + - create + - patch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - argoproj.io + resources: + - workflows + - workfloweventbindings + - workflowtemplates + - cronworkflows + - cronworkflows/finalizers + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-role +subjects: +- kind: ServiceAccount + name: argo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-server-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-server-role +subjects: +- kind: ServiceAccount + name: argo-server +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: workflow-controller-configmap +--- +apiVersion: v1 +kind: Service +metadata: + name: argo-server +spec: + ports: + - name: web + port: 2746 + targetPort: 2746 + selector: + app: argo-server +--- +apiVersion: v1 +kind: Service +metadata: + name: workflow-controller-metrics +spec: + ports: + - name: metrics + port: 9090 + protocol: TCP + targetPort: 9090 + selector: + app: workflow-controller +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: argo-server +spec: + selector: + matchLabels: + app: argo-server + template: + metadata: + labels: + app: argo-server + spec: + containers: + - args: + - server + - --namespaced + image: argoproj/argocli:v2.12.5 + name: argo-server + ports: + - containerPort: 2746 + name: web + readinessProbe: + httpGet: + path: / + port: 2746 + scheme: HTTP + initialDelaySeconds: 10 + periodSeconds: 20 + volumeMounts: + - mountPath: /tmp + name: tmp + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo-server + volumes: + - emptyDir: {} + name: tmp +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: workflow-controller +spec: + selector: + matchLabels: + app: workflow-controller + template: + metadata: + labels: + app: workflow-controller + spec: + containers: + - args: + - --configmap + - workflow-controller-configmap + - --executor-image + - argoproj/argoexec:v2.12.5 + - --namespaced + command: + - workflow-controller + image: argoproj/workflow-controller:v2.12.5 + livenessProbe: + httpGet: + path: /metrics + port: metrics + initialDelaySeconds: 30 + periodSeconds: 30 + name: workflow-controller + ports: + - containerPort: 9090 + name: metrics + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/Chart.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/Chart.yaml new file mode 100644 index 0000000000000..6fac831761d30 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/Chart.yaml @@ -0,0 +1,23 @@ +apiVersion: v2 +name: helm-guestbook +description: A Helm chart for Kubernetes + +# A chart can be either an 'application' or a 'library' chart. +# +# Application charts are a collection of templates that can be packaged into versioned archives +# to be deployed. +# +# Library charts provide useful utilities or functions for the chart developer. They're included as +# a dependency of application charts to inject those utilities and functions into the rendering +# pipeline. Library charts do not define any templates and therefore cannot be deployed. +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +appVersion: "1.0" diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/NOTES.txt b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/NOTES.txt new file mode 100644 index 0000000000000..37a1485f894ea --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/NOTES.txt @@ -0,0 +1,19 @@ +1. Get the application URL by running these commands: +{{- if .Values.ingress.enabled }} +{{- range .Values.ingress.hosts }} + http{{ if $.Values.ingress.tls }}s{{ end }}://{{ . }}{{ $.Values.ingress.path }} +{{- end }} +{{- else if contains "NodePort" .Values.service.type }} + export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "helm-guestbook.fullname" . }}) + export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}") + echo http://$NODE_IP:$NODE_PORT +{{- else if contains "LoadBalancer" .Values.service.type }} + NOTE: It may take a few minutes for the LoadBalancer IP to be available. + You can watch the status of by running 'kubectl get svc -w {{ template "helm-guestbook.fullname" . }}' + export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "helm-guestbook.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}') + echo http://$SERVICE_IP:{{ .Values.service.port }} +{{- else if contains "ClusterIP" .Values.service.type }} + export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "helm-guestbook.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}") + echo "Visit http://127.0.0.1:8080 to use your application" + kubectl port-forward $POD_NAME 8080:80 +{{- end }} diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/_helpers.tpl b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/_helpers.tpl new file mode 100644 index 0000000000000..20f5d8e4ecd96 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/_helpers.tpl @@ -0,0 +1,32 @@ +{{/* vim: set filetype=mustache: */}} +{{/* +Expand the name of the chart. +*/}} +{{- define "helm-guestbook.name" -}} +{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} +{{- end -}} + +{{/* +Create a default fully qualified app name. +We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). +If release name contains chart name it will be used as a full name. +*/}} +{{- define "helm-guestbook.fullname" -}} +{{- if .Values.fullnameOverride -}} +{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- $name := default .Chart.Name .Values.nameOverride -}} +{{- if contains $name .Release.Name -}} +{{- .Release.Name | trunc 63 | trimSuffix "-" -}} +{{- else -}} +{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} +{{- end -}} +{{- end -}} +{{- end -}} + +{{/* +Create chart name and version as used by the chart label. +*/}} +{{- define "helm-guestbook.chart" -}} +{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} +{{- end -}} diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/deployment.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/deployment.yaml new file mode 100644 index 0000000000000..980df256ebcb4 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/deployment.yaml @@ -0,0 +1,52 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: {{ template "helm-guestbook.fullname" . }} + labels: + app: {{ template "helm-guestbook.name" . }} + chart: {{ template "helm-guestbook.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + replicas: {{ .Values.replicaCount }} + revisionHistoryLimit: 3 + selector: + matchLabels: + app: {{ template "helm-guestbook.name" . }} + release: {{ .Release.Name }} + template: + metadata: + labels: + app: {{ template "helm-guestbook.name" . }} + release: {{ .Release.Name }} + spec: + containers: + - name: {{ .Chart.Name }} + image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" + imagePullPolicy: {{ .Values.image.pullPolicy }} + ports: + - name: http + containerPort: 80 + protocol: TCP + livenessProbe: + httpGet: + path: / + port: http + readinessProbe: + httpGet: + path: / + port: http + resources: +{{ toYaml .Values.resources | indent 12 }} + {{- with .Values.nodeSelector }} + nodeSelector: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.affinity }} + affinity: +{{ toYaml . | indent 8 }} + {{- end }} + {{- with .Values.tolerations }} + tolerations: +{{ toYaml . | indent 8 }} + {{- end }} diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/service.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/service.yaml new file mode 100644 index 0000000000000..b7aab0ba3c919 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/templates/service.yaml @@ -0,0 +1,19 @@ +apiVersion: v1 +kind: Service +metadata: + name: {{ template "helm-guestbook.fullname" . }} + labels: + app: {{ template "helm-guestbook.name" . }} + chart: {{ template "helm-guestbook.chart" . }} + release: {{ .Release.Name }} + heritage: {{ .Release.Service }} +spec: + type: {{ .Values.service.type }} + ports: + - port: {{ .Values.service.port }} + targetPort: http + protocol: TCP + name: http + selector: + app: {{ template "helm-guestbook.name" . }} + release: {{ .Release.Name }} diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values-production.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values-production.yaml new file mode 100644 index 0000000000000..42838b76e43d0 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values-production.yaml @@ -0,0 +1,2 @@ +service: + type: LoadBalancer diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values.yaml new file mode 100644 index 0000000000000..3666712aec29b --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook/values.yaml @@ -0,0 +1,45 @@ +# Default values for helm-guestbook. +# This is a YAML-formatted file. +# Declare variables to be passed into your templates. + +replicaCount: 1 + +image: + repository: gcr.io/heptio-images/ks-guestbook-demo + tag: 0.1 + pullPolicy: IfNotPresent + +service: + type: ClusterIP + port: 80 + +ingress: + enabled: false + annotations: {} + # kubernetes.io/ingress.class: nginx + # kubernetes.io/tls-acme: "true" + path: / + hosts: + - chart-example.local + tls: [] + # - secretName: chart-example-tls + # hosts: + # - chart-example.local + +resources: {} + # We usually recommend not to specify default resources and to leave this as a conscious + # choice for the user. This also increases chances charts run on environments with little + # resources, such as Minikube. If you do want to specify resources, uncomment the following + # lines, adjust them as necessary, and remove the curly braces after 'resources:'. + # limits: + # cpu: 100m + # memory: 128Mi + # requests: + # cpu: 100m + # memory: 128Mi + +nodeSelector: {} + +tolerations: [] + +affinity: {} diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/Chart.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/Chart.yaml new file mode 100644 index 0000000000000..10aa9d0b745b1 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/Chart.yaml @@ -0,0 +1 @@ +name: helm-prometheus-operator diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/requirements.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/requirements.yaml new file mode 100644 index 0000000000000..2e34cfccee057 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/requirements.yaml @@ -0,0 +1,4 @@ +dependencies: +- name: kube-prometheus-stack + version: 9.4.10 + repository: https://prometheus-community.github.io/helm-charts diff --git a/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/values.yaml b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/values.yaml new file mode 100644 index 0000000000000..f09e9043f5db6 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/cluster-addons/prometheus-operator/values.yaml @@ -0,0 +1 @@ +# Blank values.yaml diff --git a/applicationset/examples/git-generator-directory/excludes/git-directories-exclude-example.yaml b/applicationset/examples/git-generator-directory/excludes/git-directories-exclude-example.yaml new file mode 100644 index 0000000000000..46b20a3bb9ce1 --- /dev/null +++ b/applicationset/examples/git-generator-directory/excludes/git-directories-exclude-example.yaml @@ -0,0 +1,25 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-addons +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/git-generator-directory/excludes/cluster-addons/* + - exclude: true + path: applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook + template: + metadata: + name: '{{path.basename}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: https://kubernetes.default.svc + namespace: '{{path.basename}}' diff --git a/applicationset/examples/git-generator-directory/git-directories-example.yaml b/applicationset/examples/git-generator-directory/git-directories-example.yaml new file mode 100644 index 0000000000000..186acfc3e9569 --- /dev/null +++ b/applicationset/examples/git-generator-directory/git-directories-example.yaml @@ -0,0 +1,23 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-addons +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/git-generator-directory/cluster-addons/* + template: + metadata: + name: '{{path.basename}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: https://kubernetes.default.svc + namespace: '{{path.basename}}' diff --git a/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-deployment.yaml b/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-deployment.yaml new file mode 100644 index 0000000000000..8a0975e363539 --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: guestbook-ui +spec: + replicas: 1 + revisionHistoryLimit: 3 + selector: + matchLabels: + app: guestbook-ui + template: + metadata: + labels: + app: guestbook-ui + spec: + containers: + - image: gcr.io/heptio-images/ks-guestbook-demo:0.2 + name: guestbook-ui + ports: + - containerPort: 80 diff --git a/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-svc.yaml b/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-svc.yaml new file mode 100644 index 0000000000000..e619b5cd39f7b --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/apps/guestbook/guestbook-ui-svc.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: guestbook-ui +spec: + ports: + - port: 80 + targetPort: 80 + selector: + app: guestbook-ui diff --git a/applicationset/examples/git-generator-files-discovery/apps/guestbook/kustomization.yaml b/applicationset/examples/git-generator-files-discovery/apps/guestbook/kustomization.yaml new file mode 100644 index 0000000000000..cbaba9021cb23 --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/apps/guestbook/kustomization.yaml @@ -0,0 +1,7 @@ +namePrefix: kustomize- + +resources: +- guestbook-ui-deployment.yaml +- guestbook-ui-svc.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/dev/config.json b/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/dev/config.json new file mode 100644 index 0000000000000..51b23ca1bbe4f --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/dev/config.json @@ -0,0 +1,9 @@ +{ + "aws_account": "123456", + "asset_id": "11223344", + "cluster": { + "owner": "cluster-admin@company.com", + "name": "engineering-dev", + "address": "http://1.2.3.4" + } +} diff --git a/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/prod/config.json b/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/prod/config.json new file mode 100644 index 0000000000000..4a48be12ff3da --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/cluster-config/engineering/prod/config.json @@ -0,0 +1,9 @@ +{ + "aws_account": "123456", + "asset_id": "11223344", + "cluster": { + "owner": "cluster-admin@company.com", + "name": "engineering-prod", + "address": "http://1.2.3.4" + } +} diff --git a/applicationset/examples/git-generator-files-discovery/git-generator-files.yaml b/applicationset/examples/git-generator-files-discovery/git-generator-files.yaml new file mode 100644 index 0000000000000..99923b72c565a --- /dev/null +++ b/applicationset/examples/git-generator-files-discovery/git-generator-files.yaml @@ -0,0 +1,24 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + files: + - path: "applicationset/examples/git-generator-files-discovery/cluster-config/**/config.json" + template: + metadata: + name: '{{cluster.name}}-guestbook' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: "applicationset/examples/git-generator-files-discovery/apps/guestbook" + destination: + server: https://kubernetes.default.svc + #server: '{{cluster.address}}' + namespace: guestbook diff --git a/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-deployment.yaml b/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-deployment.yaml new file mode 100644 index 0000000000000..8a0975e363539 --- /dev/null +++ b/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: guestbook-ui +spec: + replicas: 1 + revisionHistoryLimit: 3 + selector: + matchLabels: + app: guestbook-ui + template: + metadata: + labels: + app: guestbook-ui + spec: + containers: + - image: gcr.io/heptio-images/ks-guestbook-demo:0.2 + name: guestbook-ui + ports: + - containerPort: 80 diff --git a/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-svc.yaml b/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-svc.yaml new file mode 100644 index 0000000000000..e8a4a27fbae40 --- /dev/null +++ b/applicationset/examples/list-generator/guestbook/engineering-dev/guestbook-ui-svc.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: guestbook-ui +spec: + ports: + - port: 80 + targetPort: 80 + selector: + app: guestbook-ui diff --git a/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-deployment.yaml b/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-deployment.yaml new file mode 100644 index 0000000000000..8a0975e363539 --- /dev/null +++ b/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: guestbook-ui +spec: + replicas: 1 + revisionHistoryLimit: 3 + selector: + matchLabels: + app: guestbook-ui + template: + metadata: + labels: + app: guestbook-ui + spec: + containers: + - image: gcr.io/heptio-images/ks-guestbook-demo:0.2 + name: guestbook-ui + ports: + - containerPort: 80 diff --git a/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-svc.yaml b/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-svc.yaml new file mode 100644 index 0000000000000..e8a4a27fbae40 --- /dev/null +++ b/applicationset/examples/list-generator/guestbook/engineering-prod/guestbook-ui-svc.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: guestbook-ui +spec: + ports: + - port: 80 + targetPort: 80 + selector: + app: guestbook-ui diff --git a/applicationset/examples/list-generator/list-example.yaml b/applicationset/examples/list-generator/list-example.yaml new file mode 100644 index 0000000000000..a671ee45fab02 --- /dev/null +++ b/applicationset/examples/list-generator/list-example.yaml @@ -0,0 +1,24 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + - cluster: engineering-prod + url: https://kubernetes.default.svc + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: applicationset/examples/list-generator/guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook diff --git a/applicationset/examples/matrix/cluster-addons/argo-workflows/kustomization.yaml b/applicationset/examples/matrix/cluster-addons/argo-workflows/kustomization.yaml new file mode 100644 index 0000000000000..68cd552162014 --- /dev/null +++ b/applicationset/examples/matrix/cluster-addons/argo-workflows/kustomization.yaml @@ -0,0 +1,6 @@ +#namePrefix: kustomize- + +resources: +- namespace-install.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/matrix/cluster-addons/argo-workflows/namespace-install.yaml b/applicationset/examples/matrix/cluster-addons/argo-workflows/namespace-install.yaml new file mode 100644 index 0000000000000..c140cd6297f98 --- /dev/null +++ b/applicationset/examples/matrix/cluster-addons/argo-workflows/namespace-install.yaml @@ -0,0 +1,417 @@ +# This is an auto-generated file. DO NOT EDIT +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: clusterworkflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: ClusterWorkflowTemplate + listKind: ClusterWorkflowTemplateList + plural: clusterworkflowtemplates + shortNames: + - clusterwftmpl + - cwft + singular: clusterworkflowtemplate + scope: Cluster + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: cronworkflows.argoproj.io +spec: + group: argoproj.io + names: + kind: CronWorkflow + listKind: CronWorkflowList + plural: cronworkflows + shortNames: + - cwf + - cronwf + singular: cronworkflow + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workfloweventbindings.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowEventBinding + listKind: WorkflowEventBindingList + plural: workfloweventbindings + shortNames: + - wfeb + singular: workfloweventbinding + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflows.argoproj.io +spec: + additionalPrinterColumns: + - JSONPath: .status.phase + description: Status of the workflow + name: Status + type: string + - JSONPath: .status.startedAt + description: When the workflow was started + format: date-time + name: Age + type: date + group: argoproj.io + names: + kind: Workflow + listKind: WorkflowList + plural: workflows + shortNames: + - wf + singular: workflow + scope: Namespaced + subresources: {} + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: workflowtemplates.argoproj.io +spec: + group: argoproj.io + names: + kind: WorkflowTemplate + listKind: WorkflowTemplateList + plural: workflowtemplates + shortNames: + - wftmpl + singular: workflowtemplate + scope: Namespaced + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: argo-server +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-role +rules: +- apiGroups: + - "" + resources: + - pods + - pods/exec + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - persistentvolumeclaims + verbs: + - create + - delete + - get +- apiGroups: + - argoproj.io + resources: + - workflows + - workflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete + - create +- apiGroups: + - argoproj.io + resources: + - workflowtemplates + - workflowtemplates/finalizers + verbs: + - get + - list + - watch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get +- apiGroups: + - argoproj.io + resources: + - cronworkflows + - cronworkflows/finalizers + verbs: + - get + - list + - watch + - update + - patch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - create + - patch +- apiGroups: + - policy + resources: + - poddisruptionbudgets + verbs: + - create + - get + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + name: argo-server-role +rules: +- apiGroups: + - "" + resources: + - configmaps + verbs: + - get + - watch + - list +- apiGroups: + - "" + resources: + - secrets + verbs: + - get + - create +- apiGroups: + - "" + resources: + - pods + - pods/exec + - pods/log + verbs: + - get + - list + - watch + - delete +- apiGroups: + - "" + resources: + - events + verbs: + - watch + - create + - patch +- apiGroups: + - "" + resources: + - serviceaccounts + verbs: + - get + - list +- apiGroups: + - argoproj.io + resources: + - workflows + - workfloweventbindings + - workflowtemplates + - cronworkflows + - cronworkflows/finalizers + verbs: + - create + - get + - list + - watch + - update + - patch + - delete +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-role +subjects: +- kind: ServiceAccount + name: argo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: RoleBinding +metadata: + name: argo-server-binding +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: argo-server-role +subjects: +- kind: ServiceAccount + name: argo-server +--- +apiVersion: v1 +kind: ConfigMap +metadata: + name: workflow-controller-configmap +--- +apiVersion: v1 +kind: Service +metadata: + name: argo-server +spec: + ports: + - name: web + port: 2746 + targetPort: 2746 + selector: + app: argo-server +--- +apiVersion: v1 +kind: Service +metadata: + name: workflow-controller-metrics +spec: + ports: + - name: metrics + port: 9090 + protocol: TCP + targetPort: 9090 + selector: + app: workflow-controller +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: argo-server +spec: + selector: + matchLabels: + app: argo-server + template: + metadata: + labels: + app: argo-server + spec: + containers: + - args: + - server + - --namespaced + image: argoproj/argocli:v2.12.5 + name: argo-server + ports: + - containerPort: 2746 + name: web + readinessProbe: + httpGet: + path: / + port: 2746 + scheme: HTTP + initialDelaySeconds: 10 + periodSeconds: 20 + volumeMounts: + - mountPath: /tmp + name: tmp + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo-server + volumes: + - emptyDir: {} + name: tmp +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: workflow-controller +spec: + selector: + matchLabels: + app: workflow-controller + template: + metadata: + labels: + app: workflow-controller + spec: + containers: + - args: + - --configmap + - workflow-controller-configmap + - --executor-image + - argoproj/argoexec:v2.12.5 + - --namespaced + command: + - workflow-controller + image: argoproj/workflow-controller:v2.12.5 + livenessProbe: + httpGet: + path: /metrics + port: metrics + initialDelaySeconds: 30 + periodSeconds: 30 + name: workflow-controller + ports: + - containerPort: 9090 + name: metrics + nodeSelector: + kubernetes.io/os: linux + securityContext: + runAsNonRoot: true + serviceAccountName: argo diff --git a/applicationset/examples/matrix/cluster-addons/prometheus-operator/Chart.yaml b/applicationset/examples/matrix/cluster-addons/prometheus-operator/Chart.yaml new file mode 100644 index 0000000000000..8247e4204049a --- /dev/null +++ b/applicationset/examples/matrix/cluster-addons/prometheus-operator/Chart.yaml @@ -0,0 +1,14 @@ +apiVersion: v2 +name: helm-prometheus-operator + +type: application + +# This is the chart version. This version number should be incremented each time you make changes +# to the chart and its templates, including the app version. +# Versions are expected to follow Semantic Versioning (https://semver.org/) +version: 0.1.0 + +# This is the version number of the application being deployed. This version number should be +# incremented each time you make changes to the application. Versions are not expected to +# follow Semantic Versioning. They should reflect the version the application is using. +appVersion: "1.0" \ No newline at end of file diff --git a/applicationset/examples/matrix/cluster-addons/prometheus-operator/requirements.yaml b/applicationset/examples/matrix/cluster-addons/prometheus-operator/requirements.yaml new file mode 100644 index 0000000000000..2e34cfccee057 --- /dev/null +++ b/applicationset/examples/matrix/cluster-addons/prometheus-operator/requirements.yaml @@ -0,0 +1,4 @@ +dependencies: +- name: kube-prometheus-stack + version: 9.4.10 + repository: https://prometheus-community.github.io/helm-charts diff --git a/applicationset/examples/matrix/cluster-addons/prometheus-operator/values.yaml b/applicationset/examples/matrix/cluster-addons/prometheus-operator/values.yaml new file mode 100644 index 0000000000000..f09e9043f5db6 --- /dev/null +++ b/applicationset/examples/matrix/cluster-addons/prometheus-operator/values.yaml @@ -0,0 +1 @@ +# Blank values.yaml diff --git a/applicationset/examples/matrix/cluster-and-git.yaml b/applicationset/examples/matrix/cluster-and-git.yaml new file mode 100644 index 0000000000000..4c7497f71242e --- /dev/null +++ b/applicationset/examples/matrix/cluster-and-git.yaml @@ -0,0 +1,33 @@ +# This example demonstrates the combining of the git generator with a cluster generator +# The expected output would be an application per git directory and a cluster (application_count = git directory * clusters) +# +# +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-git +spec: + generators: + - matrix: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/matrix/cluster-addons/* + - clusters: + selector: + matchLabels: + argocd.argoproj.io/secret-type: cluster + template: + metadata: + name: '{{path.basename}}-{{name}}' + spec: + project: '{{metadata.labels.environment}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: '{{server}}' + namespace: '{{path.basename}}' diff --git a/applicationset/examples/matrix/list-and-git.yaml b/applicationset/examples/matrix/list-and-git.yaml new file mode 100644 index 0000000000000..33f5511902777 --- /dev/null +++ b/applicationset/examples/matrix/list-and-git.yaml @@ -0,0 +1,39 @@ +# This example demonstrates the combining of the git generator with a list generator +# The expected output would be an application per git directory and a list entry (application_count = git directory * list entries) +# +# +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: list-git +spec: + generators: + - matrix: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/matrix/cluster-addons/* + - list: + elements: + - cluster: engineering-dev + url: https://1.2.3.4 + values: + project: dev + - cluster: engineering-prod + url: https://2.4.6.8 + values: + project: prod + template: + metadata: + name: '{{path.basename}}-{{cluster}}' + spec: + project: '{{values.project}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: '{{url}}' + namespace: '{{path.basename}}' diff --git a/applicationset/examples/matrix/list-and-list.yaml b/applicationset/examples/matrix/list-and-list.yaml new file mode 100644 index 0000000000000..7e1ac1237ad29 --- /dev/null +++ b/applicationset/examples/matrix/list-and-list.yaml @@ -0,0 +1,37 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: list-and-list + namespace: argocd +spec: + generators: + - matrix: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + values: + project: default + - cluster: engineering-prod + url: https://kubernetes.default.svc + values: + project: default + - list: + elements: + - values: + suffix: '1' + - values: + suffix: '2' + template: + metadata: + name: '{{cluster}}-{{values.suffix}}' + spec: + project: '{{values.project}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: '{{url}}' + namespace: '{{path.basename}}' diff --git a/applicationset/examples/matrix/matrix-and-union-in-matrix.yaml b/applicationset/examples/matrix/matrix-and-union-in-matrix.yaml new file mode 100644 index 0000000000000..eb5938382942d --- /dev/null +++ b/applicationset/examples/matrix/matrix-and-union-in-matrix.yaml @@ -0,0 +1,67 @@ +# The matrix generator can contain other combination-type generators (matrix and union). But nested matrix and union +# generators cannot contain further-nested matrix or union generators. +# +# The generators are evaluated from most-nested to least-nested. In this case: +# 1. The union generator joins two lists to make 3 parameter sets. +# 2. The inner matrix generator takes the cartesian product of the two lists to make 4 parameters sets. +# 3. The outer matrix generator takes the cartesian product of the 3 union and the 4 inner matrix parameter sets to +# make 3*4=12 final parameter sets. +# 4. The 12 final parameter sets are evaluated against the top-level template to generate 12 Applications. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: matrix-and-union-in-matrix +spec: + generators: + - matrix: + generators: + - union: + mergeKeys: + - cluster + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + values: + project: default + - cluster: engineering-prod + url: https://kubernetes.default.svc + values: + project: default + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + values: + project: default + - cluster: engineering-test + url: https://kubernetes.default.svc + values: + project: default + - matrix: + generators: + - list: + elements: + - values: + suffix: '1' + - values: + suffix: '2' + - list: + elements: + - values: + prefix: 'first' + - values: + prefix: 'second' + template: + metadata: + name: '{{values.prefix}}-{{cluster}}-{{values.suffix}}' + spec: + project: '{{values.project}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: '{{url}}' + namespace: '{{path.basename}}' diff --git a/applicationset/examples/merge/merge-clusters-and-list.yaml b/applicationset/examples/merge/merge-clusters-and-list.yaml new file mode 100644 index 0000000000000..5b6971238edd3 --- /dev/null +++ b/applicationset/examples/merge/merge-clusters-and-list.yaml @@ -0,0 +1,44 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: merge-clusters-and-list +spec: + generators: + - merge: + mergeKeys: + - server + generators: + - clusters: + values: + kafka: 'true' + redis: 'false' + # For clusters with a specific label, enable Kafka. + - clusters: + selector: + matchLabels: + use-kafka: 'false' + values: + kafka: 'false' + # For a specific cluster, enable Redis. + - list: + elements: + - server: https://some-specific-cluster + values.redis: 'true' + template: + metadata: + name: '{{name}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + targetRevision: HEAD + path: helm-guestbook + helm: + parameters: + - name: kafka + value: '{{values.kafka}}' + - name: redis + value: '{{values.redis}}' + destination: + server: '{{server}}' + namespace: default diff --git a/applicationset/examples/merge/merge-two-matrixes.yaml b/applicationset/examples/merge/merge-two-matrixes.yaml new file mode 100644 index 0000000000000..f47463d7293c5 --- /dev/null +++ b/applicationset/examples/merge/merge-two-matrixes.yaml @@ -0,0 +1,43 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: merge-two-matrixes +spec: + generators: + - merge: + mergeKeys: + - server + - environment + generators: + - matrix: + generators: + - clusters: + values: + replicaCount: '2' + - list: + elements: + - environment: staging + namespace: guestbook-non-prod + - environment: prod + namespace: guestbook + - list: + elements: + - server: https://kubernetes.default.svc + environment: staging + values.replicaCount: '1' + template: + metadata: + name: '{{name}}-guestbook-{{environment}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + targetRevision: HEAD + path: helm-guestbook + helm: + parameters: + - name: replicaCount + value: '{{values.replicaCount}}' + destination: + server: '{{server}}' + namespace: '{{namespace}}' diff --git a/applicationset/examples/pull-request-generator/pull-request-example.yaml b/applicationset/examples/pull-request-generator/pull-request-example.yaml new file mode 100644 index 0000000000000..e5d2d5adc0ad8 --- /dev/null +++ b/applicationset/examples/pull-request-generator/pull-request-example.yaml @@ -0,0 +1,40 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapp +spec: + generators: + - pullRequest: + github: + # The GitHub organization or user. + owner: myorg + # The Github repository + repo: myrepo + # For GitHub Enterprise. (optional) + api: https://git.example.com/ + # Reference to a Secret containing an access token. (optional) + tokenRef: + secretName: github-token + key: token + # Labels is used to filter the PRs that you want to target. (optional) + labels: + - preview + template: + metadata: + name: 'myapp-{{ branch }}-{{ number }}' + spec: + source: + repoURL: 'https://github.com/myorg/myrepo.git' + targetRevision: '{{ head_sha }}' + path: helm-guestbook + helm: + parameters: + - name: "image.tag" + value: "pull-{{ head_sha }}" + project: default + destination: + server: https://kubernetes.default.svc + namespace: "{{ branch }}-{{ number }}" + syncPolicy: + syncOptions: + - CreateNamespace=true diff --git a/applicationset/examples/scm-provider-generator/scm-provider-example.yaml b/applicationset/examples/scm-provider-generator/scm-provider-example.yaml new file mode 100644 index 0000000000000..24d8ba41c2aed --- /dev/null +++ b/applicationset/examples/scm-provider-generator/scm-provider-example.yaml @@ -0,0 +1,24 @@ +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - scmProvider: + github: + organization: argoproj + cloneProtocol: https + filters: + - repositoryMatch: example-apps + template: + metadata: + name: '{{ repository }}-guestbook' + spec: + project: "default" + source: + repoURL: '{{ url }}' + targetRevision: '{{ branch }}' + path: guestbook + destination: + server: https://kubernetes.default.svc + namespace: guestbook diff --git a/applicationset/examples/template-override/default/guestbook-ui-deployment.yaml b/applicationset/examples/template-override/default/guestbook-ui-deployment.yaml new file mode 100644 index 0000000000000..8a0975e363539 --- /dev/null +++ b/applicationset/examples/template-override/default/guestbook-ui-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: guestbook-ui +spec: + replicas: 1 + revisionHistoryLimit: 3 + selector: + matchLabels: + app: guestbook-ui + template: + metadata: + labels: + app: guestbook-ui + spec: + containers: + - image: gcr.io/heptio-images/ks-guestbook-demo:0.2 + name: guestbook-ui + ports: + - containerPort: 80 diff --git a/applicationset/examples/template-override/default/guestbook-ui-svc.yaml b/applicationset/examples/template-override/default/guestbook-ui-svc.yaml new file mode 100644 index 0000000000000..e619b5cd39f7b --- /dev/null +++ b/applicationset/examples/template-override/default/guestbook-ui-svc.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: guestbook-ui +spec: + ports: + - port: 80 + targetPort: 80 + selector: + app: guestbook-ui diff --git a/applicationset/examples/template-override/default/kustomization.yaml b/applicationset/examples/template-override/default/kustomization.yaml new file mode 100644 index 0000000000000..cbaba9021cb23 --- /dev/null +++ b/applicationset/examples/template-override/default/kustomization.yaml @@ -0,0 +1,7 @@ +namePrefix: kustomize- + +resources: +- guestbook-ui-deployment.yaml +- guestbook-ui-svc.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-deployment.yaml b/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-deployment.yaml new file mode 100644 index 0000000000000..8a0975e363539 --- /dev/null +++ b/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-deployment.yaml @@ -0,0 +1,20 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + name: guestbook-ui +spec: + replicas: 1 + revisionHistoryLimit: 3 + selector: + matchLabels: + app: guestbook-ui + template: + metadata: + labels: + app: guestbook-ui + spec: + containers: + - image: gcr.io/heptio-images/ks-guestbook-demo:0.2 + name: guestbook-ui + ports: + - containerPort: 80 diff --git a/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-svc.yaml b/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-svc.yaml new file mode 100644 index 0000000000000..e619b5cd39f7b --- /dev/null +++ b/applicationset/examples/template-override/engineering-dev-override/guestbook-ui-svc.yaml @@ -0,0 +1,10 @@ +apiVersion: v1 +kind: Service +metadata: + name: guestbook-ui +spec: + ports: + - port: 80 + targetPort: 80 + selector: + app: guestbook-ui diff --git a/applicationset/examples/template-override/engineering-dev-override/kustomization.yaml b/applicationset/examples/template-override/engineering-dev-override/kustomization.yaml new file mode 100644 index 0000000000000..cbaba9021cb23 --- /dev/null +++ b/applicationset/examples/template-override/engineering-dev-override/kustomization.yaml @@ -0,0 +1,7 @@ +namePrefix: kustomize- + +resources: +- guestbook-ui-deployment.yaml +- guestbook-ui-svc.yaml +apiVersion: kustomize.config.k8s.io/v1beta1 +kind: Kustomization diff --git a/applicationset/examples/template-override/template-overrides-example.yaml b/applicationset/examples/template-override/template-overrides-example.yaml new file mode 100644 index 0000000000000..a8fe9916b94d6 --- /dev/null +++ b/applicationset/examples/template-override/template-overrides-example.yaml @@ -0,0 +1,36 @@ +# App templates can also be defined as part of the generator's template stanza. Sometimes it is +# useful to do this in order to override the spec.template stanza, and when simple string +# parameterization are insufficient. In the below examples, the generators[].XXX.template is +# a partial definition, which overrides/patch the default template. +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + template: + metadata: {} + spec: + project: "default" + source: + targetRevision: HEAD + repoURL: https://github.com/argoproj/argo-cd.git + path: 'applicationset/examples/template-override/{{cluster}}-override' + destination: {} + + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: applicationset/examples/template-override/default + destination: + server: '{{url}}' + namespace: guestbook diff --git a/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Diagram-v2.odp b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Diagram-v2.odp new file mode 100644 index 0000000000000..47c1acf2f24d5 Binary files /dev/null and b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Diagram-v2.odp differ diff --git a/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship-v2.png b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship-v2.png new file mode 100644 index 0000000000000..b1bd6dfc5dd16 Binary files /dev/null and b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship-v2.png differ diff --git a/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship.png b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship.png new file mode 100644 index 0000000000000..cde00e6851582 Binary files /dev/null and b/docs/assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship.png differ diff --git a/docs/assets/applicationset/Introduction/List-Example-In-Argo-CD-Web-UI.png b/docs/assets/applicationset/Introduction/List-Example-In-Argo-CD-Web-UI.png new file mode 100644 index 0000000000000..faf63fe7848b0 Binary files /dev/null and b/docs/assets/applicationset/Introduction/List-Example-In-Argo-CD-Web-UI.png differ diff --git a/docs/assets/applicationset/Use-Cases/Cluster-Add-Ons.png b/docs/assets/applicationset/Use-Cases/Cluster-Add-Ons.png new file mode 100644 index 0000000000000..f6353cc29e340 Binary files /dev/null and b/docs/assets/applicationset/Use-Cases/Cluster-Add-Ons.png differ diff --git a/docs/assets/applicationset/Use-Cases/Monorepos.png b/docs/assets/applicationset/Use-Cases/Monorepos.png new file mode 100644 index 0000000000000..523af303c1d40 Binary files /dev/null and b/docs/assets/applicationset/Use-Cases/Monorepos.png differ diff --git a/docs/assets/applicationset/logo.png b/docs/assets/applicationset/logo.png new file mode 100644 index 0000000000000..b9cb18e63431b Binary files /dev/null and b/docs/assets/applicationset/logo.png differ diff --git a/docs/assets/applicationset/webhook-config-pull-request.png b/docs/assets/applicationset/webhook-config-pull-request.png new file mode 100644 index 0000000000000..434c895a8ab30 Binary files /dev/null and b/docs/assets/applicationset/webhook-config-pull-request.png differ diff --git a/docs/assets/applicationset/webhook-config.png b/docs/assets/applicationset/webhook-config.png new file mode 100644 index 0000000000000..76a6400e5c7eb Binary files /dev/null and b/docs/assets/applicationset/webhook-config.png differ diff --git a/docs/operator-manual/applicationset/Application-Deletion.md b/docs/operator-manual/applicationset/Application-Deletion.md new file mode 100644 index 0000000000000..b59a556ec7f40 --- /dev/null +++ b/docs/operator-manual/applicationset/Application-Deletion.md @@ -0,0 +1,29 @@ +# Application Pruning & Resource Deletion + +All `Application` resources created by the ApplicationSet controller (from an ApplicationSet) will contain: + +- A `.metadata.ownerReferences` reference back to the *parent* `ApplicationSet` resource +- An Argo CD `resources-finalizer.argocd.argoproj.io` finalizer in `.metadata.finalizers` of the Application if `.syncPolicy.preserveResourcesOnDeletion` is set to false. + +The end result is that when an ApplicationSet is deleted, the following occurs (in rough order): + +- The `ApplicationSet` resource itself is deleted +- Any `Application` resources that were created from this `ApplicationSet` (as identified by owner reference) +- Any deployed resources (`Deployments`, `Services`, `ConfigMaps`, etc) on the managed cluster, that were created from that `Application` resource (by Argo CD), will be deleted. + - Argo CD is responsible for handling this deletion, via [the deletion finalizer](../../../user-guide/app_deletion/#about-the-deletion-finalizer). + - To preserve deployed resources, set `.syncPolicy.preserveResourcesOnDeletion` to true in the ApplicationSet. + +Thus the lifecycle of the `ApplicationSet`, the `Application`, and the `Application`'s resources, are equivalent. + +!!! note + See also the [controlling resource modification](Controlling-Resource-Modification.md) page for more information about how to prevent deletion or modification of Application resources by the ApplicationSet controller. + +It *is* still possible to delete an `ApplicationSet` resource, while preventing `Application`s (and their deployed resources) from also being deleted, using a non-cascading delete: +``` +kubectl delete ApplicationSet (NAME) --cascade=orphan +``` + +!!! warning + Even if using a non-cascaded delete, the `resources-finalizer.argocd.argoproj.io` is still specified on the `Application`. Thus, when the `Application` is deleted, all of its deployed resources will also be deleted. (The lifecycle of the Application, and its *child* objects, are still equivalent.) + + To prevent the deletion of the resources of the Application, such as Services, Deployments, etc, set `.syncPolicy.preserveResourcesOnDeletion` to true in the ApplicationSet. This syncPolicy parameter prevents the finalizer from being added to the Application. \ No newline at end of file diff --git a/docs/operator-manual/applicationset/Argo-CD-Integration.md b/docs/operator-manual/applicationset/Argo-CD-Integration.md new file mode 100644 index 0000000000000..d6179b570eebb --- /dev/null +++ b/docs/operator-manual/applicationset/Argo-CD-Integration.md @@ -0,0 +1,29 @@ +# How ApplicationSet controller interacts with Argo CD + +When you create, update, or delete an `ApplicationSet` resource, the ApplicationSet controller responds by creating, updating, or deleting one or more corresponding Argo CD `Application` resources. + +In fact, the *sole* responsibility of the ApplicationSet controller is to create, update, and delete `Application` resources within the Argo CD namespace. The controller's only job is to ensure that the `Application` resources remain consistent with the defined declarative `ApplicationSet` resource, and nothing more. + +Thus the ApplicationSet controller: + +- Does not create/modify/delete Kubernetes resources (other than the `Application` CR) +- Does not connect to clusters other than the one Argo CD is deployed to +- Does not interact with namespaces other than the one Argo CD is deployed within + +!!!important "Use the Argo CD namespace" + All ApplicationSet resources and the ApplicationSet controller must be installed in the same namespace as Argo CD. + ApplicationSet resources in a different namespace will be ignored. + +It is Argo CD itself that is responsible for the actual deployment of the generated child `Application` resources, such as Deployments, Services, and ConfigMaps. + +The ApplicationSet controller can thus be thought of as an `Application` 'factory', taking an `ApplicationSet` resource as input, and outputting one or more Argo CD `Application` resources that correspond to the parameters of that set. + +![ApplicationSet controller vs Argo CD, interaction diagram](../../assets/applicationset/Argo-CD-Integration/ApplicationSet-Argo-Relationship-v2.png) + +In this diagram an `ApplicationSet` resource is defined, and it is the responsibility of the ApplicationSet controller to create the corresponding `Application` resources. The resulting `Application` resources are then managed Argo CD: that is, Argo CD is responsible for actually deploying the child resources. + +Argo CD generates the application's Kubernetes resources based on the contents of the Git repository defined within the Application `spec` field, deploying e.g. Deployments, Service, and other resources. + +Creation, update, or deletion of ApplicationSets will have a direct effect on the Applications present in the Argo CD namespace. Likewise, cluster events (the addition/deletion of Argo CD cluster secrets, when using Cluster generator), or changes in Git (when using Git generator), will be used as input to the ApplicationSet controller in constructing `Application` resources. + +Argo CD and the ApplicationSet controller work together to ensure a consistent set of Application resources exist, and are deployed across the target clusters. diff --git a/docs/operator-manual/applicationset/Controlling-Resource-Modification.md b/docs/operator-manual/applicationset/Controlling-Resource-Modification.md new file mode 100644 index 0000000000000..3dd92936a41da --- /dev/null +++ b/docs/operator-manual/applicationset/Controlling-Resource-Modification.md @@ -0,0 +1,165 @@ +# Controlling if/when the ApplicationSet controller modifies `Application` resources + +The ApplicationSet controller supports a number of settings that limit the ability of the controller to make changes to generated Applications, for example, preventing the controller from deleting child Applications. + +These settings allow you to exert control over when, and how, changes are made to your Applications, and to their corresponding cluster resources (`Deployments`, `Services`, etc). + +Here are some of the controller settings that may be modified to alter the ApplicationSet controller's resource-handling behaviour. + +### Dry run: prevent ApplicationSet from creating, modifying, or deleting all Applications + +To prevent the ApplicationSet controller from creating, modifying, or deleting any `Application` resources, you may enable `dry-run` mode. This essentially switches the controller into a "read only" mode, where the controller Reconcile loop will run, but no resources will be modified. + +To enable dry-run, add `--dryrun true` to the ApplicationSet Deployment's container launch parameters. + +See 'How to modify ApplicationSet container parameters' below for detailed steps on how to add this parameter to the controller. + +### Policy - `create-only`: Prevent ApplicationSet controller from modifying or deleting Applications + +The ApplicationSet controller supports a parameter `--policy`, which is specified on launch (within the controller Deployment container), and which restricts what types of modifications will be made to managed Argo CD `Application` resources. + +The `--policy` parameter takes three values: `sync`, `create-only`, and `create-update`. (`sync` is the default, which is used if the `--policy` parameter is not specified; the other policies are described below). + +To allow the ApplicationSet controller to *create* `Application` resources, but prevent any further modification, such as deletion, or modification of Application fields, add this parameter in the ApplicationSet controller: +``` +--policy create-only +``` + +### Policy - `create-update`: Prevent ApplicationSet controller from deleting Applications + +To allow the ApplicationSet controller to create or modify `Application` resources, but prevent Applications from being deleted, add the following parameter to the ApplicationSet controller `Deployment`: +``` +--policy create-update +``` + +This may be useful to users looking for additional protection against deletion of the Applications generated by the controller. + +### Prevent an `Application`'s child resources from being deleted, when the parent Application is deleted + +By default, when an `Application` resource is deleted by the ApplicationSet controller, all of the child resources of the Application will be deleted as well (such as, all of the Application's `Deployments`, `Services`, etc). + +To prevent an Application's child resources from being deleted when the parent Application is deleted, add the `preserveResourcesOnDeletion: true` field to the `syncPolicy` of the ApplicationSet: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +spec: + # (...) + syncPolicy: + preserveResourcesOnDeletion: true +``` + +More information on the specific behaviour of `preserveResourcesOnDeletion`, and deletion in ApplicationSet controller and Argo CD in general, can be found on the [Application Deletion](Application-Deletion.md) page. + +### Prevent an Application's child resources from being modified + +Changes made to the ApplicationSet will propagate to the Applications managed by the ApplicationSet, and then Argo CD will propagate the Application changes to the underlying cluster resources (as per [Argo CD Integration](Argo-CD-Integration.md)). + +The propagation of Application changes to the cluster is managed by the [automated sync settings](../../user-guide/auto_sync.md), which are referenced in the ApplicationSet `template` field: + +- `spec.template.syncPolicy.automated`: If enabled, changes to Applications will automatically propagate to the cluster resources of the cluster. + - Unset this within the ApplicationSet template to 'pause' updates to cluster resources managed by the `Application` resource. +- `spec.template.syncPolicy.automated.prune`: By default, Automated sync will not delete resources when Argo CD detects the resource is no longer defined in Git. + - For extra safety, set this to false to prevent unexpected changes to the backing Git repository from affecting cluster resources. + + +## How to modify ApplicationSet container launch parameters + +There are a couple of ways to modify the ApplicationSet container parameters, so as to enable the above settings. + +### A) Use `kubectl edit` to modify the deployment on the cluster + +Edit the applicationset-controller `Deployment` resource on the cluster: +``` +kubectl edit deployment/argocd-applicationset-controller -n argocd +``` + +Locate the `.spec.template.spec.containers[0].command` field, and add the required parameter(s): +```yaml +spec: + # (...) + template: + # (...) + spec: + containers: + - command: + - entrypoint.sh + - argocd-applicationset-controller + # Insert new parameters here, for example: + # --policy create-only + # (...) +``` + +Save and exit the editor. Wait for a new `Pod` to start containing the updated parameters. + +### Or, B) Edit the `install.yaml` manifest for the ApplicationSet installation + +Rather than directly editing the cluster resource, you may instead choose to modify the installation YAML that is used to install the ApplicationSet controller: + +Applicable for applicationset versions less than 0.4.0. +```bash +# Clone the repository + +git clone https://github.com/argoproj/applicationset + +# Checkout the version that corresponds to the one you have installed. +git checkout "(version of applicationset)" +# example: git checkout "0.1.0" + +cd applicationset/manifests + +# open 'install.yaml' in a text editor, make the same modifications to Deployment +# as described in the previous section. + +# Apply the change to the cluster +kubectl apply -n argocd -f install.yaml +``` + +## Limitations: what isn't supported as of the current release + +Here is a list of commonly requested resource modification features which are not supported as of the current release. This lack of support is *not* necessarily by design; rather these behaviours are documented here to provide clear, concise descriptions of the current state of the feature. + +### Limitation: Control resource modification on a per ApplicationSet basis + +There is currently no way to restrict modification/deletion of the Applications that are owned by an *individual* ApplicationSet. The global `--policy` parameters described above only allow targeting of *all* ApplicationSets (eg it is 'all or nothing'). + +### Limitation: No support for manual edits to individual Applications + +There is currently no way to allow modification of a single child Application of an ApplicationSet, for example, if you wanted to make manual edits to a single Application for debugging/testing purposes. + +For example: + +- Imagine that you have an ApplicationSet that created Applications `app1`, `app2`, and `app3`. +- You now want to edit `app3` with `kubectl edit application/app3`, to update one of the `app3`'s fields. +- However, as soon as you make edits to `app3` (or any of the individual Applications), they will be immediately reverted by the ApplicationSet reconciler back to the `template`-ized version (by design). + +As of this writing, there is [an issue open](https://github.com/argoproj/applicationset/issues/186) for discussion of this behaviour. + + +### Limitation: ApplicationSet controller will not selectively ignore changes to individual fields + +There is currently no way to instruct the ApplicationSet controller to ignore changes to individual fields of Applications. + +For example, imagine that we have an Application created from an ApplicationSet, but a user has attempted to add a custom annotation (to the Application) that does not exist in the `ApplicationSet` resource: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + annotations: + # This annotation exists only on this Application, and not in + # the parent ApplicationSet template: + my-custom-annotation: some-value +spec: + # (...) +``` + +As above, the `ApplicationSet` resource does not have a `my-custom-annotation: some-value` annotation in the `.spec.template.annotations` for the Application. + +Since this field is not in the ApplicationSet template, as soon as a user adds this custom annotation, it will be immediately reverted (removed) by the ApplicationSet controller. + +There is currently no support for disabling or customizing this behaviour. + +To some extent this is by design: the main principle of ApplicationSets is that we maintain a 1-to-many relationship between the ApplicationSet and the Applications that it owns, such that all the Applications necessarily conform to a strict template. + +This provides the advantages of the 'cattle not pets' philosophy of microservice/cloud native application resource management, wherein you don't need to worry about individual Applications differing from each other in subtle ways: they will all necessarily be reconciled to be consistent with the parent template. + +BUT, admittedly, that is not always desirable 100% of the time, and there may be a better balance to be found, so discussions are continuing on GitHub and Slack. diff --git a/docs/operator-manual/applicationset/Generators-Cluster-Decision-Resource.md b/docs/operator-manual/applicationset/Generators-Cluster-Decision-Resource.md new file mode 100644 index 0000000000000..8f5bb491b8b44 --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Cluster-Decision-Resource.md @@ -0,0 +1,83 @@ +# Cluster Decision Resource Generator + +The cluster decision resource generates a list of Argo CD clusters. This is done using [duck-typing](https://pkg.go.dev/knative.dev/pkg/apis/duck), which does not require knowledge of the full shape of the referenced kubernetes resource. The following is an example of a cluster-decision-resource-based ApplicationSet generator: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook + namespace: argocd +spec: + generators: + - clusterDecisionResource: + # ConfigMap with GVK information for the duck type resource + configMapRef: my-configmap + name: quak # Choose either "name" of the resource or "labelSelector" + labelSelector: + matchLabels: # OPTIONAL + duck: spotted + matchExpressions: # OPTIONAL + - key: duck + operator: In + values: + - "spotted" + - "canvasback" + # OPTIONAL: Checks for changes every 60sec (default 3min) + requeueAfterSeconds: 60 + template: + metadata: + name: '{{name}}-guestbook' + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + targetRevision: HEAD + path: guestbook + destination: + server: '{{clusterName}}' # 'server' field of the secret + namespace: guestbook +``` +The `quak` resource, referenced by the ApplicationSet `clusterDecisionResource` generator: +```yaml +apiVersion: mallard.io/v1beta1 +kind: Duck +metadata: + name: quak +spec: {} +status: + # Duck-typing ignores all other aspects of the resource except + # the "decisions" list + decisions: + - clusterName: cluster-01 + - clusterName: cluster-02 +``` +The `ApplicationSet` resource references a `ConfigMap` that defines the resource to be used in this duck-typing. Only one ConfigMap is required per `ArgoCD` instance, to identify a resource. You can support multiple resource types by creating a `ConfigMap` for each. +```yaml +apiVersion: v1 +kind: ConfigMap +metadata: + name: my-configmap +data: + # apiVersion of the target resource + apiVersion: mallard.io/v1beta1 + # kind of the target resource + kind: ducks + # status key name that holds the list of Argo CD clusters + statusListKey: decisions + # The key in the status list whose value is the cluster name found in Argo CD + matchKey: clusterName +``` + +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/clusterDecisionResource).*) + +This example leverages the cluster management capabilities of the [open-cluster-management.io community](https://open-cluster-management.io/). By creating a `ConfigMap` with the GVK for the `open-cluster-management.io` Placement rule, your ApplicationSet can provision to different clusters in a number of novel ways. One example is to have the ApplicationSet maintain only two Argo CD Applications across 3 or more clusters. Then as maintenance or outages occur, the ApplicationSet will always maintain two Applications, moving the application to available clusters under the Placement rule's direction. + +## How it works +The ApplicationSet needs to be created in the Argo CD namespace, placing the `ConfigMap` in the same namespace allows the ClusterDecisionResource generator to read it. The `ConfigMap` stores the GVK information as well as the status key definitions. In the open-cluster-management example, the ApplicationSet generator will read the kind `placementrules` with an apiVersion of `apps.open-cluster-management.io/v1`. It will attempt to extract the **list** of clusters from the key `decisions`. It then validates the actual cluster name as defined in Argo CD against the **value** from the key `clusterName` in each of the elements in the list. + +The ClusterDecisionResource generator passes the 'name', 'server' and any other key/value in the duck-type resource's status list as parameters into the ApplicationSet template. In this example, the decision array contained an additional key `clusterName`, which is now available to the ApplicationSet template. + +!!! note "Clusters listed as `Status.Decisions` must be predefined in Argo CD" + The cluster names listed in the `Status.Decisions` *must* be defined within Argo CD, in order to generate applications for these values. The ApplicationSet controller does not create clusters within Argo CD. + + The Default Cluster list key is `clusters`. \ No newline at end of file diff --git a/docs/operator-manual/applicationset/Generators-Cluster.md b/docs/operator-manual/applicationset/Generators-Cluster.md new file mode 100644 index 0000000000000..b7685cd9ffe30 --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Cluster.md @@ -0,0 +1,161 @@ +# Cluster Generator + +In Argo CD, managed clusters [are stored within Secrets](../../declarative-setup/#clusters) in the Argo CD namespace. The ApplicationSet controller uses those same Secrets to generate parameters to identify and target available clusters. + +For each cluster registered with Argo CD, the Cluster generator produces parameters based on the list of items found within the cluster secret. + +It automatically provides the following parameter values to the Application template for each cluster: + +- `name` +- `nameNormalized` *('name' but normalized to contain only lowercase alphanumeric characters, '-' or '.')* +- `server` +- `metadata.labels.` *(for each label in the Secret)* +- `metadata.annotations.` *(for each annotation in the Secret)* + +!!! note + Use the `nameNormalized` parameter if your cluster name contains characters (such as underscores) that are not valid for Kubernetes resource names. This prevents rendering invalid Kubernetes resources with names like `my_cluster-app1`, and instead would convert them to `my-cluster-app1`. + + +Within [Argo CD cluster Secrets](../../declarative-setup/#clusters) are data fields describing the cluster: +```yaml +kind: Secret +data: + # Within Kubernetes these fields are actually encoded in Base64; they are decoded here for convenience. + # (They are likewise decoded when passed as parameters by the Cluster generator) + config: "{'tlsClientConfig':{'insecure':false}}" + name: "in-cluster2" + server: "https://kubernetes.default.svc" +metadata: + labels: + argocd.argoproj.io/secret-type: cluster +# (...) +``` + +The Cluster generator will automatically identify clusters defined with Argo CD, and extract the cluster data as parameters: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook + namespace: argocd +spec: + generators: + - clusters: {} # Automatically use all clusters defined within Argo CD + template: + metadata: + name: '{{name}}-guestbook' # 'name' field of the Secret + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + targetRevision: HEAD + path: guestbook + destination: + server: '{{server}}' # 'server' field of the secret + namespace: guestbook +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/cluster).*) + +In this example, the cluster secret's `name` and `server` fields are used to populate the `Application` resource `name` and `server` (which are then used to target that same cluster). + +### Label selector + +A label selector may be used to narrow the scope of targeted clusters to only those matching a specific label: +```yaml +kind: ApplicationSet +metadata: + name: guestbook + namespace: argocd +spec: + generators: + - clusters: + selector: + matchLabels: + staging: true + template: + # (...) +``` + +This would match an Argo CD cluster secret containing: +```yaml +kind: Secret +data: + # (... fields as above ...) +metadata: + labels: + argocd.argoproj.io/secret-type: cluster + staging: "true" +# (...) +``` + +The cluster selector also supports set-based requirements, as used by [several core Kubernetes resources](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#resources-that-support-set-based-requirements). + +### Deploying to the local cluster + +In Argo CD, the 'local cluster' is the cluster upon which Argo CD (and the ApplicationSet controller) is installed. This is to distinguish it from 'remote clusters', which are those that are added to Argo CD [declaratively](../../declarative-setup/#clusters) or via the [Argo CD CLI](../../getting_started.md/#5-register-a-cluster-to-deploy-apps-to-optional). + +The cluster generator will automatically target both local and non-local clusters, for every cluster that matches the cluster selector. + +If you wish to target only remote clusters with your Applications (e.g. you want to exclude the local cluster), then use a cluster selector with labels, for example: +```yaml +spec: + generators: + - clusters: + selector: + matchLabels: + argocd.argoproj.io/secret-type: cluster +``` + +This selector will not match the default local cluster, since the default local cluster does not have a Secret (and thus does not have the `argocd.argoproj.io/secret-type` label on that secret). Any cluster selector that selects on that label will automatically exclude the default local cluster. + +However, if you do wish to target both local and non-local clusters, while also using label matching, you can create a secret for the local cluster within the Argo CD web UI: + +1. Within the Argo CD web UI, select *Settings*, then *Clusters*. +2. Select your local cluster, usually named `in-cluster`. +3. Click the *Edit* button, and change the the *NAME* of the cluster to another value, for example `in-cluster-local`. Any other value here is fine. +4. Leave all other fields unchanged. +5. Click *Save*. + +These steps might seem counterintuitive, but the act of changing one of the default values for the local cluster causes the Argo CD Web UI to create a new secret for this cluster. In the Argo CD namespace, you should now see a Secret resource named `cluster-(cluster suffix)` with label `argocd.argoproj.io/secret-type": "cluster"`. You may also create a local [cluster secret declaratively](../../declarative-setup/#clusters), or with the CLI using `argocd cluster add "(context name)" --in-cluster`, rather than through the Web UI. + +### Pass additional key-value pairs via `values` field + +You may pass additional, arbitrary string key-value pairs via the `values` field of the cluster generator. Values added via the `values` field are added as `values.(field)` + +In this example, a `revision` parameter value is passed, based on matching labels on the cluster secret: +```yaml +spec: + generators: + - clusters: + selector: + matchLabels: + type: 'staging' + # A key-value map for arbitrary parameters + values: + revision: HEAD # staging clusters use HEAD branch + - clusters: + selector: + matchLabels: + type: 'production' + values: + # production uses a different revision value, for 'stable' branch + revision: stable + template: + metadata: + name: '{{name}}-guestbook' + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argocd-example-apps/ + # The cluster values field for each generator will be substituted here: + targetRevision: '{{values.revision}}' + path: guestbook + destination: + server: '{{server}}' + namespace: guestbook +``` + +In this example the `revision` value from the `generators.clusters` fields is passed into the template as `values.revision`, containing either `HEAD` or `stable` (based on which generator generated the set of parameters). + +!!! note + The `values.` prefix is always prepended to values provided via `generators.clusters.values` field. Ensure you include this prefix in the parameter name within the `template` when using it. diff --git a/docs/operator-manual/applicationset/Generators-Git.md b/docs/operator-manual/applicationset/Generators-Git.md new file mode 100644 index 0000000000000..6a287509ca4a2 --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Git.md @@ -0,0 +1,296 @@ +# Git Generator + +The Git generator contains two subtypes: the Git directory generator, and Git file generator. + +## Git Generator: Directories + +The Git directory generator, one of two subtypes of the Git generator, generates parameters using the directory structure of a specified Git repository. + +Suppose you have a Git repository with the following directory structure: +``` +├── argo-workflows +│ ├── kustomization.yaml +│ └── namespace-install.yaml +└── prometheus-operator + ├── Chart.yaml + ├── README.md + ├── requirements.yaml + └── values.yaml +``` + +This repository contains two directories, one for each of the workloads to deploy: + +- an Argo Workflow controller kustomization YAML file +- a Prometheus Operator Helm chart + +We can deploy both workloads, using this example: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-addons + namespace: argocd +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/git-generator-directory/cluster-addons/* + template: + metadata: + name: '{{path[0]}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: https://kubernetes.default.svc + namespace: '{{path.basename}}' +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/git-generator-directory).*) + +The generator parameters are: + +- `{{path}}`: The directory paths within the Git repository that match the `path` wildcard. +- `{{path[n]}}`: The directory paths within the Git repository that match the `path` wildcard, split into array elements (`n` - array index) +- `{{path.basename}}`: For any directory path within the Git repository that matches the `path` wildcard, the right-most path name is extracted (e.g. `/directory/directory2` would produce `directory2`). +- `{{path.basenameNormalized}}`: This field is the same as `path.basename` with unsupported characters replaced with `-` (e.g. a `path` of `/directory/directory_2`, and `path.basename` of `directory_2` would produce `directory-2` here). + +Whenever a new Helm chart/Kustomize YAML/Application/plain subfolder is added to the Git repository, the ApplicationSet controller will detect this change and automatically deploy the resulting manifests within new `Application` resources. + +As with other generators, clusters *must* already be defined within Argo CD, in order to generate Applications for them. + +### Exclude directories + +The Git directory generator will automatically exclude folders that begin with `.` (such as `.git`). + +The Git directory generator also supports an `exclude` option in order to exclude directories in the repository from being scanned by the ApplicationSet controller: + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-addons + namespace: argocd +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/git-generator-directory/excludes/cluster-addons/* + - path: applicationset/examples/git-generator-directory/excludes/cluster-addons/exclude-helm-guestbook + exclude: true + template: + metadata: + name: '{{path.basename}}' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: https://kubernetes.default.svc + namespace: '{{path.basename}}' +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/examples/applicationset/git-generator-directory/excludes).*) + +This example excludes the `exclude-helm-guestbook` directory from the list of directories scanned for this `ApplictionSet` resource. + +!!! note "Exclude rules have higher priority than include rules" + + If a directory matches at least one `exclude` pattern, it will be excluded. Or, said another way, *exclude rules take precedence over include rules.* + + As a corollary, which directories are included/excluded is not affected by the order of `path`s in the `directories` field list (because, as above, exclude rules always take precedence over include rules). + +For example, with these directories: + +``` +. +└── d + ├── e + ├── f + └── g +``` +Say you want to include `/d/e`, but exclude `/d/f` and `/d/g`. This will *not* work: + +```yaml +- path: /d/e + exclude: false +- path: /d/* + exclude: true +``` +Why? Because the exclude `/d/*` exclude rule will take precedence over the `/d/e` include rule. When the `/d/e` path in the Git repository is processed by the ApplicationSet controller, the controller detects that at least one exclude rule is matched, and thus that directory should not be scanned. + +You would instead need to do: + +```yaml +- path: /d/* +- path: /d/f + exclude: true +- path: /d/g + exclude: true +``` + +Or, a shorter way (using [path.Match](https://golang.org/pkg/path/#Match) syntax) would be: + +```yaml +- path: /d/* +- path: /d/[f|g] + exclude: true +``` + +## Git Generator: Files + +The Git file generator is the second subtype of the Git generator. The Git file generator generates parameters using the contents of JSON/YAML files found within a specified repository. + +Suppose you have a Git repository with the following directory structure: +``` +├── apps +│ └── guestbook +│ ├── guestbook-ui-deployment.yaml +│ ├── guestbook-ui-svc.yaml +│ └── kustomization.yaml +├── cluster-config +│ └── engineering +│ ├── dev +│ │ └── config.json +│ └── prod +│ └── config.json +└── git-generator-files.yaml +``` + +The folders are: + +- `guestbook` contains the Kubernetes resources for a simple guestbook application +- `cluster-config` contains JSON/YAML files describing the individual engineering clusters: one for `dev` and one for `prod`. +- `git-generator-files.yaml` is the example `ApplicationSet` resource that deploys `guestbook` to the specified clusters. + +The `config.json` files contain information describing the cluster (along with extra sample data): +```json +{ + "aws_account": "123456", + "asset_id": "11223344", + "cluster": { + "owner": "cluster-admin@company.com", + "name": "engineering-dev", + "address": "https://1.2.3.4" + } +} +``` + +Git commits containing changes to the `config.json` files are automatically discovered by the Git generator, and the contents of those files are parsed and converted into template parameters. Here are the parameters generated for the above JSON: +```text +aws_account: 123456 +asset_id: 11223344 +cluster.owner: cluster-admin@company.com +cluster.name: engineering-dev +cluster.address: https://1.2.3.4 +``` + + +And the generated parameters for all discovered `config.json` files will be substituted into ApplicationSet template: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook + namespace: argocd +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + files: + - path: "applicationset/examples/git-generator-files-discovery/cluster-config/**/config.json" + template: + metadata: + name: '{{cluster.name}}-guestbook' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: "applicationset/examples/git-generator-files-discovery/apps/guestbook" + destination: + server: '{{cluster.address}}' + namespace: guestbook +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/git-generator-files-discovery).*) + +Any `config.json` files found under the `cluster-config` directory will be parameterized based on the `path` wildcard pattern specified. Within each file JSON fields are flattened into key/value pairs, with this ApplicationSet example using the `cluster.address` as `cluster.name` parameters in the template. + +As with other generators, clusters *must* already be defined within Argo CD, in order to generate Applications for them. + +In addition to the flattened key/value pairs from the configuration file, the following generator parameters are provided: + +- `{{path}}`: The path to the folder containing matching configuration file within the Git repository. Example: `/clusters/clusterA`, if the config file was `/clusters/clusterA/config.json` +- `{{path[n]}}`: The path to the matching configuration file within the Git repository, split into array elements (`n` - array index). Example: `path[0]: clusters`, `path[1]: clusterA` +- `{{path.basename}}`: Basename of the path to the folder containing the configuration file (e.g. `clusterA`, with the above example.) +- `{{path.basenameNormalized}}`: This field is the same as `path.basename` with unsupported characters replaced with `-` (e.g. a `path` of `/directory/directory_2`, and `path.basename` of `directory_2` would produce `directory-2` here). + +## Webhook Configuration + +When using a Git generator, ApplicationSet polls Git repositories every three minutes to detect changes. To eliminate +this delay from polling, the ApplicationSet webhook server can be configured to receive webhook events. ApplicationSet supports +Git webhook notifications from GitHub and GitLab. The following explains how to configure a Git webhook for GitHub, but the same process should be applicable to other providers. + +!!! note + ApplicationSet exposes the webhook server as a service of type ClusterIP. An Ingress resource needs to be created to expose this service to the webhook source. + +### 1. Create the webhook in the Git provider + +In your Git provider, navigate to the settings page where webhooks can be configured. The payload +URL configured in the Git provider should use the `/api/webhook` endpoint of your ApplicationSet instance +(e.g. `https://applicationset.example.com/api/webhook`). If you wish to use a shared secret, input an +arbitrary value in the secret. This value will be used when configuring the webhook in the next step. + +![Add Webhook](../../assets/applicationset/webhook-config.png "Add Webhook") + +!!! note + When creating the webhook in GitHub, the "Content type" needs to be set to "application/json". The default value "application/x-www-form-urlencoded" is not supported by the library used to handle the hooks + +### 2. Configure ApplicationSet with the webhook secret (Optional) + +Configuring a webhook shared secret is optional, since ApplicationSet will still refresh applications +generated by Git generators, even with unauthenticated webhook events. This is safe to do since +the contents of webhook payloads are considered untrusted, and will only result in a refresh of the +application (a process which already occurs at three-minute intervals). If ApplicationSet is publicly +accessible, then configuring a webhook secret is recommended to prevent a DDoS attack. + +In the `argocd-secret` kubernetes secret, include the Git provider's webhook secret configured in step 1. + +Edit the Argo CD kubernetes secret: + +```bash +kubectl edit secret argocd-secret -n argocd +``` + +TIP: for ease of entering secrets, kubernetes supports inputting secrets in the `stringData` field, +which saves you the trouble of base64 encoding the values and copying it to the `data` field. +Simply copy the shared webhook secret created in step 1, to the corresponding +GitHub/GitLab/BitBucket key under the `stringData` field: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: argocd-secret + namespace: argocd +type: Opaque +data: +... + +stringData: + # github webhook secret + webhook.github.secret: shhhh! it's a github secret + + # gitlab webhook secret + webhook.gitlab.secret: shhhh! it's a gitlab secret +``` + +After saving, please restart the ApplicationSet pod for the changes to take effect. diff --git a/docs/operator-manual/applicationset/Generators-List.md b/docs/operator-manual/applicationset/Generators-List.md new file mode 100644 index 0000000000000..8c2721a2fb36b --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-List.md @@ -0,0 +1,55 @@ +# List Generator + +The List generator generates parameters based on an arbitrary list of key/value pairs (as long as the values are string values). In this example, we're targeting a local cluster named `engineering-dev`: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook + namespace: argocd +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc +# - cluster: engineering-prod +# url: https://kubernetes.default.svc +# foo: bar + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: default + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: applicationset/examples/list-generator/guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/list-generator).*) + +In this example, the List generator passes the `url` and `cluster` fields as parameters into the template. If we wanted to add a second environment, we could uncomment the second element and the ApplicationSet controller would automatically target it with the defined application. + +With the ApplicationSet v0.1.0 release, one could *only* specify `url` and `cluster` element fields (plus arbitrary `values`). As of ApplicationSet v0.2.0, any key/value `element` pair is supported (which is also fully backwards compatible with the v0.1.0 form): +```yaml +spec: + generators: + - list: + elements: + # v0.1.0 form - requires cluster/url keys: + - cluster: engineering-dev + url: https://kubernetes.default.svc + values: + additional: value + # v0.2.0+ form - does not require cluster/URL keys + # (but they are still supported). + - staging: "true" + gitRepo: https://kubernetes.default.svc +# (...) +``` + +!!! note "Clusters must be predefined in Argo CD" + These clusters *must* already be defined within Argo CD, in order to generate applications for these values. The ApplicationSet controller does not create clusters within Argo CD (for instance, it does not have the credentials to do so). diff --git a/docs/operator-manual/applicationset/Generators-Matrix.md b/docs/operator-manual/applicationset/Generators-Matrix.md new file mode 100644 index 0000000000000..4cc149060241b --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Matrix.md @@ -0,0 +1,138 @@ +# Matrix Generator + +The Matrix generator combines the parameters generated by two child generators, iterating through every combination of each generator's generated parameters. + +By combining both generators parameters, to produce every possible combination, this allows you to gain the instrinsic properties of both generators. For example, a small subset of the many possible use cases include: + +- *SCM Provider Generator + Cluster Generator*: Scanning the repositories of a GitHub organization for application resources, and targeting those resources to all available clusters. +- *Git File Generator + List Generator*: Providing a list of applicatations to deploy via configuration files, with optional configuration options, and deploying them to a fixed list of clusters. +- *Git Directory Generator + Cluster Decision Resource Generator*: Locate application resources contained within folders of a Git repository, and deploy them to a list of clusters provided via an external custom resource. +- And so on... + +Any set of generators may be used, with the combined values of those generators inserted into the `template` parameters, as usual. + +## Example: Git Directory generator + Cluster generator + +As an example, imagine that we have two clusters: + +- A `staging` cluster (at `https://1.2.3.4`) +- A `production` cluster (at `https://2.4.6.8`) + +And our application YAMLs are defined in a Git repository: + +- Argo Workflows controller (examples/git-generator-directory/cluster-addons/argo-workflows) +- Prometheus operator (/examples/git-generator-directory/cluster-addons/prometheus-operator) + +Our goal is to deploy both applications onto both clusters, and, more generally, in the future to automatically deploy new applications in the Git repository, and to new clusters defined within Argo CD, as well. + +For this we will use the Matrix generator, with the Git and the Cluster as child generators: + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-git +spec: + generators: + # matrix 'parent' generator + - matrix: + generators: + # git generator, 'child' #1 + - git: + repoURL: https://github.com/argoproj/argo-cd.git + revision: HEAD + directories: + - path: applicationset/examples/matrix/cluster-addons/* + # cluster generator, 'child' #2 + - clusters: + selector: + matchLabels: + argocd.argoproj.io/secret-type: cluster + template: + metadata: + name: '{{path.basename}}-{{name}}' + spec: + project: '{{metadata.labels.environment}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: '{{path}}' + destination: + server: '{{server}}' + namespace: '{{path.basename}}' +``` + +First, the Git directory generator will scan the Git repository, discovering directories under the specified path. It discovers the argo-workflows and prometheus-operator applications, and produces two corresponding sets of parameters: +```yaml +- path: /examples/git-generator-directory/cluster-addons/argo-workflows + path.basename: argo-workflows + +- path: /examples/git-generator-directory/cluster-addons/prometheus-operator + path.basename: prometheus-operator +``` + +Next, the Cluster generator scans the [set of clusters defined in Argo CD](Generators-Cluster.md), finds the staging and production cluster secrets, and produce two corresponding sets of parameters: +```yaml +- name: staging + server: https://1.2.3.4 + +- name: production + server: https://2.4.6.8 +``` + +Finally, the Matrix generator will combine both sets of outputs, and produce: +```yaml +- name: staging + server: https://1.2.3.4 + path: /examples/git-generator-directory/cluster-addons/argo-workflows + path.basename: argo-workflows + +- name: staging + server: https://1.2.3.4 + path: /examples/git-generator-directory/cluster-addons/prometheus-operator + path.basename: prometheus-operator + +- name: production + server: https://2.4.6.8 + path: /examples/git-generator-directory/cluster-addons/argo-workflows + path.basename: argo-workflows + +- name: production + server: https://2.4.6.8 + path: /examples/git-generator-directory/cluster-addons/prometheus-operator + path.basename: prometheus-operator +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/matrix).*) + +## Restrictions + +1. The Matrix generator currently only supports combining the outputs of only two child generators (eg does not support generating combinations for 3 or more). +1. You should specify only a single generator per array entry, eg this is not valid: +```yaml +- matrix: + generators: + - list: # (...) + git: # (...) +``` + - While this *will* be accepted by Kubernetes API validation, the controller will report an error on generation. Each generator should be specified in a separate array element, as in the examples above. +1. The Matrix generator does not currently support [`template` overrides](Template.md#generator-templates) specified on child generators, eg this `template` will not be processed: +```yaml +- matrix: + generators: + - list: + elements: + - # (...) + template: { } # Not processed +``` +1. Combination-type generators (matrix or merge) can only be nested once. For example, this will not work: +```yaml +- matrix: + generators: + - matrix: + generators: + - matrix: # This third level is invalid. + generators: + - list: + elements: + - # (...) +``` diff --git a/docs/operator-manual/applicationset/Generators-Merge.md b/docs/operator-manual/applicationset/Generators-Merge.md new file mode 100644 index 0000000000000..fd2501d26e17a --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Merge.md @@ -0,0 +1,144 @@ +# Merge Generator + +The Merge generator combines parameters produced by the base (first) generator with matching parameter sets produced by subsequent generators. A _matching_ parameter set has the same values for the configured _merge keys_. _Non-matching_ parameter sets are discarded. Override precedence is bottom-to-top: the values from a matching parameter set produced by generator 3 will take precedence over the values from the corresponding parameter set produced by generator 2. + +Using a Merge generator is appropriate when a subset of parameter sets require overriding. + +## Example: Base Cluster generator + override Cluster generator + List generator + +As an example, imagine that we have two clusters: + +- A `staging` cluster (at `https://1.2.3.4`) +- A `production` cluster (at `https://2.4.6.8`) + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: cluster-git +spec: + generators: + # merge 'parent' generator + - merge: + mergeKeys: + - server + generators: + - clusters: + values: + kafka: 'true' + redis: 'false' + # For clusters with a specific label, enable Kafka. + - clusters: + selector: + matchLabels: + use-kafka: 'false' + values: + kafka: 'false' + # For a specific cluster, enable Redis. + - list: + elements: + - server: https://2.4.6.8 + values.redis: 'true' + template: + metadata: + name: '{{name}}' + spec: + project: '{{metadata.labels.environment}}' + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + path: app + helm: + parameters: + - name: kafka + value: '{{values.kafka}}' + - name: redis + value: '{{values.redis}}' + destination: + server: '{{server}}' + namespace: default +``` + +The base Cluster generator scans the [set of clusters defined in Argo CD](Generators-Cluster.md), finds the staging and production cluster secrets, and produces two corresponding sets of parameters: +```yaml +- name: staging + server: https://1.2.3.4 + values.kafka: 'true' + values.redis: 'false' + +- name: production + server: https://2.4.6.8 + values.kafka: 'true' + values.redis: 'false' +``` + +The override Cluster generator scans the [set of clusters defined in Argo CD](Generators-Cluster.md), finds the staging cluster secret (which has the required label), and produces the following parameters: +```yaml +- name: staging + server: https://1.2.3.4 + values.kafka: 'false' +``` + +When merged with the base generator's parameters, the `values.kafka` value for the staging cluster is set to `'false'`. +```yaml +- name: staging + server: https://1.2.3.4 + values.kafka: 'false' + values.redis: 'false' + +- name: production + server: https://2.4.6.8 + values.kafka: 'true' + values.redis: 'false' +``` + +Finally, the List cluster generates a single set of parameters: +```yaml +- server: https://2.4.6.8 + values.redis: 'true' +``` + +When merged with the updated base parameters, the `values.redis` value for the production cluster is set to `'true'`. This is the merge generator's final output: +```yaml +- name: staging + server: https://1.2.3.4 + values.kafka: 'false' + values.redis: 'false' + +- name: production + server: https://2.4.6.8 + values.kafka: 'true' + values.redis: 'true' +``` + +## Restrictions + +1. You should specify only a single generator per array entry. This is not valid: +```yaml +- merge: + generators: + - list: # (...) + git: # (...) +``` + - While this *will* be accepted by Kubernetes API validation, the controller will report an error on generation. Each generator should be specified in a separate array element, as in the examples above. +1. The Merge generator does not support [`template` overrides](Template.md#generator-templates) specified on child generators. This `template` will not be processed: +```yaml +- merge: + generators: + - list: + elements: + - # (...) + template: { } # Not processed +``` +1. Combination-type generators (Matrix or Merge) can only be nested once. For example, this will not work: +```yaml +- merge: + generators: + - merge: + generators: + - merge: # This third level is invalid. + generators: + - list: + elements: + - # (...) +``` diff --git a/docs/operator-manual/applicationset/Generators-Pull-Request.md b/docs/operator-manual/applicationset/Generators-Pull-Request.md new file mode 100644 index 0000000000000..5336f3d2b20c2 --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-Pull-Request.md @@ -0,0 +1,111 @@ +# Pull Request Generator + +The Pull Request generator uses the API of an SCMaaS provider (eg GitHub/GitLab) to automatically discover open pull requests within an repository. This fits well with the style of building a test environment when you create a pull request. + + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - pullRequest: + # See below for provider specific options. + github: + # ... +``` + +## GitHub + +Specify the repository from which to fetch the Github Pull requests. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - pullRequest: + github: + # The GitHub organization or user. + owner: myorg + # The Github repository + repo: myrepository + # For GitHub Enterprise (optional) + api: https://git.example.com/ + # Reference to a Secret containing an access token. (optional) + tokenRef: + secretName: github-token + key: token + # Labels is used to filter the PRs that you want to target. (optional) + labels: + - preview + requeueAfterSeconds: 1800 + template: + # ... +``` + +* `owner`: Required name of the GitHub organization or user. +* `repo`: Required name of the Github repositry. +* `api`: If using GitHub Enterprise, the URL to access it. (Optional) +* `tokenRef`: A `Secret` name and key containing the GitHub access token to use for requests. If not specified, will make anonymous requests which have a lower rate limit and can only see public repositories. (Optional) +* `labels`: Labels is used to filter the PRs that you want to target. (Optional) + +## Template + +As with all generators, several keys are available for replacement in the generated application. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - pullRequest: + # ... + template: + metadata: + name: 'myapp-{{branch}}-{{number}}' + spec: + source: + repoURL: 'https://github.com/myorg/myrepo.git' + targetRevision: '{{head_sha}}' + path: kubernetes/ + helm: + parameters: + - name: "image.tag" + value: "pull-{{head_sha}}" + project: default + destination: + server: https://kubernetes.default.svc + namespace: default +``` + +* `number`: The ID number of the pull request. +* `branch`: The name of the branch of the pull request head. +* `head_sha`: This is the SHA of the head of the pull request. + +## Webhook Configuration + +When using a Pull Request generator, the ApplicationSet controller polls every `requeueAfterSeconds` interval (defaulting to every 30 minutes) to detect changes. To eliminate this delay from polling, the ApplicationSet webhook server can be configured to receive webhook events, which will trigger Application generation by the Pull Request generator. + +The configuration is almost the same as the one described [in the Git generator](Generators-Git.md), but there is one difference: if you want to use the Pull Request Generator as well, additionally configure the following settings. + +In section 1, _"Create the webhook in the Git provider"_, add an event so that a webhook request will be sent when a pull request is created, closed, or label changed. +Select `Let me select individual events` and enable the checkbox for `Pull requests`. + +![Add Webhook](../../assets/applicationset/webhook-config-pull-request.png "Add Webhook Pull Request") + +The Pull Request Generator will requeue when the next action occurs. + +- `opened` +- `closed` +- `reopened` +- `labeled` +- `unlabeled` +- `synchronized` + +For more information about each event, please refer to the [official documentation](https://docs.github.com/en/developers/webhooks-and-events/webhooks/webhook-events-and-payloads). diff --git a/docs/operator-manual/applicationset/Generators-SCM-Provider.md b/docs/operator-manual/applicationset/Generators-SCM-Provider.md new file mode 100644 index 0000000000000..4107e4ac6563a --- /dev/null +++ b/docs/operator-manual/applicationset/Generators-SCM-Provider.md @@ -0,0 +1,158 @@ +# SCM Provider Generator + +The SCM Provider generator uses the API of an SCMaaS provider (eg GitHub) to automatically discover repositories within an organization. This fits well with GitOps layout patterns that split microservices across many repositories. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - scmProvider: + # Which protocol to clone using. + cloneProtocol: ssh + # See below for provider specific options. + github: + # ... +``` + +* `cloneProtocol`: Which protocol to use for the SCM URL. Default is provider-specific but ssh if possible. Not all providers necessarily support all protocols, see provider documentation below for available options. + +## GitHub + +The GitHub mode uses the GitHub API to scan and organization in either github.com or GitHub Enterprise. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - scmProvider: + github: + # The GitHub organization to scan. + organization: myorg + # For GitHub Enterprise: + api: https://git.example.com/ + # If true, scan every branch of every repository. If false, scan only the default branch. Defaults to false. + allBranches: true + # Reference to a Secret containing an access token. (optional) + tokenRef: + secretName: github-token + key: token + template: + # ... +``` + +* `organization`: Required name of the GitHub organization to scan. If you have multiple orgs, use multiple generators. +* `api`: If using GitHub Enterprise, the URL to access it. +* `allBranches`: By default (false) the template will only be evaluated for the default branch of each repo. If this is true, every branch of every repository will be passed to the filters. If using this flag, you likely want to use a `branchMatch` filter. +* `tokenRef`: A `Secret` name and key containing the GitHub access token to use for requests. If not specified, will make anonymous requests which have a lower rate limit and can only see public repositories. + +For label filtering, the repository topics are used. + +Available clone protocols are `ssh` and `https`. + +## Gitlab + +The Gitlab mode uses the Gitlab API to scan and organization in either gitlab.com or self-hosted gitlab. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - scmProvider: + gitlab: + # The base Gitlab group to scan. You can either use the group id or the full namespaced path. + group: "8675309" + # For GitLab Enterprise: + api: https://gitlab.example.com/ + # If true, scan every branch of every repository. If false, scan only the default branch. Defaults to false. + allBranches: true + # If true, recurses through subgroups. If false, it searches only in the base group. Defaults to false. + includeSubgroups: true + # Reference to a Secret containing an access token. (optional) + tokenRef: + secretName: gitlab-token + key: token + template: + # ... +``` + +* `group`: Required name of the base Gitlab group to scan. If you have multiple base groups, use multiple generators. +* `api`: If using GitHub Enterprise, the URL to access it. +* `allBranches`: By default (false) the template will only be evaluated for the default branch of each repo. If this is true, every branch of every repository will be passed to the filters. If using this flag, you likely want to use a `branchMatch` filter. +* `includeSubgroups`: By default (false) the controller will only search for repos directly in the base group. If this is true, it will recurse through all the subgroups searching for repos to scan. +* `tokenRef`: A `Secret` name and key containing the Gitlab access token to use for requests. If not specified, will make anonymous requests which have a lower rate limit and can only see public repositories. + +For label filtering, the repository tags are used. + +Available clone protocols are `ssh` and `https`. + +## Filters + +Filters allow selecting which repositories to generate for. Each filter can declare one or more conditions, all of which must pass. If multiple filters are present, any can match for a repository to be included. If no filters are specified, all repositories will be processed. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - scmProvider: + filters: + # Include any repository starting with "myapp" AND including a Kustomize config AND labeled with "deploy-ok" ... + - repositoryMatch: ^myapp + pathsExist: [kubernetes/kustomization.yaml] + labelMatch: deploy-ok + # ... OR any repository starting with "otherapp" AND a Helm folder. + - repositoryMatch: ^otherapp + pathsExist: [helm] + template: + # ... +``` + +* `repositoryMatch`: A regexp matched against the repository name. +* `pathsExist`: An array of paths within the repository that must exist. Can be a file or directory, but do not include the trailing `/` for directories. +* `labelMatch`: A regexp matched against repository labels. If any label matches, the repository is included. +* `branchMatch`: A regexp matched against branch names. + +## Template + +As with all generators, several parameters are generated for use within the `ApplicationSet` resource template. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: myapps +spec: + generators: + - scmProvider: + # ... + template: + metadata: + name: '{{ repository }}' + spec: + source: + repoURL: '{{ url }}' + targetRevision: '{{ branch }}' + path: kubernetes/ + project: default + destination: + server: https://kubernetes.default.svc + namespace: default +``` + +* `organization`: The name of the organization the repository is in. +* `repository`: The name of the repository. +* `url`: The clone URL for the repository. +* `branch`: The default branch of the repository. +* `sha`: The Git commit SHA for the branch +* `labels`: A comma-separated list of repository labels \ No newline at end of file diff --git a/docs/operator-manual/applicationset/Generators.md b/docs/operator-manual/applicationset/Generators.md new file mode 100644 index 0000000000000..12e6440818fe7 --- /dev/null +++ b/docs/operator-manual/applicationset/Generators.md @@ -0,0 +1,18 @@ +# Generators + +Generators are responsible for generating *parameters*, which are then rendered into the `template:` fields of the ApplicationSet resource. See the [Introduction](index.md) for an example of how generators work with templates, to create Argo CD Applications. + +Generators are primarily based on the data source that they use to generate the template parameters. For example: the List generator provides a set of parameters from a *literal list*, the Cluster generator uses the *Argo CD cluster list* as a source, the Git generator uses files/directories from a *Git repository*, and so. + +As of this writing there are seven generators: + +- [List generator](Generators-List.md): The List generator allows you to target Argo CD Applications to clusters based on a fixed list of cluster name/URL values. +- [Cluster generator](Generators-Cluster.md): The Cluster generator allows you to target Argo CD Applications to clusters, based on the list of clusters defined within (and managed by) Argo CD (which includes automatically responding to cluster addition/removal events from Argo CD). +- [Git generator](Generators-Git.md): The Git generator allows you to create Applications based on files within a Git repository, or based on the directory structure of a Git repository. +- [Matrix generator](Generators-Matrix.md): The Matrix generator may be used to combine the generated parameters of two separate generators. +- [Merge generator](Generators-Merge.md): The Merge generator may be used to merge the generated parameters of two or more generators. Additional generators can override the values of the base generator. +- [SCM Provider generator](Generators-SCM-Provider.md): The SCM Provider generator uses the API of an SCM provider (eg GitHub) to automatically discover repositories within an organization. +- [Pull Request generator](Generators-Pull-Request.md): The Pull Request generator uses the API of an SCMaaS provider (eg GitHub) to automatically discover open pull requests within an repository. +- [Cluster Decision Resource generator](Generators-Cluster-Decision-Resource.md): The Cluster Decision Resource generator is used to interface with Kubernetes custom resources that use custom resource-specific logic to decide which set of Argo CD clusters to deploy to. + +If you are new to generators, begin with the **List** and **Cluster** generators. For more advanced use cases, see the documentation for the remaining generators above. diff --git a/docs/operator-manual/applicationset/Getting-Started.md b/docs/operator-manual/applicationset/Getting-Started.md new file mode 100644 index 0000000000000..2ff509b1230fe --- /dev/null +++ b/docs/operator-manual/applicationset/Getting-Started.md @@ -0,0 +1,107 @@ +# Getting Started + +This guide assumes you are familiar with Argo CD and its basic concepts. See the [Argo CD documentation](../../core_concepts.md) for more information. + +## Requirements + +* Installed [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) command-line tool +* Have a [kubeconfig](https://kubernetes.io/docs/tasks/access-application-cluster/configure-access-multiple-clusters/) file (default location is `~/.kube/config`). + +## Installation + +There are a few options for installing the ApplicationSet controller. + + +### A) Install ApplicationSet as part of Argo CD + +Starting with Argo CD v2.3, the ApplicationSet controller is bundled with Argo CD. It is no longer necessary to install the ApplicationSet controller separately from Argo CD. + +Follow the [Argo CD Getting Started](../../getting_started.md) instructions for more information. + + + +### B) Install ApplicationSet into an existing Argo CD install (pre-Argo CD v2.3) + +**Note**: These instruction only apply to versions of Argo CD before v2.3.0. + +The ApplicationSet controller *must* be installed into the same namespace as the Argo CD it is targetting. + +Presuming that Argo CD is installed into the `argocd` namespace, run the following command: + +```bash +kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/applicationset/v0.4.0/manifests/install.yaml +``` + +Once installed, the ApplicationSet controller requires no additional setup. + +The `manifests/install.yaml` file contains the Kubernetes manifests required to install the ApplicationSet controller: + +- CustomResourceDefinition for `ApplicationSet` resource +- Deployment for `argocd-applicationset-controller` +- ServiceAccount for use by ApplicationSet controller, to access Argo CD resources +- Role granting RBAC access to needed resources, for ServiceAccount +- RoleBinding to bind the ServiceAccount and Role + + + + + + +## Enabling high availability mode + +To enable high availability, you have to set the command ``` --enable-leader-election=true ``` in argocd-applicationset-controller container and increase the replicas. + +do following changes in manifests/install.yaml + +```bash + spec: + containers: + - command: + - entrypoint.sh + - argocd-applicationset-controller + - --enable-leader-election=true +``` + +### Optional: Additional Post-Upgrade Safeguards + +See the [Controlling Resource Modification](Controlling-Resource-Modification.md) page for information on additional parameters you may wish to add to the ApplicationSet Resource in `install.yaml`, to provide extra security against any initial, unexpected post-upgrade behaviour. + +For instance, to temporarily prevent the upgraded ApplicationSet controller from making any changes, you could: + +- Enable dry-run +- Use a create-only policy +- Enable `preserveResourcesOnDeletion` on your ApplicationSets +- Temporarily disable automated sync in your ApplicationSets' template + +These parameters would allow you to observe/control the behaviour of the new version of the ApplicationSet controller in your environment, to ensure you are happy with the result (see the ApplicationSet log file for details). Just don't forget to remove any temporary changes when you are done testing! + +However, as mentioned above, these steps are not strictly necessary: upgrading the ApplicationSet controller should be a minimally invasive process, and these are only suggested as an optional precaution for extra safety. + +## Next Steps + +Once your ApplicationSet controller is up and running, proceed to [Use Cases](Use-Cases.md) to learn more about the supported scenarios, or proceed directly to [Generators](Generators.md) to see example `ApplicationSet` resources. diff --git a/docs/operator-manual/applicationset/Template.md b/docs/operator-manual/applicationset/Template.md new file mode 100644 index 0000000000000..6ab3076f0497b --- /dev/null +++ b/docs/operator-manual/applicationset/Template.md @@ -0,0 +1,108 @@ +# Templates + +The template fields of the ApplicationSet `spec` are used to generate Argo CD `Application` resources. + +## Template fields + +An Argo CD Application is created by combining the parameters from the generator with fields of the template (via `{{values}}`), and from that a concrete `Application` resource is produced and applied to the cluster. + +Here is the template subfield from a Cluster generator: +```yaml +# (...) + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook +``` + +The template subfields correspond directly to [the spec of an Argo CD `Application` resource](../../declarative-setup/#applications): + +- `project` refers to the [Argo CD Project](../../user-guide/projects.md) in use (`default` may be used here to utilize the default Argo CD Project) +- `source` defines from which Git repostory to extract the desired Application manifests + - **repoURL**: URL of the repository (eg `https://github.com/argoproj/argocd-example-apps.git`) + - **targetRevision**: Revision (tag/branch/commit) of the repository (eg `HEAD`) + - **path**: Path within the repository where Kubernetes manifests (and/or Helm, Kustomize, Jsonnet resources) are located +- `destination`: Defines which Kubernetes cluster/namespace to deploy to + - **name**: Name of the cluster (within Argo CD) to deploy to + - **server**: API Server URL for the cluster (Example: `https://kubernetes.default.svc`) + - **namespace**: Target namespace in which to deploy the manifests from `source` (Example: `my-app-namespace`) + +Note: + +- Referenced clusters must already be defined in Argo CD, for the ApplicationSet controller to use them +- Only **one** of `name` or `server` may be specified: if both are specified, an error is returned. + +The `metadata` field of template may also be used to set an Application `name`, or to add labels or annotations to the Application. + +While the ApplicationSet spec provides a basic form of templating, it is not intended to replace the full-fledged configuration management capabilities of tools such as Kustomize, Helm, or Jsonnet. + +### Deploying ApplicationSet resources as part of a Helm chart + +ApplicationSet uses the same templating notation as Helm (`{{}}`). If the ApplicationSet templates aren't written as +Helm string literals, Helm will throw an error like `function "cluster" not defined`. To avoid that error, write the +template as a Helm string literal. For example: + +```yaml + metadata: + name: '{{`{{cluster}}`}}-guestbook' +``` + +This _only_ applies if you use Helm to deploy your ApplicationSet resources. + +## Generator templates + +In addition to specifying a template within the `.spec.template` of the `ApplicationSet` resource, templates may also be specified within generators. This is useful for overriding the values of the `spec`-level template. + +The generator's `template` field takes precedence over the `spec`'s template fields: + +- If both templates contain the same field, the generator's field value will be used. +- If only one of those templates' fields has a value, that value will be used. + +Generator templates can thus be thought of as patches against the outer `spec`-level template fields. + +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://kubernetes.default.svc + template: + metadata: {} + spec: + project: "default" + source: + revision: HEAD + repoURL: https://github.com/argoproj/argo-cd.git + # New path value is generated here: + path: 'applicationset/examples/template-override/{{cluster}}-override' + destination: {} + + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + project: "default" + source: + repoURL: https://github.com/argoproj/argo-cd.git + targetRevision: HEAD + # This 'default' value is not used: it is is replaced by the generator's template path, above + path: applicationset/examples/template-override/default + destination: + server: '{{url}}' + namespace: guestbook +``` +(*The full example can be found [here](https://github.com/argoproj/argo-cd/tree/master/applicationset/examples/template-override).*) + +In this example, the ApplicationSet controller will generate an `Application` resource using the `path` generated by the List generator, rather than the `path` value defined in `.spec.template`. diff --git a/docs/operator-manual/applicationset/Use-Cases.md b/docs/operator-manual/applicationset/Use-Cases.md new file mode 100644 index 0000000000000..025607eea4f1e --- /dev/null +++ b/docs/operator-manual/applicationset/Use-Cases.md @@ -0,0 +1,92 @@ +# Use cases supported by the ApplicationSet controller + +With the concept of generators, the ApplicationSet controller provides a powerful set of tools to automate the templating and modification of Argo CD Applications. Generators produce template parameter data from a variety of sources, including Argo CD clusters and Git repositories, supporting and enabling new use cases. + +While these tools may be utilized for whichever purpose is desired, here are some of the specific use cases that the ApplicationSet controller was designed to support. + +## Use case: cluster add-ons + +An initial design focus of the ApplicationSet controller was to allow an infrastructure team's Kubernetes cluster administrators the ability to automatically create a large, diverse set of Argo CD Applications, across a significant number of clusters, and manage those Applications as a single unit. One example of why this is needed is the *cluster add-on use case*. + +In the *cluster add-on use case*, an administrator is responsible for provisioning cluster add-ons to one or more Kubernete clusters: cluster-addons are operators such as the [Prometheus operator](https://github.com/prometheus-operator/prometheus-operator), or controllers such as the [argo-workflows controller](https://argoproj.github.io/argo-workflows/) (part of the [Argo ecosystem](https://argoproj.github.io/)). + +Typically these add-ons are required by the applications of development teams (as tenants of a multi-tenant cluster, for instance, they may wish to provide metrics data to Prometheus or orchestrate workflows via Argo Workflows). + +Since installing these add-ons requires cluster-level permissions not held by individual development teams, installation is the responsibility of the infrastructure/ops team of an organization, and within a large organization this team might be responsible for tens, hundreds, or thousands of Kubernetes clusters (with new clusters being added/modified/removed on a regular basis). + +The need to scale across a large number of clusters, and automatically respond to the lifecycle of new clusters, necessarily mandates some form of automation. A further requirement would be allowing the targeting of add-ons to a subset of clusters using specific criteria (eg staging vs production). + +![Cluster add-on diagram](../../assets/applicationset/Use-Cases/Cluster-Add-Ons.png) + +In this example, the infrastructure team maintains a Git repository containing application manifests for the Argo Workflows controller, and Prometheus operator. + +The infrastructure team would like to deploy both these add-on to a large number of clusters, using Argo CD, and likewise wishes to easily manage the creation/deletion of new clusters. + +In this use case, we may use either the List, Cluster, or Git generators of the ApplicationSet controller to provide the required behaviour: + +- *List generator*: Administrators maintain two `ApplicationSet` resources, one for each application (Workflows and Prometheus), and include the list of clusters they wish to target within the List generator elements of each. + - With this generator, adding/removing clusters requires manually updating the `ApplicationSet` resource's list elements. +- *Cluster generator*: Administrators maintain two `ApplicationSet` resources, one for each application (Workflows and Prometheus), and ensure that all new cluster are defined within Argo CD. + - Since the Cluster generator automatically detects and targets the clusters defined within Argo CD, [adding/remove a cluster from Argo CD](../../declarative-setup/#clusters) will automatically cause Argo CD Application resources (for each application) to be created by the ApplicationSet controller. +- *Git generator*: The Git generator is the most flexible/powerful of the generators, and thus there are a number of different ways to tackle this use case. Here are a couple: + - Using the Git generator `files` field: A list of clusters is kept as a JSON file within a Git repository. Updates to the JSON file, through Git commits, cause new clusters to be added/removed. + - Using the Git generator `directories` field: For each target cluster, a corresponding directory of that name exists in a Git repository. Adding/modifying a directory, through Git commits, would trigger an update for the cluster that has shares the directory name. + +See the [generators section](Generators.md) for details on each of the generators. + +## Use case: monorepos + +In the *monorepo use case*, Kubernetes cluster administrators manage the entire state of a single Kubernetes cluster from a single Git repository. + +Manifest changes merged into the Git repository should automatically deploy to the cluster. + +![Monorepo diagram](../../assets/applicationset/Use-Cases/Monorepos.png) + +In this example, the infrastructure team maintains a Git repository containing application manifests for an Argo Workflows controller, and a Prometheus operator. Independent development teams also have added additional services they wish to deploy to the cluster. + +Changes made to the Git repository -- for example, updating the version of a deployed artifact -- should automatically cause that update to be applied to the corresponding Kubernetes cluster by Argo CD. + +The Git generator may be used to support this use case: + +- The Git generator `directories` field may be used to specify particular subdirectories (using wildcards) containing the individual applications to deploy. +- The Git generator `files` field may reference Git repository files containing JSON metadata, with that metadata describing the individual applications to deploy. +- See the Git generator documentation for more details. + +## Use case: self-service of Argo CD Applications on multitenant clusters + +The *self-service use case* seeks to allow developers (as the end users of a multitenant Kubernetes cluster) greater flexibility to: + +- Deploy multiple applications to a single cluster, in an automated fashion, using Argo CD +- Deploy to multiple clusters, in an automated fashion, using Argo CD +- But, in both cases, to empower those developers to be able to do so without needing to involve a cluster administrator (to create the necessarily Argo CD Applications/AppProject resources on their behalf) + +One potential solution to this use case is for development teams to define Argo CD `Application` resources within a Git repository (containing the manifests they wish to deploy), in an [app-of-apps pattern](../../cluster-bootstrapping/#app-of-apps-pattern), and for cluster administrators to then review/accept changes to this repository via merge requests. + +While this might sound like an effective solution, a major disadvantage is that a high degree of trust/scrutiny is needed to accept commits containing Argo CD `Application` spec changes. This is because there are many sensitive fields contained within the `Application` spec, including `project`, `cluster`, and `namespace`. An inadvertent merge might allow applications to access namespaces/clusters where they did not belong. + +Thus in the self-service use case, administrators desire to only allow some fields of the `Application` spec to be controlled by developers (eg the Git source repository) but not other fields (eg the target namespace, or target cluster, should be restricted). + +Fortunately, the ApplicationSet controller presents an alternative solution to this use case: cluster administrators may safely create an `ApplicationSet` resource containing a Git generator that restricts deployment of application resources to fixed values with the `template` field, while allowing customization of 'safe' fields by developers, at will. + +```yaml +kind: ApplicationSet +# (...) +spec: + generators: + - git: + repoURL: https://github.com/argoproj/argo-cd.git + files: + - path: "apps/**/config.json" + template: + spec: + project: dev-team-one # project is restricted + source: + # developers may customize app details using JSON files from above repo URL + repoURL: {{app.source}} + targetRevision: {{app.revision}} + path: {{app.path}} + destination: + name: production-cluster # cluster is restricted + namespace: dev-team-one # namespace is restricted +``` +See the [Git generator](Generators-Git.md) for more details. diff --git a/docs/operator-manual/applicationset/index.md b/docs/operator-manual/applicationset/index.md new file mode 100644 index 0000000000000..f603ea5b793e7 --- /dev/null +++ b/docs/operator-manual/applicationset/index.md @@ -0,0 +1,105 @@ +# Introduction to ApplicationSet controller + +## Introduction + +The ApplicationSet controller is a [Kubernetes controller](https://kubernetes.io/docs/concepts/architecture/controller/) that adds support for an `ApplicationSet` [CustomResourceDefinition](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/) (CRD). This controller/CRD enables both automation and greater flexibility managing [Argo CD](../../index.md) Applications across a large number of clusters and within monorepos, plus it makes self-service usage possible on multitenant Kubernetes clusters. + +The ApplicationSet controller works alongside an existing [Argo CD installation](../../index.md). Argo CD is a declarative, GitOps continuous delivery tool, which allows developers to define and control deployment of Kubernetes application resources from within their existing Git workflow. + +Starting with Argo CD v2.3, the ApplicationSet controller is bundled with Argo CD. + +The ApplicationSet controller, supplements Argo CD by adding additional features in support of cluster-administrator-focused scenarios. The `ApplicationSet` controller provides: + +- The ability to use a single Kubernetes manifest to target multiple Kubernetes clusters with Argo CD +- The ability to use a single Kubernetes manifest to deploy multiple applications from one or multiple Git repositories with Argo CD +- Improved support for monorepos: in the context of Argo CD, a monorepo is multiple Argo CD Application resources defined within a single Git repository +- Within multitenant clusters, improves the ability of individual cluster tenants to deploy applications using Argo CD (without needing to involve privileged cluster administrators in enabling the destination clusters/namespaces) + +## The ApplicationSet resource + +This example defines a new `guestbook` resource of kind `ApplicationSet`: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: ApplicationSet +metadata: + name: guestbook +spec: + generators: + - list: + elements: + - cluster: engineering-dev + url: https://1.2.3.4 + - cluster: engineering-prod + url: https://2.4.6.8 + - cluster: finance-preprod + url: https://9.8.7.6 + template: + metadata: + name: '{{cluster}}-guestbook' + spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: guestbook/{{cluster}} + destination: + server: '{{url}}' + namespace: guestbook +``` + +In this example, we want to deploy our `guestbook` application (with the Kubernetes resources for this application coming from Git, since this is GitOps) to a list of Kubernetes clusters (with the list of target clusters defined in the List items element of the `ApplicationSet` resource). + +While there are multiple types of *generators* that are available to use with the `ApplicationSet` resource, this example uses the List generator, which simply contains a fixed, literal list of clusters to target. This list of clusters will be the clusters upon which Argo CD deploys the `guestbook` application resources, once the ApplicationSet controller has processed the `ApplicationSet` resource. + +Generators, such as the List generator, are responsible for generating *parameters*. Parameters are key-values pairs that are substituted into the `template:` section of the ApplicationSet resource during template rendering. + +There are multiple generators currently supported by the ApplicationSet controller: + +- **List generator**: Generates parameters based on a fixed list of cluster name/URL values, as seen in the example above. +- **Cluster generator**: Rather than a literal list of clusters (as with the list generator), the cluster generator automatically generates cluster parameters based on the clusters that are defined within Argo CD. +- **Git generator**: The Git generator generates parameters based on files or folders that are contained within the Git repository defined within the generator resource. + - Files containing JSON values will be parsed and converted into template parameters. + - Individual directory paths within the Git repository may be used as parameter values, as well. +- **Matrix generator**: The Matrix generators combines the generated parameters of two other generators. + +See the [generator section](Generators.md) for more information about individual generators, and the other generators not listed above. + +## Parameter substitution into templates + +Independent of which generator is used, parameters generated by a generator are substituted into `{{parameter name}}` values within the `template:` section of the `ApplicationSet` resource. In this example, the List generator defines `cluster` and `url` parameters, which are then substituted into the template's `{{cluster}}` and `{{url}}` values, respectively. + +After substitution, this `guestbook` `ApplicationSet` resource is applied to the Kubernetes cluster: + +1. The ApplicationSet controller processes the generator entries, producing a set of template parameters. +2. These parameters are substituted into the template, once for each set of parameters. +3. Each rendered template is converted into an Argo CD `Application` resource, which is then created (or updated) within the Argo CD namespace. +4. Finally, the Argo CD controller is notified of these `Application` resources and is responsible for handling them. + + +With the three different clusters defined in our example -- `engineering-dev`, `engineering-prod`, and `finance-preprod` -- this will produce three new Argo CD `Application` resources: one for each cluster. + +Here is an example of one of the `Application` resources that would be created, for the `engineering-dev` cluster at `1.2.3.4`: +```yaml +apiVersion: argoproj.io/v1alpha1 +kind: Application +metadata: + name: engineering-dev-guestbook +spec: + source: + repoURL: https://github.com/infra-team/cluster-deployments.git + targetRevision: HEAD + path: guestbook/engineering-dev + destination: + server: https://1.2.3.4 + namespace: guestbook +``` +We can see that the generated values have been substituted into the `server` and `path` fields of the template, and the template has been rendered into a fully-fleshed out Argo CD Application. + +The Applications are now also visible from within the Argo CD UI: + +![List generator example in Argo CD Web UI](../../assets/applicationset/Introduction/List-Example-In-Argo-CD-Web-UI.png) + +The ApplicationSet controller will ensure that any changes, updates, or deletions made to `ApplicationSet` resources are automatically applied to the corresponding `Application`(s). + +For instance, if a new cluster/URL list entry was added to the List generator, a new Argo CD `Application` resource would be accordingly created for this new cluster. Any edits made to the `guestbook` `ApplicationSet` resource will affect all the Argo CD Applications that were instantiated by that resource, including the new Application. + +While the List generator's literal list of clusters is fairly simplistic, much more sophisticated scenarios are supported by the other available generators in the ApplicationSet controller. diff --git a/docs/user-guide/application-set.md b/docs/user-guide/application-set.md index de0a29013f122..15d4190f99559 100644 --- a/docs/user-guide/application-set.md +++ b/docs/user-guide/application-set.md @@ -1,6 +1,6 @@ ### Automating the generation of Argo CD Applications with the ApplicationSet Controller -The [ApplicationSet controller](https://github.com/argoproj/applicationset) is a sub-project of Argo CD which adds Application automation, and seeks to improve multi-cluster support and cluster multitenant support within Argo CD. Argo CD Applications may be templated from multiple different sources, including from Git or Argo CD's own defined cluster list. +The [ApplicationSet controller](../operator-manual/applicationset/index.md) is a part of Argo CD adds Application automation, and seeks to improve multi-cluster support and cluster multitenant support within Argo CD. Argo CD Applications may be templated from multiple different sources, including from Git or Argo CD's own defined cluster list. The set of tools provided by the ApplicationSet controller may also be used to allow developers (without access to the Argo CD namespace) to independently create Applications without cluster-administrator intervention. @@ -28,9 +28,9 @@ spec: spec: project: default source: - repoURL: https://github.com/argoproj/applicationset.git + repoURL: https://github.com/argoproj/argo-cd.git targetRevision: HEAD - path: examples/list-generator/guestbook/{{cluster}} + path: applicationset/examples/list-generator/guestbook/{{cluster}} destination: server: '{{url}}' namespace: guestbook @@ -42,6 +42,6 @@ Likewise, changes made to the ApplicationSet `template` fields will automaticall Within ApplicationSet there exist other more powerful generators in addition to the List generator, including the Cluster generator (which automatically uses Argo CD-defined clusters to template Applications), and the Git generator (which uses the files/directories of a Git repository to template applications). -To learn more about the ApplicationSet controller, check out [ApplicationSet documentation](https://argocd-applicationset.readthedocs.io/en/stable/) and [Getting Started](https://argocd-applicationset.readthedocs.io/en/stable/Getting-Started/) to install the ApplicationSet controller alongside Argo CD. +To learn more about the ApplicationSet controller, check out [ApplicationSet documentation](../operator-manual/applicationset/index.md) to install the ApplicationSet controller alongside Argo CD. **Note:** Starting `v2.3` of Argo CD, we don't need to install ApplicationSet Controller separately. It would be instead as part of Argo CD installation. \ No newline at end of file diff --git a/docs/user-guide/parameters.md b/docs/user-guide/parameters.md index 76ed7e7d83d1d..83d5b6b49f43d 100644 --- a/docs/user-guide/parameters.md +++ b/docs/user-guide/parameters.md @@ -78,7 +78,7 @@ The `.argocd-source` is trying to solve two following main use cases: - Provide the unified way to "override" application parameters in Git and enable the "write back" feature for projects like [argocd-image-updater](https://github.com/argoproj-labs/argocd-image-updater). - Support "discovering" applications in the Git repository by projects like [applicationset](https://github.com/argoproj/applicationset) -(see [git files generator](https://github.com/argoproj/applicationset/blob/master/examples/git-generator-files-discovery/git-generator-files.yaml)) +(see [git files generator](https://github.com/argoproj/argo-cd/blob/master/applicationset/examples/git-generator-files-discovery/git-generator-files.yaml)) > The following is available from v1.9 or later diff --git a/mkdocs.yml b/mkdocs.yml index de7510d472a99..9068661e81f29 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -71,6 +71,24 @@ nav: - operator-manual/notifications/services/telegram.md - operator-manual/notifications/services/webhook.md - operator-manual/troubleshooting.md + - ApplicationSet: + - Introduction: operator-manual/applicationset/index.md + - Installations: operator-manual/applicationset/Getting-Started.md + - Use Cases: operator-manual/applicationset/Use-Cases.md + - How ApplicationSet controller interacts with Argo CD: operator-manual/applicationset/Argo-CD-Integration.md + - Generators: + - operator-manual/applicationset/Generators.md + - operator-manual/applicationset/Generators-List.md + - operator-manual/applicationset/Generators-Cluster.md + - operator-manual/applicationset/Generators-Git.md + - operator-manual/applicationset/Generators-Matrix.md + - operator-manual/applicationset/Generators-Merge.md + - operator-manual/applicationset/Generators-SCM-Provider.md + - operator-manual/applicationset/Generators-Cluster-Decision-Resource.md + - operator-manual/applicationset/Generators-Pull-Request.md + - Template fields: operator-manual/applicationset/Template.md + - Controlling Resource Modification: operator-manual/applicationset/Controlling-Resource-Modification.md + - Application Pruning & Resource Deletion: operator-manual/applicationset/Application-Deletion.md - Server Configuration Parameters: - operator-manual/server-commands/argocd-server.md - operator-manual/server-commands/argocd-application-controller.md