feat(ApplicationSet): Templating - Add possibility of using advanced templating across template field

[mrmm]

Summary

Currently, with the ApplicationSet templating system, we cannot use the advanced function (although we have #10026) on the whole body of the template field as the rendering is scoped per field.

Motivation

• Usecase 1: When using git generator with the files.path set we can delegate some specific configs to the maintainer of the path with *.yaml or *.json files

{
  "appConfig": {
    "selfHeal": true
  }
}

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: my-app-set
spec:
  generators:
    - git:
        files:
          - path: path/**/config.json
        repoURL: https://github.com/argoproj/argo-cd.git
        revision: HEAD
  goTemplate: true
  template:
    metadata:
      name: "{{ .path.basename }}"
    spec:
      destination:
        name: "in-cluster"
      project: "default"
      source:
        helm:
          valueFiles:
            - values.yaml
        path: "{{ .path.path }}"
        repoURL: https://github.com/argoproj/argo-cd.git
        targetRevision: HEAD
      syncPolicy:
        automated:
          selfHeal: {{ .appConfig.selfHeal }} # This is not possible as the field should be bool

• Usecase 2: When you want to have dynamically generated annotations/labels like the below (from feat: ApplicationSet Go template #10026 (comment)), the current way is not enough the schema of the ApplicationSet is not valid this way

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
spec:
  goTemplate: true
  template:
    metadata:
      annotation:
        {{ range .annotations -}}
        {{ .key }}: "{{ .value }}"
        {{ end -}}

Proposal

Introduce a new field in the ApplicationSet CRD of type string let's call it stringTemplate that will contain the go template.

A quick overview of the implementation:

1. Render using Replace method implemented in Render struct (thanks to @speedfl for the suggestion here feat(appset): Add stringTemplate field to spec #11183 (comment))
2. Marshal the previously rendered template to an Application inside RenderTemplateParams
3. If the tmpl is not nil it will be merged with the previously Marshaled stringTemplate

[mrmm]

@crenshaw-dev can you take a look at this please, I will complete the PR with some examples to facilitate the review.

[nerzhul]

It's also a workflow we want to permit

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: cluster-main
spec:
  generators:
  - matrix:
      generators:
      - git:
          files:
          - path: '*/config.json'
          repoURL: https://git.repo/git
          revision: 'main'
      - clusters: {}
  template:
    metadata:
      name: '{{path[0]}}-{{name}}'
    spec:
      ignoreDifferences:
      - group: '{{ignoreDifferences.0.group}}'
        kind: '{{ignoreDifferences.0.kind}}'
        name: '{{ignoreDifferences.0.name}}'
        namespace: '{{ignoreDifferences.0.namespace}}'
        jsonPointers:
          - '{{ignoreDifferences.0.jsonPointers.0}}'
          - '{{ignoreDifferences.0.jsonPointers.1}}'
          - '{{ignoreDifferences.0.jsonPointers.2}}'
          - '{{ignoreDifferences.0.jsonPointers.3}}'
          - '{{ignoreDifferences.0.jsonPointers.4}}'

We want to prevent this ^

[phclark]

Any update on this? The two uses cases mentioned above would be incredibly valuable to us! Do you need any assistance?

[GuySaar8]

FYI - until this update is coming out
this is not a dynamic set up - but at least this way we can decide when to use ignoreDifferences

      - group: autoscaling
        kind: HorizontalPodAutoscaler
        jqPathExpressions:
        - select(.metadata.annotations.someflag=="true") | .spec.minReplicas

hope that it might be useful for someone 🙏

[dharma-rise]

Any update this issue?