Matrix generator where a generator can reference items of another one

[gaeljw]

I'm wondering if this would be possible that a generator use another's generator items.

For instance, imagine the following manifests structure on a GitOps repository:

- dev
    - instanceA
    - instanceB
- prod
    - instanceA
    - instanceB
    - instanceC
    - instanceD

Each instance being defined as a single Argo application. Thus 6 instances, 6 Argo applications.

Right now, I use a List generator with 6 items corresponding to the 6 variants.

I would like to be able to define a Matrix generator that would combine:

• a list generator for the 1st level (dev/prod)
• a git directory generator for the 2nd level but here the path should be able to reuse the list generator attribute

Something like (draft):

generators:
  - matrix: 
    - list:
        elements:
          - platform: dev
            cluster: dev
          - platform: prod
            cluster: prod
    - git:
       directories:
         - path: {{platform}}/* # <-- here reuse {{platform}} of first generator

That's not possible for now, right? Or did I miss something?

[KojoRising]

This would be a great feature to have. Also have a similar use-case, but for merge generator. In this scenario, we could target specific clusters w/ specific configurations per cluster, without the need to modify the applicationSet (we could just continue modifying the git repository).

   - merge:
       mergeKeys:
         - metadata.labels.kubernetes.io/environment
       generators:
         - git:
             repoURL: "-"
             revision: HEAD
             files:
             - path: "environments/**/config.yaml" # Say, dev, tst, prod
         - clusters:
             selector:
               matchLabels:
                 argocd.argoproj.io/secret-type: cluster
                 kubernetes.io/environment: '{{path.basename}}' # Then here, we could select dev, tst, prod clusters

[crenshaw-dev]

I agree, this would be really useful. Is anyone interested in opening a PR?

[KojoRising]

Yeah, I'd be happy to give it a go.

[KojoRising]

FYI, have gotten this working. Currently only the 2nd generator can reference the 1st, so

• A) The order will matter
• B) Both child generators' can't depend on params from one another (almost like a nested dependency issue).

Anyways, example of a cluster-generator interpolated from a git-files' values below:

[jannis-a]

@KojoRising What's the status of your implementation? I will most likely have a use case for this feature in the near future, pretty excited to see this hitting an official release :)

[KojoRising]

@jannis-a - I'm working on it right now. argoproj/argo-cd#9080

I already have a working version, but a reviewer on my PR (above) pointed out an edge case I didn't anticipate. Aiming to have it out by the end of the week.

Thanks for your interest tho! Feel like this will be useful for many folks.

[KojoRising]

Also @jannis-a, I ran into a small problem on my PR (somewhat of an edge case). Feel free to take a look and provide some opinions on how I should tackle the problem. Thanks!

argoproj/argo-cd#9080

[jannis-a]

@KojoRising Great, I will take a look at that PR :)

[gaeljw]

Closing as awesome work has been done in argoproj/argo-cd#9080