Re-evaluate using overlays/bases for this repo

[HumairAK]

We now have a situation where one argocd app is deploying manifests for 2 different environments (infra/cnv), but both these environments may want to consume from the same base, which could yield conflicting application names (hence CR names), this is obviously a conflict that cannot occur. This is an inherent flaw with the design of using base/overlays for this purpose, so we may want to re-consider that format for this repo. Thoughts?

[tumido]

Yeah, I'd prefer a flat structure - like an overlay per multi-cluster deployment, not an overlay per cluster as we're used to.

[4n4nd]

let's make a list of components that we might want to move/copy to the infra cluster so we know which components we will need to start with.

[tumido]

it's not about migration in this case, it's more about naming conflicts - multiple overlays would end up being managed by a single argo cd deployment. Which means name conflicts, since you can't have for example cluster-resources application defined twice.

[HumairAK]

like an overlay per multi-cluster deployment, not an overlay per cluster as we're used to.

@tumido can you elaborate here with an example? Not sure I fully understand.

[tumido]

The current approach worked just fine when we had the same mapping in different environments:

1 application path ~ 1 app in argocd ~ 1 cluster ~ 1 overlay in the argocd-apps ~ 1 deployment environment

Like for example for quicklab environment, the apps we're pointing to a different source path than for moc environment, while referencing the same app base.

Our current problem is that our mapping changed to:

>1 application paths ~ >1 apps in argocd ~ >1 clusters ~ 1 overlay in the argocd-apps ~ 1 deployment environment

In other words we'd like to use a single resource from the base multiple times in a single overlay. And that can't work.

If we choose to add those "additional" and "multiple-time used from base" applications into the base, we're resigning on the overlay principle now, because we're having environment specific content in the base.

Our environments are not equal anymore, that means our overlays are not "just a set of light patches" on top of the base. Our overlays are becoming independent bases, and that's not the kustomize way either.

I don't have an easy recipe for this. Maybe we should consider converting this repository to a helm chart and build the application resources on the fly for each environment using different chart variables. That would be a solution...

[HumairAK]

Hrm I'd like to avoid helm if we can. But if it's the best way forward I'm not entirely against it.

What if we take this current structure which looks like:

.
└── argocd_apps
    ├── base
    │   ├── project_1
    │   └── project_n
    └── overlays
        ├── cluster_1
        │   ├── project_1
        │   └── project_n
        ├── cluster_2
        └── cluster_3

And changed it to:

.
└── argocd_apps
    ├── cluster_1
    │   ├── project_1
    │   └── project_n
    ├── cluster_2
    └── cluster_3

Still done via kustomize, but without overlays/base. We keep an app-of-apps for for each cluster, so for example if moc-cnv is cluster_1 and moc_infra is cluster_2, they both get an app-of-apps-moc-infra and app-of-apps-moc-cnv. Of course, the downsides is that there may be some repetition, but I don't think it's that bad honestly, it's just one manifest per app. The biggest plus side I think for this is that it makes it really easy for users to figure out (that are not kustomize savvy) where to add their argocd manifests. We just tell them "you want to deploy to cluster_2? just add it to the cluster_2 folder". If they want to deploy to multiple clusters, they add it to both cluster folders without knowing about bases, patching, etc.

[HumairAK]

bad bot

[HumairAK]

I think this is done? Please re-open if there's more left.