Helm deployment flows with Flux

[duncanwraight]

Hi folks

Trying to get my head round how I can use Flux for two quite dissimilar workflows.

Context

• Two repositories
  • flux containing flux manifests for clusters and applications
  • applications containing helm charts (values and templates)
• Four environments
  • Three "production-like" envs with 1 cluster per env
  • One "test" env with ... 10 or more clusters

Production-like env workflow

This one is pretty straightforward.

• Colleagues make changes to their Helm charts in applications repo. Raise a PR to merge to main. CI pipeline runs, Helm chart is pushed to ECR OCI repository
• Colleagues, or myself, raise a PR in flux repo to change (for example) dev cluster's HelmRelease version patch
• Once merged to main, Flux agent in dev cluster reconciles the change and installs the newer version of the Helm chart from ECR

If we wanted to move to a full CI/CD process here, it wouldn't be arduous to automate the version patch as part of the main branch merge pipeline.

Test env workflow

This is the bit I can't quite get my head around. I could follow the same workflow as above... but that would be horrendously tedious for colleagues trying to onboard applications to my team's clusters.

Currently, without Flux, this is what they do:

• Branch off main in the applications repo
• Add their chart. Configure it, etc.
• Use a CI/CD pipeline to helm upgrade --install on their personal cluster in the test env

This is a pretty quick turnaround for them. Their application changes are deployed in less than 10 minutes. What would be the best way to replicate something similar with Flux?

I considered polling our ECR repositories to deploy the latest chart... but that's a bit tricky too, thanks to ECR's OCI implementation whereby repositories have to match chart names (e.g. Chart.Name: my-app has to go in .1234567.amazonecr.com/my-app). We will regularly have situations where Joe is working on my-app v1.0.2 on his cluster, and Mary is working on my-app v1.0.1 from a different branch on a different cluster. I guess we could move to a non-OCI implementation such as Artifactory hosting, but I'd rather not do that if possible.

Could HelmRelease resources from a GitRepository source be an option here perhaps? We don't have to store the charts in ECR at this point... but we do need to integrate a process where we pull the chart's container images, push them to ECR, and update the HelmRelease with override values to deploy the chart's images from our ECR repositories.

We then have to think about promoting this work into our production-like environments. Happy for that to be a relatively manual process.

Would love to hear thoughts from anyone who has done something similar. I'm possibly over-thinking the whole thing, and missing a simple solution, but it feels like a lot of moving parts that I'm struggling to bring together.