Make flux diff easy usable via CRD from within a Kubernetes cluster

[jrauschenbusch]

Flux diff requires access to the Kubernetes cluster. Therefore the usage of flux diff from within a CI/CD environment like GitHub, Gitlab etc. is not the best idea. Exactly this is the main advantage of Flux: Not to be required to expose Cluster credentials to an CI/CD environment, but to have access from the cluster to the sources which should be deployed within the cluster (inversion of control).

My proposal would be, the have an easy way of using flux diff from within a cluster, instead of executing it from outside. The goal is to get an early change report before an Pull/Merge request is applied to the branch observed by Flux (e.g. main)

Results/Reports should could be sent by Notification controller. Either to a Chat system or as report to a source control system (e.g. enrich Pull/Merge request).

A custom CRD to enable flux diff operations triggered on a change (all non default branches or branches by pattern) would be really cool to provide such a feature.

[souleb]

We are working on adding an option to be able to specify a local path to retrieve the kustomization versus having to pull it from the cluster.

Would that answer your need?

[jrauschenbusch]

From my point of view the main advantage of a flux diff in contrast to a simple git diff/kustomize build/dyff combo is, that one get's the information which changes would really be applied on the cluster. Therefore a cluster dry run is necessary. Hence flux diff needs to access the kustomization controller API on the cluster, right?

[stefanprodan]

flux diff is like kubectl diff, it can't be run without Kubernetes API access because diffing is built on top of server-side apply, so no matter if we allow a path to a local Flux Kustomization file, you still need cluster access.

[souleb]

that's right!

So if I understand correctly and you want the kustomize-controller to return just after the ssa dry-run, could we not just add a dry-run field to the kustomization crd?

[stefanprodan]

You could create a Kubernetes Job using the gchr.io/fluxcd/flux-cli container, mount the Git repo in there, then run the command, capture the output and use curl to post it to Slack or something. Doing this natively in Flux could be an option after we get to GA.

[mraerino]

i like your idea of this being solved through the notification controller a lot and am looking into ways to build what you described - natively running the diff inside the cluster based on webhooks from branches