Support review-based bootstrap

[aristapimenta]

Hi,

The product offered by my company needs compliance with the SOC2 certification, which requires all changes to be reviewed by someone else. This makes the current bootstrap workflow implemented by flux bootstrap unsuitable for us because it relies on committing and pushing changes to the main branch automatically.

It would be really nice if flux bootstrap could support a bootstrap workflow where the commits are pushed to a second branch/place for review (most specifically the equivalent of git push origin HEAD:refs/for/main in my case), then a prompt with something like Git change was filed. Please merge it and press enter to continue. is printed and the command blocks until the user presses enter or types Y to proceed or something (or even better, until it detects automatically that the proposed Git changes were merged to the main branch). If the changes are now present, the command resumes the bootstrap by applying the manifests as expected.

[makkes]

I see your point here but flux bootstrap also directly installs Flux onto the target cluster to solve the chicken-and-egg problem. Wouldn't that pose the same issues with your SOC2 compliance?

If the changes are now present, the command resumes the bootstrap by applying the manifests as expected.

Ah, I see. Hmm, that doesn't sound totally unreasonable. 🤔

[hiddeco]

The problem I would have with it being "pending and waiting for review", is that you would need a second person ready-to-go.

Given this, I am wondering if you would not be better off with automating things in CI using flux install. Every time there is a new Flux release, you could then automatically get a PR which updates your manifests (and is then later pulled in automatically by the already active GitRepository and Kustomization).

[aristapimenta]

I'm not even concerned about upgrade yet. I just wanna be able to easily bootstrap Flux in a given cluster. right now I'm making my own shell script to behave as described above, but it would be really nice if the Flux CLI supported this use case as well (I'm not asking for this use case to be the default, but just a supported use case).

is that you would need a second person ready-to-go

that's exactly what SOC2 expects from us for every single Git change. bootstrapping Flux in a given cluster would not be different, so we're 100% ok with it

[makkes]

We will very likely not implement this in Flux because

1. it really is a pretty uncommon edge case
2. it can be solved with little effort wrapping the CLI
3. making it reliable is hard: what if the CLI is terminated while it's waiting for the PR to get merged (think pressing CTRL+C, the machine going down, etc.)? Should it pick up the PR? Should it create a new one?

[aristapimenta]

1. why is it so uncommon? it's pretty common to have your main branch be protected by rules like requiring pull requests, requiring CI status checks to pass and things like that. supporting a main branch with protection against direct pushes does not sound unusual to me at all. actually, I was really surprised that Flux was not covering this already. this alternative bootstrap/upgrade process would indeed be a lil bit more manual, but yeah some companies relying on those kinds of certifications would be pretty happy with that.
2. can you pls share an example?
3. well, hard is solving NP-complete problems in deterministic polynomial time, or overcoming the limitations explained by the CAP theorem when designing systems. in this case it's really just a matter of thinking it through, making design choices and implementing the best compromise, but yeah it may take a substantial time from one person or two. actually, I can 100% volunteer myself to implement this if the Flux team decides to support the use case and helps drive the discussions 🤷 CTRL+C/the machine going down can happen in the middle of the fully-automated command as well since it actually takes a while for some parts to complete, and if it still works idempotently I'm sure the reason is because somebody took the time to consider the potential problems and address them

[hiddeco]

$ GIT_URL="ssh://git@github.com/<myorg>/<myrepository>.git
$ PATH="some/path"

$ # Stage manifests

$ mkdir -p $INSTALL_PATH

$ cd $INSTALL_PATH
$ flux install --export > gotk-components.yaml
$ flux create source git flux-system \
  --url=$GIT_URL \
  --branch=main \
  --secret-ref flux-system \
  --export > gotk-sync.yaml
$ kustomize create --autodetect

$ # Commit

$ # ...once merged
$ flux create secret flux-system \
  --url=$GIT_URL \
  --private-key-file=/path/to/private-key # or other auth options
$ kubectl apply -k $INSTALL_PATH

is pretty much equal to what Flux bootstrap does.

[aristapimenta]

yeah from an automation reproducibility point of view this example script really just needs to consider a few more things that can go wrong, which is what I'm doing in my script right now because it's what my team is gonna use for booting up Flux in new clusters. so, the core steps are actually not many, it's true, but implementing it reliably should probably be a job for the Flux CLI (and I'm really surprised that nobody raised this use case so far, is everybody using Flux with Git repos that do not prevent direct pushes to the main branch at all? this is really surprising 😨)