test: Add createcluster action to provisioner cli

[mkulke]

The test itself will perform an installation of the operator/CAA on the cluster as a setup step, so installing it twice is redundant.

The installation should be idempotent, but having 2 places performing the installation is probably suboptimal and might lead to undesired effects if different provisioner config files are used. If we want to test whether a re-installation works, we should have a dedicated test for that.

I would opt for removing the CAA installation from the provision action, but a createcluster action might be a good middle ground.

[stevenhorsman]

@katexochen might be interested in this in relation to #1543

[mkulke]

@katexochen might be interested in this in relation to #1543

If we replace cluster provisioning w/ tf code, this PR wouldn't be needed. Not sure how much of an effort the tf conversion will be, maybe that could be a temporary fix?

[stevenhorsman]

If we replace cluster provisioning w/ tf code, this PR wouldn't be needed. Not sure how much of an effort the tf conversion will be, maybe that could be a temporary fix?

Yes - sorry, I didn't mean to dismiss as not needed, but it might not be strategic long term. I think it's fine as a fix though.
I've though about trying to scrap and re-architecture the actions to be more modular where they validate all their own required variables rather than the fix of actions and environment variables we have at the moment and we could then string together actions easier with different steps like:

• Create cluster (which could be replaced with terraform)
• Install CoCo operator including peer-pods
• Run e2e tests
• Uninstall CoCo operator including peer-pods
• Delete cluster (again could terraform)

might I've not got around to prototyping, or articulating it clearly yet, but I think this aligns with this PR

[stevenhorsman]

LGTM. Thanks!

[mkulke]

• Create cluster (which could be replaced with terraform)
• Install CoCo operator including peer-pods
• Run e2e tests
• Uninstall CoCo operator including peer-pods
• Delete cluster (again could terraform)

Maybe there's value in keeping steps 2-3 in the test suite, so the tests would always be coupled to a known configuration of a peer-pod setup. step 4 we probably only need to provide in the provisioner cli in tests. removing clusters should be sufficient.

we might also need a step 6 which is garbage collecting resource leftovers, bugs in CAA will leak resources that aren't cleaned up properly. we cannot avoid having those bugs, so we need to account for that in the tests.

[stevenhorsman]

Maybe there's value in keeping steps 2-3 in the test suite, so the tests would always be coupled to a known configuration of a peer-pod setup. step 4 we probably only need to provide in the provisioner cli in tests. removing clusters should be sufficient.

we might also need a step 6 which is garbage collecting resource leftovers, bugs in CAA will leak resources that aren't cleaned up properly. we cannot avoid having those bugs, so we need to account for that in the tests.

Yeah, those are great points. My thoughts around this were to have the separate actions/units available and independently runnable, but that the test suite would be set to run 2 and 3, but things like the 1 and 5 with are controlled by environment variables would be decoupled. Anyway thoughts for the future!

[surajssd]

I have not tested this myself, but looks good at first impression. Thanks @mkulke