bootstrapper: wipe disk and reboot on non-recoverable error

[daniel-weisse]

Context

Constellation's bootstrapper may fail at multiple stages.
Some of these failures are transient, like joining an existing cluster using kubeadm and may be retried.
Others are not recoverable, meaning one can no longer re-run constellation apply on the same node, or the node can no longer retry joining an existing cluster.
To "fix" a VM that is in this failed state, the VM needs to be rebooted, and the state-disk wiped.

Proposed change(s)

• Change how joinClient and initServer are run
  • Both routines now have return values and are run in parallel
  • If any error occurs in run(), the VM is rebooted
• Adapt the flow of joinClient to more easily differentiate between transient and non recoverable errors
• In both joinClient and initServer, mark the state disk as not initialized if a non recoverable error occurs (any error after the VM was marked as initialized)
• Remove the ability to call JoinClient.Start() and JoinClient.Stop() idempotently
  • This was and is not required for the bootstrapper
  • This was unnecessarily complicating the set up and testing of the client

Related issue

• Fixes https://github.com/edgelesssys/issues/issues/421

Additional info

• AB#3930
• Why not use systemd native ways to restart the VM when constelllation-bootstrapper.service fails?
  • On debug images, the service is expected to fail until a bootstrapper is uploaded using cdbg. This makes using debug images impossible
• How to test?
  • This PR currently contains a commit marked with [drop] that returns an error in the joinClient which causes a fatal error and causes the VM to reboot. The state disk should then be marked as non initialized and be wiped again on start.

Checklist

• Run the E2E tests that are relevant to this PR's changes
  • e2e Test (Azure SNP) to make sure regular behavior is still as expected
• Update docs
• Add labels (e.g., for changelog category)
• Is PR title adequate for changelog?
• Link to Milestone

[netlify]

✅ Deploy Preview for constellation-docs ready!

Name	Link
🔨 Latest commit	cac3b47
🔍 Latest deploy log	https://app.netlify.com/sites/constellation-docs/deploys/65f02c12070d98000894cb2b
😎 Deploy Preview	https://deploy-preview-2971--constellation-docs.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify site configuration.

[3u13r]

LGTM

[github-actions]

Coverage report

Package	Old	New	Trend
bootstrapper/cmd/bootstrapper	0.00%	0.00%	🚧
bootstrapper/internal/diskencryption	6.50%	6.50%	🚧
bootstrapper/internal/initserver	17.90%	23.00%	↗️
bootstrapper/internal/joinclient	32.90%	40.00%	↗️