Acceptance tests

[mudler]

Currently our smoke tests are testing the following aspects (the vbox ones are more complete, due to accelleration support in the runners):

• CD datasource and first-user configuration
• First services on boot starts correctly (ros-operator, rancherd, k3s, etc)
• Starting up a k3s cluster on rancherd and installing the ros-operator on top (rancherd included)
• Creation of MachineRegistration resources

In order to have an acceptance test suite, we should at least cover the following aspects:

• Node upgrades managed by ros-operator
• Manual node upgrades from the CLI
• Installation and bootstrap of nodes from ISO and Support container images as installation bootstrap #6
• Multi-machine tests

Action items

Enhance our test suite on github actions with new test scenarios:

• Bootstrapping nodes with MachineRegistration. Currently our test suite generates MachineRegistration resources but are not consumed. Consider tests for both ISO and container images (depends on Support container images as installation bootstrap #6) (split in Acceptance tests: bootstrapping nodes with MachineRegistration #41)
• Node upgrades following https://rancher.github.io/os2/upgrade/#rancheros (split in Acceptance tests: node upgrades #42)
• Manual upgrades, following cOS default usage and ros-installer (split in Acceptance tests: manual upgrades #43) - In progress
• Upgrades in a multi-node environment. After machines are bootstrapped, the test suite should try to upgrade the nodes with the ManagedOs resource and also manually from the cli (split in Acceptance tests: upgrades in a multi-node environment #44)
• Use ISO and container image to boot some VMs in the E2E tests (ci/e2e: modify tests to boot a VM with ISO #147)

[kkaempf]

considered done