e2e test: new framework + gen2 EKS cluster (MachineDeployment, MachinePool, AWSMachinePool) deployment & teardown

[bcle]

We need an integration test to regression test the main features of arlon:

• cluster creation from base cluster, with optional profile
• applying a profile to a base cluster
• modifying a base cluster (for e.g. k8s version upgrade) and verifying that linked clusters created from it are updated
• modifying a profile (e.g. its app/bundle composition) and verifying that clusters linked to it are updated

Subtasks:

• Explore integration test frameworks such as kubetest-2 and/or kubetest. #256
• Setup ArgoCD and Arlon on the testbed using the selected framework. #257
• Github action integration for e2e tests #264
• Look for alternatives to Git server container(current gitserver-http fails with a 504 error) #273
• Add docs for e2e tests #279
• e2e test: eks clusters with AWSMachinePool and AWSManagedMachinePool #317

(More subtasks will be added as we progress)

Aha! Link: https://pf9.aha.io/features/ARLON-136

[cruizen]

Hi @bcle , Rohit and I would like to contribute here but have no experience with the related frameworks for our tech-stack. Can you get us started with the framework selection and some initial examples?
I can help you with a spike next week for any related investigations.

The rest of us can will then chime in a couple of weeks later to add integration tests for respective areas that we have worked on.

[bcle]

Sure, will do.

[cruizen]

It will be good to add the following to the existing framework

• programmatic assertions
• integration in CI/CD
• Use of golang/ python framework that can be integrated in CI system

[cruizen]

The following frameworks look good candidates for integration test frameworks

• kubetest2 : golang based
• kubetest : python based (PyTest addon)

Reference: https://dev.to/danquack/kubernetes-integration-testing-283d

[bcle]

Renamed issue. We currently have the bash script based smoke test using testing/ensure_testbed.sh and testing/test_basecluster_deploy_with_capd.sh. The smoke test uses CAPD and might be lightweight enough to run in a Github Action VM.

We probably need a new framework/setup for true E2E test, and since EKS has become the most important type of cluster, I combined the framework with the EKS e2e test(s) for this issue.

[Rohitrajak1807]

@cruizen , @bcle, @jayanth-tjvrr and @ShaunakJoshi1407 Should we close this issue once #317 is closed?