New odo testing system

[kadel]

We need to get into the point where we have tests running for each PR and also periodic/nightly jobs.

Jobs executed for each PR

The PR tests should verify that the code that is about to get merge works.
All PR tests are executed on IBM Cloud, and OCP cluster can be of any stable version.

• unit tests started on linux (previw on A Test for IBM Pipelines feloy/odo#7)
• unit tests started on windows
• unit tests started on mac - tracked with Run e2e and integration test on MacOS using github actions #5301
• integrations test executed from mac against OCP cluster - tracked with Run e2e and integration test on MacOS using github actions #5301
• integrations test executed from windows against OCP cluster
• integrations test executed from linux against OCP cluster (preview on A Test for IBM Pipelines feloy/odo#7)
• integrations tests executed from linux against the latest stable kubernetes cluster (can be minikube) (preview on A Test for IBM Pipelines feloy/odo#7)

The periodic/nightly jobs

Jobs should verify that what is in odo main branch works with all supported OCP versions.
All nightly jobs are executed using OpenShift CI (Prow)

• integrations tests executed from linux against all supported OCP versions (usually the latest one and 2 previous versions)
• integrations tests executed from linux against future OCP version (passing is optional)

/kind test
/area infra

[openshift-ci]

@kadel: The label(s) kind/test cannot be applied, because the repository doesn't have them.

In response to this:

We need to get into the point where we have tests running for each PR and also periodic/nightly jobs.

The PR tests should verify that the code that is about to get merge works.
Jobs that should be executed for each PR:

• unit tests started on linux
• unit tests started on windows
• unit tests started on mac
• integrations test executed from mac against OCP cluster
• integrations tests executed from linux against the latest stable kubernetes cluster (can be minikube)

The periodic/nightly jobs should verify that what is in odo main branch works with all supported OCP versions.

• integrations test executed from linux against OCP cluster (ideally the latest stable one, but it can be any supported version)
• integrations test executed from windows against OCP cluster (ideally the latest stable one, but it can be any supported version)
• integrations tests executed form linux against all supported OCP versions (usually the latest one and 2 previous versions)
• integrations tests executed from linux against future OCP version

/kind test
/area infra

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[rnapoles-rh]

Tracking the MacOS portion with #5301