Revamp e2e test suite #623
Conversation
|
Currently this is still WIP, the webhook suite is not migrated, the old suite is not deleted and this will also need a proper documentation PR, so I send this as a draft to get some comments about this, so please still take a look and tell me. |
diconico07
force-pushed
the
revamp-e2e
branch
6 times, most recently
from
074e959
to
02a8ba9
Compare
|
Only the documentation part is lacking before getting this out of draft |
diconico07
requested review from
bfjelds,
kate-goldenring,
jiria,
Britel,
romoh,
adithyaj and
johnsonshih
as code owners
|
/add-same-version-label |
|
👋 Added [same version] label :)! |
|
👋 Added [same version] label :)! |
diconico07
force-pushed
the
revamp-e2e
branch
4 times, most recently
from
0f2a072
to
b4a6e47
Compare
| def distribution_config(pytestconfig): | ||
| distribution = pytestconfig.getoption("--distribution", None) | ||
| if distribution == "k3s": | ||
| yield Distribution("k3s", Path.home() / ".kube/config", "kubectl") |
There was a problem hiding this comment.
Should we update the tests and workflow to match the distribution defaults to better enable running the tests locally without having to copy over the kubeconfig?
| yield Distribution("k3s", Path.home() / ".kube/config", "kubectl") | |
| yield Distribution("k3s", Path.home() / "/etc/rancher/k3s/k3s.yaml", "kubectl") |
There was a problem hiding this comment.
I'm not sure of the added benefit for local runs, as I think most people will either have a long running instance they use for tests (so probably already configured) or use tools like k3d or rancher desktop that already manage the configuration.
In fact I first wanted to just get rid of that element here (and let the python kubernetes client handle it the same way the helm or kubectl commands do), but wasn't really sure it was the right thing to do.
Move to pytest based test-suite, and ensure tests can be run more easily locally. The main reasons to move to pytest are: - Allows to run a subset of the tests rather than full suite - Ensure better separation of tests vs helpers - Easily share code for setup/teardown between suites This commit also change dependency management to use poetry, this allows to setup the test environment locally in an easy and predictible way, it also ensure more reproductability for our test-suite as the dependencies are pinned. On a more functional end, switch to slim agent to allow test suites to chose their discovery handlers. Also stop using helm for configuration deployment to decorelate it from akri's installation, preventing possible noise in tests. Also makes use of watch instead of polling as much as possible to reduce waiting time (goes from about 3 minutes for e2e run to about 80 seconds). Hopefully this will make it easier to write more e2e tests. Signed-off-by: Nicolas Belouin <nicolas.belouin@suse.com>
Signed-off-by: Nicolas Belouin <nicolas.belouin@suse.com>
What this PR does / why we need it:
Move to pytest based test-suite, and ensure tests can be run more easily locally.
The main reasons to move to pytest are:
This commit also change dependency management to use poetry, this allows to setup the test environment locally in an easy and predictable way, it also ensure more reproducibility for our test-suite as the dependencies are pinned.
On a more functional end, switch to slim agent to allow test suites to chose their discovery handlers. Also stop using helm for configuration deployment to decorrelate it from akri's installation, preventing possible noise in tests.
Also makes use of watch instead of polling as much as possible to reduce waiting time (goes from about 3 minutes for e2e run to about 80 seconds).
Hopefully this will make it easier to write more e2e tests.
Special notes for your reviewer:
If applicable:
cargo fmt)cargo build)cargo clippy)cargo test)cargo doc)