Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep k8s resources on test failure #236

Merged
merged 4 commits into from
Jun 30, 2023

Conversation

pablochacin
Copy link
Collaborator

@pablochacin pablochacin commented Jun 27, 2023

Description

This PR adds a helper for e2e tests that allows retaining the namespace used for a test if the test fails.

Fixes #228

Note to reviewers

Testing this PR is tricky as the expected behavior is triggered by a failed test. Therefore, a unit test was added that actually fails, but still checks if the expected behavior happened.

Another important note is that this test requires the sub-test not to run in parallel or otherwise the validation would happen before the subtests ends. However, the tparallel linter would complain if the sub-test does not specifies t.Parallel() and it does not accept the nolint directive.

A workaround for the two issues about is to make this test build only if the keeponfail tag is set in the go test command

The command for running this test and the expected output is shown below:

go test -tags keeponfail ./pkg/testutils/e2e/kubernetes/namespace/

--- FAIL: Test_KeepOnFail (0.00s)
    --- FAIL: Test_KeepOnFail/Sub-test (0.00s)
    namespace_test.go:104: test succeeded. Namespace testns was preserved after subtest failed
FAIL
FAIL	github.com/grafana/xk6-disruptor/pkg/testutils/e2e/kubernetes/namespace	0.025s
FAIL

Checklist:

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works.
  • I have run linter locally (make lint) and all checks pass.
  • I have run tests locally (make test) and all tests pass.
  • I have run relevant e2e test locally (make e2e-xxx for agent, disruptors, kubernetes or cluster related changes)
  • Any dependent changes have been merged and published in downstream modules

@pablochacin pablochacin force-pushed the keep-k8s-resources-on-test-failure branch 2 times, most recently from 7761f58 to 20a90bb Compare June 27, 2023 20:09
@pablochacin pablochacin mentioned this pull request Jun 28, 2023
3 tasks
@pablochacin pablochacin force-pushed the keep-k8s-resources-on-test-failure branch from 20a90bb to 7763aa9 Compare June 28, 2023 12:36
@pablochacin pablochacin marked this pull request as ready for review June 28, 2023 17:21
@pablochacin pablochacin requested a review from roobre June 28, 2023 17:22
Signed-off-by: Pablo Chacin <pablochacin@gmail.com>
Signed-off-by: Pablo Chacin <pablochacin@gmail.com>
Signed-off-by: Pablo Chacin <pablochacin@gmail.com>
@pablochacin pablochacin force-pushed the keep-k8s-resources-on-test-failure branch from 7763aa9 to a148748 Compare June 28, 2023 17:24
Copy link
Collaborator

@roobre roobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great, I think this is a very useful feature for debugging e2e scenarios. Thanks for taking care of this!

if err != nil {
t.Errorf("error creating test namespace: %v", err)
t.Errorf("failed to create test namespace: %v", err)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably a discussion for another PR, but t.Fatalf would be preferred in this cases, as it protects us from forgetting the return line below.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer to unify this in another PR.

options: []TestNamespaceOption{WithPrefix("prefix-")},
expectError: false,
check: func(k8s kubernetes.Interface, ns string) error {
return nil
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check here that strings.HasPrefix(ns, "prefix-")?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, I forgot to implement the checks 😅

Signed-off-by: Pablo Chacin <pablochacin@gmail.com>
@pablochacin pablochacin requested a review from roobre June 30, 2023 09:44
Copy link
Collaborator

@roobre roobre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@pablochacin pablochacin merged commit 5e1c4be into main Jun 30, 2023
7 checks passed
@pablochacin pablochacin deleted the keep-k8s-resources-on-test-failure branch June 30, 2023 10:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add option for keeping kubernetes resources when an e2e test fails
2 participants