[release-4.5] Bug 1851839: Make e2e tests more stable #130

openshift-cherrypick-robot · 2020-06-29T08:05:08Z

This is an automated cherry-pick of #124

/assign martinkunc

martinkunc · 2020-06-29T08:16:49Z

/retitle [release-4.5] Bug 1851839: Make e2e tests more stable

openshift-ci-robot · 2020-06-29T08:17:09Z

@openshift-cherrypick-robot: This pull request references Bugzilla bug 1851839, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "4.5.z" instead
expected Bugzilla bug 1851839 to depend on a bug targeting a release in 4.6.0, 4.6.z and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-06-29T08:18:28Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected Bugzilla bug 1851839 to depend on a bug targeting a release in 4.6.0, 4.6.z and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-06-29T08:21:18Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "4.5.z" instead
expected Bugzilla bug 1851839 to depend on a bug targeting a release in 4.6.0, 4.6.z and in one of the following states: MODIFIED, ON_QA, VERIFIED, but no dependents were found

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-06-29T08:32:32Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "4.5.z" instead
expected dependent Bugzilla bug 1851840 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is NEW instead
expected dependent Bugzilla bug 1851840 to target a release in 4.6.0, 4.6.z, but it targets "4.5.z" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

martinkunc · 2020-06-29T08:35:37Z

/bugzilla refresh

openshift-ci-robot · 2020-06-29T08:35:39Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "4.5.z" instead
expected dependent Bugzilla bug 1851840 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is NEW instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

martinkunc · 2020-06-29T08:38:27Z

/bugzilla refresh

openshift-ci-robot · 2020-06-29T08:38:29Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected dependent Bugzilla bug 1851840 to be in one of the following states: MODIFIED, ON_QA, VERIFIED, but it is NEW instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

martinkunc · 2020-06-29T11:57:04Z

/bugzilla refresh
/test e2e-aws

openshift-ci-robot · 2020-06-29T11:57:07Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is invalid:

expected the bug to target the "4.5.0" release, but it targets "---" instead

Comment /bugzilla refresh to re-evaluate validity if changes to the Bugzilla bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

martinkunc · 2020-06-29T12:06:02Z

/bugzilla refresh

openshift-ci-robot · 2020-06-29T12:06:12Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.5.0) matches configured target release for branch (4.5.0)
bug is in the state NEW, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1851840 is in the state VERIFIED, which is one of the valid states (MODIFIED, ON_QA, VERIFIED)
dependent Bugzilla bug 1851840 targets the "4.6.0" release, which is one of the valid target releases: 4.6.0, 4.6.z
bug has dependents

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

martinkunc · 2020-06-30T05:13:01Z

/test e2e-aws

iNecas · 2020-07-01T12:37:33Z

What was the change that actually makes the tests more stable?

Also I would love for @lina-nikiforova to give her blessing, as main author of this area of the repo.

lina-is-here · 2020-07-01T13:02:46Z

Makefile

@@ -7,6 +7,7 @@ test-unit:
 .PHONY: test-unit

 test-e2e:
+	go test ./test/integration -run ^\(TestPullSecretExists\)$$ -timeout 2h


do we need 2h here?

I have created another PR with shorter period here: #135

lina-is-here · 2020-07-01T13:04:26Z

test/integration/main_test.go

@@ -102,7 +133,7 @@ func restartInsightsOperator(t *testing.T) {

 	for _, pod := range pods.Items {
 		clientset.CoreV1().Pods("openshift-insights").Delete(pod.Name, &metav1.DeleteOptions{})
-		err := wait.PollImmediate(1*time.Second, 20*time.Minute, func() (bool, error) {
+		err := wait.PollImmediate(1*time.Second, 1*time.Minute, func() (bool, error) {


I don't think pods are usually deleted within 1 minute, I'd give it more time

Actually 1 minute is with quite a reserve. Normally it takes few seconds and Kubernetes is making sure that deletion wont be longer then Grace period, which is 30s in IO pod.
You can see moure about it here: https://cloud.google.com/blog/products/gcp/kubernetes-best-practices-terminating-with-grace

lina-is-here · 2020-07-01T13:05:00Z

test/integration/main_test.go

@@ -115,7 +146,7 @@ func restartInsightsOperator(t *testing.T) {
 	}

 	// check new pods are created and running
-	errPod := wait.PollImmediate(1*time.Second, 10*time.Minute, func() (bool, error) {
+	errPod := wait.PollImmediate(1*time.Second, 1*time.Minute, func() (bool, error) {


same, I don't think 1 minute is enough

martinkunc · 2020-07-03T07:51:46Z

@iNecas Actually this is just a backport of #124, so I would need to create another PR to master with changes suggested here, if there will be some.
The main problem was that tests were modifying the cluster, for example removing secret, while other test was relying on existence of the same secret.
Code I added is reverting changes done in tests, so that other tests can run safely.

martinkunc · 2020-07-03T08:13:47Z

/test e2e-aws

martinkunc · 2020-07-03T10:13:46Z

/test e2e-aws

martinkunc · 2020-07-08T04:42:21Z

/test e2e-aws

martinkunc · 2020-07-09T07:00:22Z

@lina-nikiforova Can you please approve, so that Ivan could add the label ?

lina-is-here · 2020-07-09T07:14:15Z

lgtm

iNecas · 2020-07-17T09:11:09Z

/lgtm

openshift-ci-robot · 2020-07-17T09:11:30Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: iNecas, openshift-cherrypick-robot

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [iNecas]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

martinkunc · 2020-07-27T12:37:42Z

/bugzilla refresh

openshift-ci-robot · 2020-07-27T12:37:46Z

@martinkunc: This pull request references Bugzilla bug 1851839, which is valid.

6 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target release (4.5.z) matches configured target release for branch (4.5.z)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)
dependent bug Bugzilla bug 1851840 is in the state VERIFIED, which is one of the valid states (VERIFIED, RELEASE_PENDING, CLOSED (ERRATA))
dependent Bugzilla bug 1851840 targets the "4.6.0" release, which is one of the valid target releases: 4.6.0, 4.6.z
bug has dependents

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

openshift-ci-robot · 2020-07-30T17:37:37Z

@openshift-cherrypick-robot: All pull requests linked via external trackers have merged: openshift/insights-operator#130. Bugzilla bug 1851839 has been moved to the MODIFIED state.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Make e2e tests more stable

597d958

openshift-ci-robot assigned martinkunc Jun 29, 2020

openshift-cherrypick-robot mentioned this pull request Jun 29, 2020

Make e2e tests more stable #124

Merged

openshift-ci-robot requested review from iNecas and smarterclayton June 29, 2020 08:05

openshift-ci-robot changed the title ~~[release-4.5] Make e2e tests more stable~~ [release-4.5] Bug 1851839: Make e2e tests more stable Jun 29, 2020

openshift-ci-robot added bugzilla/severity-unspecified Referenced Bugzilla bug's severity is unspecified for the PR. bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. labels Jun 29, 2020

openshift-ci-robot removed the bugzilla/invalid-bug Indicates that a referenced Bugzilla bug is invalid for the branch this PR is targeting. label Jun 29, 2020

lina-is-here reviewed Jul 1, 2020

View reviewed changes

openshift-ci-robot assigned iNecas Jul 17, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Jul 17, 2020

openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 17, 2020

sdodson added the cherry-pick-approved Indicates a cherry-pick PR into a release branch has been approved by the release branch manager. label Jul 30, 2020

openshift-merge-robot merged commit 8aafa0d into openshift:release-4.5 Jul 30, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[release-4.5] Bug 1851839: Make e2e tests more stable #130

[release-4.5] Bug 1851839: Make e2e tests more stable #130

openshift-cherrypick-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 30, 2020

iNecas commented Jul 1, 2020

lina-is-here Jul 1, 2020

martinkunc Jul 9, 2020

lina-is-here Jul 1, 2020

martinkunc Jul 3, 2020

lina-is-here Jul 1, 2020

martinkunc commented Jul 3, 2020

martinkunc commented Jul 3, 2020

martinkunc commented Jul 3, 2020

martinkunc commented Jul 8, 2020

martinkunc commented Jul 9, 2020

lina-is-here commented Jul 9, 2020

iNecas commented Jul 17, 2020

openshift-ci-robot commented Jul 17, 2020

martinkunc commented Jul 27, 2020

openshift-ci-robot commented Jul 27, 2020

openshift-ci-robot commented Jul 30, 2020

[release-4.5] Bug 1851839: Make e2e tests more stable #130

[release-4.5] Bug 1851839: Make e2e tests more stable #130

Conversation

openshift-cherrypick-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 29, 2020

openshift-ci-robot commented Jun 29, 2020

martinkunc commented Jun 30, 2020

iNecas commented Jul 1, 2020

lina-is-here Jul 1, 2020

Choose a reason for hiding this comment

martinkunc Jul 9, 2020

Choose a reason for hiding this comment

lina-is-here Jul 1, 2020

Choose a reason for hiding this comment

martinkunc Jul 3, 2020

Choose a reason for hiding this comment

lina-is-here Jul 1, 2020

Choose a reason for hiding this comment

martinkunc commented Jul 3, 2020

martinkunc commented Jul 3, 2020

martinkunc commented Jul 3, 2020

martinkunc commented Jul 8, 2020

martinkunc commented Jul 9, 2020

lina-is-here commented Jul 9, 2020

iNecas commented Jul 17, 2020

openshift-ci-robot commented Jul 17, 2020

martinkunc commented Jul 27, 2020

openshift-ci-robot commented Jul 27, 2020

openshift-ci-robot commented Jul 30, 2020