Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[systemtest] Small fixes for failing tests #3908

Merged
merged 2 commits into from
Nov 11, 2020
Merged

[systemtest] Small fixes for failing tests #3908

merged 2 commits into from
Nov 11, 2020

Conversation

im-konge
Copy link
Member

@im-konge im-konge commented Nov 3, 2020 •
edited

Signed-off-by: Lukas Kral lukywill16@gmail.com

Type of change

  • Test fixes

Description

  • CustomResourceStatusST - problem here was that when the timeout for operation was set to 5 minutes, the connection to Kafka/MM2 wasn't established and the time for MM2 to become ready ran out
  • RollingUpdateST - here was similar problem, on ocp4.x when we wanted to change bootstrap for listener, the operation timeout was too short, so the pod didn't roll in time

Checklist

  • Make sure all tests pass

@im-konge im-konge added this to the 0.21.0 milestone Nov 3, 2020
@im-konge im-konge self-assigned this Nov 3, 2020
@im-konge im-konge changed the title [systemtest] Tests fixes [systemtest] Small fixes for failing tests Nov 3, 2020
@im-konge im-konge marked this pull request as ready for review November 3, 2020 15:53
@im-konge
Copy link
Member Author

im-konge commented Nov 3, 2020

@strimzi-ci run tests test_only profile=regression testcase=RollingUpdateST,CustomResourceStatusST

Frawless
Frawless approved these changes Nov 3, 2020
View reviewed changes
Copy link
Member

@Frawless Frawless left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, do we have some issue for these timing problems?

@im-konge
Copy link
Member Author

im-konge commented Nov 3, 2020

@Frawless I think we don't have the GH issue, gonna create one.

@im-konge
Copy link
Member Author

im-konge commented Nov 3, 2020 •
edited

@Frawless @scholzj #3913

@strimzi-ci
Copy link

❌ Test Summary ❌

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: RollingUpdateST,CustomResourceStatusST
TOTAL: 2
PASS: 1
FAIL: 1
SKIP: 0
BUILD_NUMBER: 1669
BUILD_ENV: oc cluster up

Re-run command:
@strimzi-ci run tests true profile=regression testcase=io.strimzi.systemtest.operators.CustomResourceStatusST#testKafkaMirrorMaker2Status,io.strimzi.systemtest.rollingupdate.RollingUpdateST#testManualKafkaConfigMapChangeDontTriggerRollingUpdate

@strimzi-ci
Copy link

❌ Test Summary ❌

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: RollingUpdateST,CustomResourceStatusST
TOTAL: 16
PASS: 15
FAIL: 1
SKIP: 0
BUILD_NUMBER: 1668
BUILD_ENV: oc cluster up

❗ Test Failures ❗

  • testKafkaConnectAndConnectorStatus in io.strimzi.systemtest.operators.CustomResourceStatusST

Re-run command:
@strimzi-ci run tests true profile=regression testcase=io.strimzi.systemtest.operators.CustomResourceStatusST#testKafkaConnectAndConnectorStatus

@im-konge
Copy link
Member Author

im-konge commented Nov 3, 2020

@strimzi-ci run tests test_only profile=regression testcase=RollingUpdateST,CustomResourceStatusST

@strimzi-ci
Copy link

❌ Test Summary ❌

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: RollingUpdateST,CustomResourceStatusST
TOTAL: 11
PASS: 10
FAIL: 1
SKIP: 0
BUILD_NUMBER: 1670
BUILD_ENV: oc cluster up

❗ Test Failures ❗

  • testKafkaConnectAndConnectorStatus in io.strimzi.systemtest.operators.CustomResourceStatusST

Re-run command:
@strimzi-ci run tests true profile=regression testcase=io.strimzi.systemtest.operators.CustomResourceStatusST#testKafkaMirrorMaker2Status,io.strimzi.systemtest.operators.CustomResourceStatusST#testKafkaConnectAndConnectorStatus

@im-konge
fixes
ba745b7 
Signed-off-by: Lukas Kral <lukywill16@gmail.com>
@im-konge
fixup! fixes
ab01082 
Signed-off-by: Lukas Kral <lukywill16@gmail.com>
@im-konge
Copy link
Member Author

im-konge commented Nov 4, 2020

@strimzi-ci run tests test_only profile=regression testcase=RollingUpdateST,CustomResourceStatusST

@strimzi-ci
Copy link

✔️ Test Summary ✔️

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: RollingUpdateST,CustomResourceStatusST
TOTAL: 12
PASS: 12
FAIL: 0
SKIP: 0
BUILD_NUMBER: 1677
BUILD_ENV: oc cluster up

@im-konge
Copy link
Member Author

im-konge commented Nov 4, 2020

@strimzi-ci run tests test_only profile=regression testcase=CustomResourceStatusST

@strimzi-ci
Copy link

✔️ Test Summary ✔️

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: CustomResourceStatusST
TOTAL: 16
PASS: 16
FAIL: 0
SKIP: 0
BUILD_NUMBER: 1678
BUILD_ENV: oc cluster up

@im-konge
Copy link
Member Author

im-konge commented Nov 5, 2020

@strimzi-ci run tests test_only profile=regression testcase=CustomResourceStatusST

@strimzi-ci
Copy link

❌ Test Summary ❌

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: CustomResourceStatusST
TOTAL: 1
PASS: 0
FAIL: 1
SKIP: 0
BUILD_NUMBER: 1679
BUILD_ENV: oc cluster up

❗ Test Failures ❗

  • testKafkaConnectAndConnectorStatus in io.strimzi.systemtest.operators.CustomResourceStatusST

Re-run command:
@strimzi-ci run tests true profile=regression testcase=io.strimzi.systemtest.operators.CustomResourceStatusST#testKafkaConnectAndConnectorStatus

@scholzj
Copy link
Member

scholzj commented Nov 9, 2020

@im-konge Are the Jenkins failures something what needs to be addressed? Or can this be merged?

@im-konge
Copy link
Member Author

im-konge commented Nov 9, 2020

@scholzj I'm looking at the failures, as it is still sometimes (very randomly) flaky. Give me one more day, I'll let you know. Thanks

@im-konge
Copy link
Member Author

@strimzi-ci run tests test_only profile=regression testcase=CustomResourceStatusST

@strimzi-ci
Copy link

✔️ Test Summary ✔️

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: CustomResourceStatusST
TOTAL: 16
PASS: 16
FAIL: 0
SKIP: 0
BUILD_NUMBER: 1688
BUILD_ENV: oc cluster up

@im-konge
Copy link
Member Author

@strimzi-ci run tests test_only profile=regression testcase=CustomResourceStatusST

@strimzi-ci
Copy link

✔️ Test Summary ✔️

TEST_PROFILE: regression
EXCLUDED_GROUPS: networkpolicies,flaky
TEST_CASE: CustomResourceStatusST
TOTAL: 16
PASS: 16
FAIL: 0
SKIP: 0
BUILD_NUMBER: 1689
BUILD_ENV: oc cluster up

@im-konge
Copy link
Member Author

Hmm it seems that this is okay 😄

So let's merge it @scholzj @Frawless ! 😄

@im-konge im-konge added the ready for merge Label for PRs which are ready for merge label Nov 11, 2020
@Frawless Frawless merged commit e863d7a into strimzi:master Nov 11, 2020
@im-konge im-konge deleted the fixes branch February 2, 2021 09:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@scholzj scholzj scholzj approved these changes

@ppatierno ppatierno ppatierno approved these changes

@Frawless Frawless Frawless approved these changes

Assignees

@im-konge im-konge

Labels
ready for merge Label for PRs which are ready for merge System tests
Projects
None yet
Milestone
0.21.0
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@im-konge @strimzi-ci @scholzj @ppatierno @Frawless