Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USHIFT-741: MicroShift invariant adaptations #27989

Closed
wants to merge 1 commit into from

Conversation

chiragkyal
Copy link
Member

@chiragkyal chiragkyal commented Jun 20, 2023

This change will skip alert invariant if "openshift-monitoring" namespace does not exist. It will also skip disruption invariant if it's a MicroShift cluster.

USHIFT-741

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jun 20, 2023
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jun 20, 2023

@chiragkyal: This pull request references USHIFT-741 which is a valid jira issue.

In response to this:

USHIFT-741

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 20, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jun 20, 2023

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@chiragkyal chiragkyal changed the title USHIFT-741: MicroShift invariant adaptions USHIFT-741: MicroShift invariant adaptations Jun 20, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 14, 2023

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: chiragkyal
Once this PR has been reviewed and has the lgtm label, please assign bparees for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@chiragkyal chiragkyal marked this pull request as ready for review July 17, 2023 13:13
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 17, 2023
@openshift-ci openshift-ci bot requested review from deads2k and mfojtik July 17, 2023 13:18
@chiragkyal
Copy link
Member Author

/retest

1 similar comment
@chiragkyal
Copy link
Member Author

/retest

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Jul 24, 2023

@chiragkyal: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-openstack-ovn 59acace link false /test e2e-openstack-ovn
ci/prow/e2e-aws-ovn-single-node-upgrade 59acace link false /test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-aws-ovn-serial 59acace link true /test e2e-aws-ovn-serial
ci/prow/e2e-metal-ipi-ovn-ipv6 59acace link false /test e2e-metal-ipi-ovn-ipv6

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@chiragkyal
Copy link
Member Author

/assign ingvagabund

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2023

@chiragkyal: This pull request references USHIFT-741 which is a valid jira issue.

In response to this:

This change will skip alert invariant if "openshift-monitoring" namespace does not exist. It will also skip disruption invariant if it's a MicroShift cluster.

USHIFT-741

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chiragkyal
Copy link
Member Author

/cc @copejon

@openshift-ci openshift-ci bot requested a review from copejon July 27, 2023 14:33
@copejon
Copy link
Contributor

copejon commented Jul 27, 2023

Changes here seem good @chiragkyal

@chiragkyal
Copy link
Member Author

Changes here seem good @chiragkyal

@copejon could you please add the label. Thanks!

@copejon
Copy link
Contributor

copejon commented Jul 31, 2023

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 31, 2023
@chiragkyal
Copy link
Member Author

/assign @soltysh

if err != nil {
panic(err)
}
isMicroShift, err := exutil.IsMicroShiftCluster(kubeClient)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of giving empty list of junit test cases, we should ask ourselves whether we need to run testServerAvailability at all. Checking the code:

$ grep -rn "testServerAvailability"
pkg/synthetictests/disruption.go:37:func testServerAvailability(
pkg/synthetictests/disruption.go:190:		ret = append(ret, testServerAvailability("sig-network-edge", locator, allDisruptionEventsIntervals, jobRunDuration, jobType)...)
pkg/synthetictests/disruption.go:209:		ret = append(ret, testServerAvailability("sig-trt", locator, allDisruptionEventsIntervals, jobRunDuration, jobType)...)

which leads to https://github.com/openshift/origin/blob/master/pkg/synthetictests/event_junits.go#L38-L39:

	tests = append(tests, TestAllIngressBackendsForDisruption(events, duration, jobType)...)
	tests = append(tests, TestExternalBackendsForDisruption(events, duration, jobType)...)

Question here is whether it is safe to skip TestAllIngressBackendsForDisruption and TestExternalBackendsForDisruption? If the answer is not, then instead of checking whether an installation is MicroShift one should check whether the ingress/externalbackends are present.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All these disruptions depend upon JobType and for MicroShift this will always be nil. So, I think we don't want to run any of these disruption invariants for MicroShift.

And to make this happen, I see the possible way is either call IsMicroShiftCluster() inside testServerAvailability and return an empty list of junit test cases (which I've done above), or

not to append the following to tests based on the same IsMicroShiftCluster() check inside event_junits.go

	tests = append(tests, TestAllAPIBackendsForDisruption(events, duration, jobType)...)
	tests = append(tests, TestAllIngressBackendsForDisruption(events, duration, jobType)...)
	tests = append(tests, TestExternalBackendsForDisruption(events, duration, jobType)...)

/cc @pacevedom @dhellmann let me know your thoughts.

@ingvagabund
Copy link
Member

IsMicroShiftCluster defined in https://github.com/openshift/origin/pull/27996/files#diff-35a89a7a7362642eebb559fb8564e857b00d6f7dd6322c3adabaf1adbd609d35R2097 should be used only in very limited cases. Where there's no other way.

panic(err)
}
if !nsExist {
return []*junitapi.JUnitTestCase{}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to avoid invoking testAlerts completely in https://github.com/openshift/origin/blob/master/pkg/synthetictests/event_junits.go#L54-L55. Question here to ask is whether there's a sufficient replacement for absence of the monitoring stack when checking the alerts in MicroShift. Does it make sense to e.g. run an external monitoring stack (or its simplified version)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to avoid invoking testAlerts completely in https://github.com/openshift/origin/blob/master/pkg/synthetictests/event_junits.go#L54-L55.

Based on the existence of "openshift-monitoring" ns right? So, IIUC the suggestion is to add this above check inside event_junits.go ?

Question here to ask is whether there's a sufficient replacement for absence of the monitoring stack when checking the alerts in MicroShift. Does it make sense to e.g. run an external monitoring stack (or its simplified version)?

AFAIK there is no plan to add any monitoring stack in MicroShift as of now. So, we might want to skip them as well.

@chiragkyal
Copy link
Member Author

In the last 1-2 days, a lot of refactoring was done through #28130 PR, and now I believe the changes suggested above will not hold true. @pacevedom should we consider accommodating these changes in your PR?

@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 7, 2023
@openshift-merge-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@pacevedom
Copy link
Contributor

Included in #28136

@chiragkyal
Copy link
Member Author

/close

@openshift-ci openshift-ci bot closed this Aug 9, 2023
@openshift-ci
Copy link
Contributor

openshift-ci bot commented Aug 9, 2023

@chiragkyal: Closed this PR.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@chiragkyal chiragkyal deleted the USHIFT-741 branch August 10, 2023 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

7 participants