New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up master reconciler tests #93721
Speed up master reconciler tests #93721
Conversation
Welcome @gitlawr! |
Hi @gitlawr. Thanks for your PR. I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. |
/assign @timothysc |
/ok-to-test |
@gitlawr: The following test failed, say
Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks, this looks promising. I'd also consider lowering the numbers of masters in the test from 3 to 2, like
diff --git a/test/integration/master/kube_apiserver_test.go b/test/integration/master/kube_apiserver_test.go
index d4d52a7506f..01becece577 100644
--- a/test/integration/master/kube_apiserver_test.go
+++ b/test/integration/master/kube_apiserver_test.go
@@ -488,15 +488,15 @@ func testReconcilersMasterLease(t *testing.T, leaseCount int, masterCount int) {
}
func TestReconcilerMasterLeaseCombined(t *testing.T) {
- testReconcilersMasterLease(t, 1, 3)
+ testReconcilersMasterLease(t, 1, 2)
}
func TestReconcilerMasterLeaseMultiMoreMasters(t *testing.T) {
- testReconcilersMasterLease(t, 3, 2)
+ testReconcilersMasterLease(t, 2, 1)
}
func TestReconcilerMasterLeaseMultiCombined(t *testing.T) {
- testReconcilersMasterLease(t, 3, 3)
+ testReconcilersMasterLease(t, 2, 2)
}
func TestMultiMasterNodePortAllocation(t *testing.T) {
The main thing we want to exercise is multiple processes interacting, which that still does. I'd be curious how that impacted the test speed as well.
Start apiservers in parallel. Reduce the number of masters. Reduce the time for ReconcilerMasterLease.* tests from ~85s to ~41s on average.
83db1cb
to
11b347c
Compare
By reducing the number of masters the testing duration drops to ~41s. @liggitt I've updated the pr per your suggestion |
/lgtm thanks! I still think we should look for ways to improve this in the future, but this is a big initial improvement for a minimal change |
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: gitlawr, liggitt The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
/milestone v1.19 this package timeout is currently the primary source of flakes in the pull-integration job |
What type of PR is this?
/kind bug
/kind flake
What this PR does / why we need it:
Speed up master reconciler tests.
Start apiservers in parallel. It reduce the time for starting 3 instances of apiservers from ~13s to ~6s.
The comparison result is shown in the following diagram. On average, it reduces the test time from 85s to 57s(-33%). Tests are done by running the following command for 10 times:
Which issue(s) this PR fixes:
Related to issue #93609
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.: