Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

discovery: speedup kubectl restmapper cache misses by a two-digit factor #73345

Merged
merged 3 commits into from Feb 12, 2019

Conversation

@sttts
Copy link
Contributor

sttts commented Jan 26, 2019

The restmapper code did not call the optimized, parallelized API group resources discovery func, but did its own serial requests, one per API group version. Each took some hundred milliseconds, adding up to >10 sec for cluster with 40ish many API group versions.

Note: the tests in the first commit are also green before the second and third commits are applied.

Speedup kubectl by >10 when calling out to kube-apiserver for discovery information.
@@ -145,27 +145,34 @@ func NewDiscoveryRESTMapper(groupResources []*APIGroupResources) meta.RESTMapper
// GetAPIGroupResources uses the provided discovery client to gather
// discovery information and populate a slice of APIGroupResources.
func GetAPIGroupResources(cl discovery.DiscoveryInterface) ([]*APIGroupResources, error) {
// TODO: we don't want this ServerGroups call. It is only there to get the metav1.APIGroup struct used below.

This comment has been minimized.

@sttts

sttts Jan 26, 2019

Author Contributor

we have to fix this

@sttts

This comment has been minimized.

Copy link
Contributor Author

sttts commented Jan 26, 2019

@yliaog

This comment has been minimized.

Copy link
Contributor

yliaog commented Jan 31, 2019

/cc

@k8s-ci-robot k8s-ci-robot requested a review from yliaog Jan 31, 2019

@roycaihw

This comment has been minimized.

Copy link
Member

roycaihw commented Jan 31, 2019

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from a913a2c to 5f88a2c Feb 5, 2019

@sttts

This comment has been minimized.

Copy link
Contributor Author

sttts commented Feb 5, 2019

/hold cancel

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from d340629 to eaaca48 Feb 5, 2019

@liggitt

This comment has been minimized.

Copy link
Member

liggitt commented Feb 5, 2019

the ServerGroupsAndResources change looks good (modulo interface documentation). The GetAPIGroupResources change needs thought to either behave the same in partial discovery cases, or to update callers to handle partial discovery results correctly.

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from eaaca48 to 4a32ce1 Feb 12, 2019

@soltysh
Copy link
Contributor

soltysh left a comment

The current state, including partial discover (previous logic) and tests are in place.
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm label Feb 12, 2019

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from 4a32ce1 to ad4f045 Feb 12, 2019

@k8s-ci-robot k8s-ci-robot added lgtm and removed lgtm labels Feb 12, 2019

@soltysh
Copy link
Contributor

soltysh left a comment

/lgtm

@soltysh
Copy link
Contributor

soltysh left a comment

/approve

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from ad4f045 to 0fcf5ba Feb 12, 2019

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Feb 12, 2019

New changes are detected. LGTM label has been removed.

@k8s-ci-robot k8s-ci-robot removed the lgtm label Feb 12, 2019

@sttts sttts added the lgtm label Feb 12, 2019

@deads2k

This comment has been minimized.

Copy link
Contributor

deads2k commented Feb 12, 2019

/approve

@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

k8s-ci-robot commented Feb 12, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: deads2k, soltysh, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sttts sttts removed the lgtm label Feb 12, 2019

@sttts sttts force-pushed the sttts:sttts-speedup-cache-miss-kubectl branch from 0fcf5ba to 2cdddd8 Feb 12, 2019

@sttts sttts added the lgtm label Feb 12, 2019

@k8s-ci-robot k8s-ci-robot merged commit 64ce2e5 into kubernetes:master Feb 12, 2019

17 checks passed

cla/linuxfoundation sttts authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-godeps Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped
pull-kubernetes-local-e2e-containerized Skipped
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped
tide In merge pool.
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.