Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The garbage collector and quota counter should use the metadata client and protobuf to access resources #78742

Merged
merged 3 commits into from Jul 11, 2019

Conversation

@smarterclayton
Copy link
Contributor

commented Jun 5, 2019

The garbage collector and the generic object count quota controller currently uses dynamic client and JSON to perform operations only on resource metadata. This PR changes the both to use the metadata client - a client that uses the server-side conversion logic to retrieve only the metadata about the object in protobuf - to more efficiently interact with the server.

Renames GenericInformerFactory to make it clear that these informers return only a subset of objects.

Built on #77819 and #78743

Part of https://github.com/kubernetes/enhancements/blob/master/keps/sig-api-machinery/20190322-server-side-get-to-ga.md

/kind cleanup

The garbage collector and generic object quota controller have been updated to use the metadata client which improves memory
and CPU usage of the Kube controller manager.
@fedebongio

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2019

@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from c39bf95 to 49be184 Jun 24, 2019
@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from 49be184 to fb16dc8 Jul 8, 2019
@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch 3 times, most recently from 97ae9f5 to d76e6b5 Jul 8, 2019
@smarterclayton

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

/retest

@smarterclayton

This comment has been minimized.

Copy link
Contributor Author

commented Jul 9, 2019

/retest

@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from 9e474e8 to cc5baee Jul 10, 2019
@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from 565deca to dd47b50 Jul 10, 2019
@smarterclayton smarterclayton changed the title The garbage collector should use the metadata client and protobuf to access resources The garbage collector and quota counter should use the metadata client and protobuf to access resources Jul 10, 2019
@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch 2 times, most recently from 4e2cb87 to dad5612 Jul 10, 2019
@k8s-ci-robot k8s-ci-robot added size/L and removed size/XXL labels Jul 11, 2019
@k8s-ci-robot

This comment has been minimized.

Copy link
Contributor

commented Jul 11, 2019

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: smarterclayton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from dad5612 to 0f49648 Jul 11, 2019
@smarterclayton

This comment has been minimized.

Copy link
Contributor Author

commented Jul 11, 2019

Ok, this is green. Will run e2e-gce a couple of times looking for flakes, but haven't seen any see far. Two new commits, rebased on top of master which merged the informer/fake code from the NS PR.

/test pull-kubernetes-e2e-gce

@liggitt

This comment has been minimized.

Copy link
Member

commented Jul 11, 2019

lgtm, just the one comment about metav1.PartialObjectMetadata vs metav1beta1

@liggitt

This comment has been minimized.

Copy link
Member

commented Jul 11, 2019

/lgtm
/hold

feel free to retag once the version is changed

Updated name to match dynamic client
…ager

All controllers in controller-manager that deal with objects generically
work with those objects without needing the full object. Update the GC
and quota controller to use PartialObjectMetadata input objects which
is faster and more efficient.
@smarterclayton smarterclayton force-pushed the smarterclayton:client_gc branch from 0f49648 to d631f9b Jul 11, 2019
@k8s-ci-robot k8s-ci-robot removed the lgtm label Jul 11, 2019
@liggitt

This comment has been minimized.

Copy link
Member

commented Jul 11, 2019

/lgtm
/retest

@k8s-ci-robot k8s-ci-robot added the lgtm label Jul 11, 2019
@k8s-ci-robot k8s-ci-robot merged commit 91b2634 into kubernetes:master Jul 11, 2019
23 checks passed
23 checks passed
cla/linuxfoundation smarterclayton authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-conformance-image-test Skipped.
pull-kubernetes-cross Skipped.
pull-kubernetes-dependencies Job succeeded.
Details
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-100-performance Job succeeded.
Details
pull-kubernetes-e2e-gce-csi-serial Skipped.
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-gce-iscsi Skipped.
pull-kubernetes-e2e-gce-iscsi-serial Skipped.
pull-kubernetes-e2e-gce-storage-slow Skipped.
pull-kubernetes-godeps Skipped.
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce-big Job succeeded.
Details
pull-kubernetes-local-e2e Skipped.
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-node-e2e-containerd Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details
pull-publishing-bot-validate Skipped.
tide In merge pool.
Details
@liggitt liggitt added this to the v1.16 milestone Aug 6, 2019
@RainbowMango RainbowMango referenced this pull request Aug 29, 2019
6 of 6 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.