Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ttl of discovery cache configurable and raise default #107130

Closed
jonnylangefeld opened this issue Dec 20, 2021 · 4 comments · Fixed by #107141
Closed

Make ttl of discovery cache configurable and raise default #107130

jonnylangefeld opened this issue Dec 20, 2021 · 4 comments · Fixed by #107141
Assignees
Labels
kind/feature Categorizes issue or PR as related to a new feature. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/cli Categorizes an issue or PR as relevant to SIG CLI.

Comments

@jonnylangefeld
Copy link
Contributor

What would you like to be added?

The discovery cache for kubectl is defaulted to 10 minutes:

return diskcached.NewCachedDiscoveryClientForConfig(config, discoveryCacheDir, httpCacheDir, time.Duration(10*time.Minute))

That means that every 10 minutes of using kubectl against a cluster, it runs API requests against all group versions, which can take quite some time on clusters with many operators installed (like crossplane.io, GCP Config Connector, Azure Service Operator, etc.). I previously wrote this report that shows that every 10 minutes a simple kubectl get pods makes 170 API requests against a test cluster, vs 4 API requests when the cache exists. That results in significant productivity implications against such clusters. Here is a time comparison between a request that runs the discovery cache vs one that doesn't:

$ time kubectl get pods -n default
kubectl get pods -n default  0.50s user 1.19s system 11% cpu 14.174 total

$ time kubectl get --raw  "/api/v1/namespaces/default/pods"
kubectl get --raw "/api/v1/namespaces/default/pods"  0.06s user 0.03s system 23% cpu 0.398 total

The discovery cache doesn't really have to run every 10 minutes, as CRDs don't change that often. A lot of unnecessary load is created on clients and servers.

This issue is raised to change the discovery cache default to 24 hours and also to make the discovery cache ttl configurable.

Why is this needed?

Running the discovery cache every 10 minutes has a significant productivity impact on using kubectl on clusters with many CRDs as it is taking time to run these unnecessary requests.
The discovery cache doesn't really have to run every 10 minutes, as CRDs don't change that often. A lot of unnecessary load is created on clients and servers.

@jonnylangefeld jonnylangefeld added the kind/feature Categorizes issue or PR as related to a new feature. label Dec 20, 2021
@k8s-ci-robot k8s-ci-robot added needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. labels Dec 20, 2021
@k8s-ci-robot
Copy link
Contributor

@jonnylangefeld: This issue is currently awaiting triage.

If a SIG or subproject determines this is a relevant issue, they will accept it by applying the triage/accepted label and provide further guidance.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jonnylangefeld
Copy link
Contributor Author

/sig cli

@k8s-ci-robot k8s-ci-robot added sig/cli Categorizes an issue or PR as relevant to SIG CLI. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Dec 20, 2021
@mk46
Copy link
Member

mk46 commented Dec 20, 2021

I would like to work on this issue.

@mk46
Copy link
Member

mk46 commented Dec 20, 2021

/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/feature Categorizes issue or PR as related to a new feature. needs-triage Indicates an issue or PR lacks a `triage/foo` label and requires one. sig/cli Categorizes an issue or PR as relevant to SIG CLI.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants