kubectl get managed takes a long time to gain results

[jonnylangefeld]

What problem are you facing?

On a test cluster I have the aws and gcp provider installed. I have a single composition for an encrypted storage bucket that generates 6 gcp provider resources. I created one instance of such a bucket.
If I try to list the generated resources via kubectl get managed, it takes over 1 minute to gather the results:

time kubectl get -v 8 managed
...
kubectl get -v 8 managed  19.17s user 14.88s system 46% cpu 1:13.92 total

That is due to the fact that kubectl makes an API request to every single VersionKind combination that is in the managed category, which happens to be all cloud provider resources, which is a lot. The -v 8 parameter in the kubectl get query reveals 100s of API calls like the following:

I1122 08:56:43.614282   23158 round_trippers.go:463] GET https://10.216.1.114/apis/sql.gcp.upbound.io/v1beta1/sslcerts?limit=500
I1122 08:56:43.614290   23158 round_trippers.go:469] Request Headers:
I1122 08:56:43.614296   23158 round_trippers.go:473]     Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json
I1122 08:56:43.614301   23158 round_trippers.go:473]     User-Agent: kubectl/v1.25.0 (darwin/arm64) kubernetes/a866cbe
I1122 08:56:43.785959   23158 round_trippers.go:574] Response Status: 200 OK in 171 milliseconds
I1122 08:56:43.785995   23158 round_trippers.go:577] Response Headers:
I1122 08:56:43.786005   23158 round_trippers.go:580]     Content-Type: application/json
I1122 08:56:43.786012   23158 round_trippers.go:580]     X-Kubernetes-Pf-Flowschema-Uid: 51df69e7-ba1a-4a2a-8ce3-b22f0a4cc875
I1122 08:56:43.786020   23158 round_trippers.go:580]     X-Kubernetes-Pf-Prioritylevel-Uid: f98d6278-13cb-49b2-a65c-fa56199f8b4a
I1122 08:56:43.786027   23158 round_trippers.go:580]     Content-Length: 1249
I1122 08:56:43.786034   23158 round_trippers.go:580]     Date: Tue, 22 Nov 2022 16:56:43 GMT
I1122 08:56:43.786041   23158 round_trippers.go:580]     Audit-Id: a648324e-8b26-4f01-84db-170699e193c6
I1122 08:56:43.786048   23158 round_trippers.go:580]     Cache-Control: no-cache, private
I1122 08:56:43.786082   23158 request.go:1073] Response Body: {"kind":"Table","apiVersion":"meta.k8s.io/v1","metadata":{"resourceVersion":"1174733304"},"columnDefinitions":[{"name":"Name","type":"string","format":"name","description":"Name must be unique within a namespace. Is required when creating resources, although some resources may allow a client to request the generation of an appropriate name automatically. Name is primarily intended for creation idempotence and configuration definition. Cannot be updated. More info: http://kubernetes.io/docs/user-guide/identifiers#names","priority":0},{"name":"READY","type":"string","format":"","description":"Custom resource definition column (in JSONPath format): .status.conditions[?(@.type=='Ready')].status","priority":0},{"name":"SYNCED","type":"string","format":"","description":"Custom resource definition column (in JSONPath format): .status.conditions[?(@.type=='Synced')].status","priority":0},{"name":"EXTERNAL-NAME","type":"string","format":"","description":"Custom resource definition column (in JSONPath format): .metadat [truncated 225 chars]

How could Crossplane help solve your problem?

Maybe there is another way to see all managed resources? Maybe via the kubectl crossplane cli?

[negz]

Yeah, I thought we had an issue tracking this already but apparently not - thanks for raising it.

This is a really tough one to fix thanks to the very request-heavy nature of CRD category queries. I can't imagine there's much that could be done other than server-side aggregation, such that the API server exposed a single endpoint allowing folks to list a category of resources rather than a single type.

Short of changing Kubernetes, one option would be to use https://github.com/upbound/xgql and a client thereof. It's able to make and cache this kind of inefficient queries on clients behalf.

[ivanovaleksandar]

I think that this is closely related to this: #2869 . The performance degradation is tied with the bloat that crossplane needs to go through. I faced the same thing when working with compositions and wanted to have a congested view of related resources, so using kubectl get managed is the most logical approach.

[github-actions]

Crossplane does not currently have enough maintainers to address every issue and pull request. This issue has been automatically marked as stale because it has had no activity in the last 90 days. It will be closed in 7 days if no further activity occurs. Leaving a comment starting with /fresh will mark this issue as not stale.

[ytsarev]

/fresh kubectl get managed is still not the fastest thing around :)