benchmark different crd count

[Shaked]

Description

This is a benchmark test based on my conversation with @stefanprodan on Flux's Slack.

The test automatically generate N amount of CRDs directly after installing flux.

I am currently experiencing restarts with helm-controller. After testing collecting profile logs, @stefanprodan suggested that the problem might be related to the amount of CRDs I have in my cluster.

Profile (partial logs)

(pprof) top10
Showing nodes accounting for 668.27MB, 89.98% of 742.69MB total
Dropped 280 nodes (cum <= 3.71MB)
Showing top 10 nodes out of 113
      flat  flat%   sum%        cum   cum%
  335.59MB 45.19% 45.19%   335.59MB 45.19%  reflect.New
  112.06MB 15.09% 60.28%   112.06MB 15.09%  google.golang.org/protobuf/internal/impl.consumeStringValidateUTF8
   82.07MB 11.05% 71.33%    82.07MB 11.05%  io.ReadAll
   41.01MB  5.52% 76.85%   105.53MB 14.21%  k8s.io/kube-openapi/pkg/util/proto.(*Definitions).parseKind
   20.50MB  2.76% 79.61%       29MB  3.91%  k8s.io/kube-openapi/pkg/util/proto.(*Definitions).parsePrimitive
   18.01MB  2.43% 82.03%    18.01MB  2.43%  github.com/go-openapi/swag.(*NameProvider).GetJSONNames
   17.50MB  2.36% 84.39%    33.01MB  4.44%  k8s.io/kube-openapi/pkg/util/proto.VendorExtensionToMap
      15MB  2.02% 86.41%       15MB  2.02%  google.golang.org/protobuf/internal/impl.consumeStringSliceValidateUTF8
   14.01MB  1.89% 88.30%    54.03MB  7.27%  k8s.io/kube-openapi/pkg/validation/spec.(*Schema).UnmarshalNextJSON
   12.51MB  1.68% 89.98%    12.51MB  1.68%  reflect.mapassign0

CRDs and HR

k get crds | wc -l
109

k get hr -A | wc -l 
61

[Shaked]

This yellow line shows a test on an Azure AMD D2as_v5 with HR=100 and CRD=100, while limits where set to 1 CPU, 2Gi memory.

The next lines are from the same experiment, using HR=100 and CRD=150:

Added this for documentation purposes

[stefanprodan]

Superseded by #6

Thanks @Shaked I've included your CRD generator in main