Evaluate kubebuilder for controllers

[bobcatfish]

Expected Behavior

Though our controllers (e.g. pipeline run controller) don't do much, we should be happy with how they are built and prepared to move forward with the skeletons we do have as we start actually implementing functionality.

Actual Behavior

We used kubebuilder to bootstrap our controllers which allowed us to get up and moving pretty quickly. However the controllers it generates depend on a lot of code in controller-tools and controller-runtime.

We should compare a controller creating using these libs to one implemented from scratch (e.g. the knative/serving controllers) and make sure we want to keep using kubebuilder in the long run.

Additional Info

n/a

[bobcatfish]

We have decided not to use kubebuilder for now (removing it in #57). Here is a summary of our findings (also authored by @aaron-prindle).

Kubebuilder

Pros

• Easy to keep all files in sync as types are plumbed through (*_types.go, examples, controllers, etc.)
• A lot less boilerplate has to be copy/pasted around, e.g this logic in the knative/build controller is just controller.AddToManager in the kubebuilder version.
• Adds and maintains schema validation of types

Cons

• We cannot use the Knative implementation to guide us
• There aren’t examples of controllers using v1 kubebuilder style (the samples use the v0 style)
• In v1 of kubebuilder, there are no client libraries generated that can be compatible with existing libs such as client-go
  • Instead the generated controller operates at a “higher level” and has all of the type watching patched through in the controller-runtime library kubebuilder provides
    • This could be bad though as it is unclear how external/3rd-party people will be able to use client libraries as they are not generated (e.g. to create a client to access these types, for example like the knative end to end tests)
  • i.e it doesn’t seem to generate everything that update-codegen does

Imitating knative controllers and using knative/pkg

Pros

Uses common/documented k8s pattern of client library for types (i.e. libs generated by update-codegen which are compatible with libs like client-go)

Cons

knative/pkg has no documentation so it’s pretty hard to say what’s in there @_@ though there are generated go docs.

[bobcatfish]

After a huge delay, following here with some info that came up in #build re. a question from @dprotaso:

In v1 of kubebuilder, there are no client libraries generated that can be compatible with existing libs such as client-go

Question:

I don’t understand what it means to be compatible with the client-go lib?

Answer:

It means to be able to use the client-go libs with the CRDs and controllers, for example in our integration tests we're going to use the libs generated by updated-codegen.sh:
https://github.com/knative/build-pipeline/blob/9b3b398e909f231fb1a9cb5250bc8b94736e71a5/test/clients.go#L51

Which provide functions for like basic CRUD operations such as List:
https://github.com/knative/build-pipeline/blob/9b3b398e909f231fb1a9cb5250bc8b94736e71a5/test/pipeline_test.go#L48

Also functionality like parsing your local kubeconfig to for configuration etc.

But kubebuilder does plan to support client-go type functionality in the long run so we will want to revisit this, I think kubernetes-sigs/kubebuilder#403 is the relevant ticket.