Start custom resource controllers when the CRD appears after KIC startup

[czeslavo]

Is there an existing issue for this?

• I have searched the existing issues

Problem Statement

When KIC detects that required CRDs are not installed, it disables their controllers on startup. Once CRDs are installed, in order to make KIC reconcile their resources, a pod restart is required.

From the UX perspective, the desired behavior would be to make it possible to spin up the CRD controller once it gets installed dynamically in runtime, with no restart requirement.

Proposed Solution

Watch CustomResourceDefinition installations and run particular controllers dynamically on demand, once a new CRD installation is detected.

Acceptance Criteria

• When a user doesn't install CRD before running KIC, the CRD controller is disabled at startup
• When a user installs CRD with KIC running in the cluster, KIC picks this event up and starts reconciling its resources
• Helm chart is updated with roles required for watching CustomResourceDefinitions

[mflendrich]

@czeslavo: TBD if this could be implemented upstream in controller-runtime

@czeslavo please comment with the link to the controller-runtime upstream issue requesting this.

[czeslavo]

There was an issue in the controller-runtime that was mentioning a similar feature: kubernetes-sigs/controller-runtime#1994. Watching CRD resources and starting up the controller manually was advised.

[pmalek]

Grooming notes: 04/04/2023

• mostly relevant before Gateway API gets to GA. After that's reached we wouldn't really need it.
• @czeslavo has most of the work done anyway so we might as well include it in KIC for better UX
• currently slated for v3.0 but that will not happen if I'm not mistaken anytime soon. So if we want to have it, it's better to add it sooner rather than later because Gateway API getting to GA will make this irrelevant.
• not relevant for Kong CRDs since those are include both in the helm chart and all on one manifests

[czeslavo]

POC: 5675231