Unified Kuadrant - Kuadrant Operator Single Policy Controller

[maleck13]

What

Currently we have a 2 policy controllers. One for DNS and TLS policy and second for RateLimit and Auth Policy. In addition to this we have the "enforcement" of DNS available with a DNSRecord and ManagedZone controller. We want to get to a place where we have only one policy controller for Kuadrant that can operate in a multi-cluster or single cluster environment.

Why
The requirements and responsibilities of these policy controllers are very similar. They watch policy and associated network resources and convert the policy into specific configuration to be applied by an "enfocement" component such as Limitador or cert-manager. Having two separate controllers that share a lot of the same responsibilities and requirements is not ideal as we end up duplicating code, and creating awkward deployments. We already see this happening with the policy controller and CRDs being bundled into the CSV of the kuadrant operator and scripts from the kuadrant-operator depending on scripts from the multicluster gateway controller.

Goal

End up with a single policy controller that is installed into any environment that wants to use the policy APIs. Remove the need for the separate policy controller that has DNS and TLS in it.

How

We will move the DNSPolicy and TLSPolicy controllers into the kuadrant-operator. We will also have a way to indicate to policy controllers whether they should reconcile a given type of policy or instead just add context to the status of the policy object.

Examples:

• Kuadrant Operator installed into a hub and a spoke. Hub will reconcile the DNSPolicy and enforce it, spoke will simple update the status of the DNSPolicy to call out it is handled by the hub

Phase1

• Move DNS Controller into its own component multicluster-gateway-controller#779
• Publish operator to operatorhub dns-operator#11
• Remove OCM module dependencies from DNSPolicy multicluster-gateway-controller#622
• Move TLSPolicy to Kuadrant Operator multicluster-gateway-controller#766
• Move DNSPolicy to Kuadrant Operator multicluster-gateway-controller#767
• Combine OCM add-on manager and gateway controller multicluster-gateway-controller#661
• Install kuadrant-operator on the hub and remove policy controller multicluster-gateway-controller#784
• Update README and CRD documentation dns-operator#10

Note: At the end of phase 1 all kuadrant policies and controllers will be present and running on both hub and spoke. This should not be an issue though since we do not sync or create polices from one to the other in MGC currently.

Followed up by #69

[maleck13]

related Kuadrant/multicluster-gateway-controller#430

[mikenairn]

Phase 1

PRs:

• Unified kuadrant - Add TLSPolicy and DNSPolicy kuadrant-operator#416
• Unified kuadrant - Remove TLSPolicy and DNSPolicy multicluster-gateway-controller#785
• GH-661 join addon-manager and gw-controller binaries multicluster-gateway-controller#781
• update docs for unified kuadrant docs.kuadrant.io#62

New Components:

• https://github.com/Kuadrant/kuadrant-dns-operator

Operator Catalog:

• operator [N] [CI] dns-operator (0.1.0) k8s-operatorhub/community-operators#3881
• operator dns-operator (0.1.0) redhat-openshift-ecosystem/community-operators-prod#4027