fix addon controller reconciling too frequently

[xrstf]

What this PR does / why we need it:
In #4773 we tried to make the addon controller reconcile less frequently. The logic was to only react to Cluster changes if the AddonControllerReconcilingSuccess condition changed. The only controller setting this condition is the addon controller itself (looking back at the original PR, I don't really understand it... but I was young and dumber).

However this condition on the Cluster object isn't really saying much. If you have 3 addons, one of them fails, it is basically random which status the AddonControllerReconcilingSuccess on the Cluster object will have.

Even worse: Suppose all your addons are happy and healthy. Now you change a field like the Cluster Owner or any other field that is part of the addon TemplateData. Even though you changed the Cluster object, no reconciliation will happen because neither the CNI values nor the condition has changed.

Additionally, Addon objects are reconciled even when their status changes. This isn't much of a problem right now, because there is only 1 Condition and it only gets set exactly once and never heartbeats, but if the status is extended (an I have a branch ready for that 😁 ), then status changes on an addon should also be ignored.

To that effect, this PR adjusts the watches:

• Clusters are reconciled when the TemplateData we derive from them changed. TemplateData is the only thing that could influence how an addon is rendered. Note that external resources, like the kubeconfig, are not taken into account: We'd need to store somehow the revision of each related object, so we can diff those as well (recursively) and at that point we begin to re-implement the Applications feature. So for addons I chose the simple approach of relying on the auto-resync interval of controller-runtime to reconcile all addons after a number of hours, regardless of state.
• Addons are only reconciled if their generation is changed, i.e. not just the status.
• I also noticed some diffs where we removed the UID we set ourselves. In Addons, the cluster ref doesn't care about the UID and it should be left blank. I adjusted the code a bit to ensure we ourselves do not set it, saving one more reconciliation per addon.

What type of PR is this?
/kind cleanup

Does this PR introduce a user-facing change? Then add your Release Note here:

Addons reconciliation is triggered more consistently for changes to Cluster objects, reducing the overall number of unnecessary addon reconciliations.

Documentation:

NONE

[xrstf]

/retest

[embik]

/approve

[kubermatic-bot]

LGTM label has been added.

Git tree hash: 7c0daa622e1bcdf674ef842a5a1661239bc8f163

[kubermatic-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: embik

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [embik]
• pkg/controller/OWNERS [embik]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[xrstf]

/retest

[xrstf]

/cherrypick release/v2.25

[kubermatic-bot]

@xrstf: new pull request created: #13262

In response to this:

/cherrypick release/v2.25

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[xrstf]

/cherrypick release/v2.24

[kubermatic-bot]

@xrstf: new pull request created: #13263

In response to this:

/cherrypick release/v2.24

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.