Early-exit reconcile when the object is being deleted in all operators

[soenkeliebau]

Problem

Every product-cluster controller re-applies its owned children on any reconcile with no deletionTimestamp gate. During teardown, owned-resource watches fire as GC deletes the children, so the reconcile re-enters and tries to re-apply into a terminating namespace, failing with 403 ... namespace <ns> is being terminated (NamespaceTerminating). Publishing the error as a k8s Event fails the same way.

Impact:

• Does not leak resources — owner-reference GC still completes (all applied children carry controller: true owner refs).
• Does not itself block namespace deletion — the API server rejects the writes, nothing is recreated.
• Does waste reconciles, spam error logs, and add reconcile-queue pressure

This is nearly universal (affected: airflow, druid, hbase, hdfs, hive, kafka, nifi, opa, opensearch, spark-k8s, superset, trino, zookeeper, plus listener at low severity). Not needed for commons or secret.

Fix

Early-return at the top of the cluster reconcile, before any apply:

if obj.metadata.deletion_timestamp.is_some() {
    // Children carry controller ownerReferences and are GC'd by Kubernetes;
    // nothing to do, and re-applying would 403 against the terminating namespace.
    return Ok(controller::Action::await_change());
}

No finalizer is needed — owner-reference GC suffices for these operators. This is the standard controller-runtime / kube-rs guidance.

[adwk67]

This can be done while the object is still behind the DeserializeGuard. e.g. for HBase:

    if hbase.meta().deletion_timestamp.is_some() {
        return Ok(Action::await_change());
    }

This also requires that the operator watches or owns all relevant resources (owns for PDBs, service accounts and rolebindings are currently missing from many operators).