Flightdeck Reconciliation Loop

[ian-flores]

Tailing the team operator logs in a cluster shows that it is constantly reconciling the Flightdeck resources as if there are constant changes happening. Some part of the logic for Flightdeck might be pushing an "update" with every run, triggering another reconciliation.

At first glance, the Flightdeck controller calls Owns() on child resources in SetupWithManager() , which none of our other controllers do. Perhaps this related? If this is the correct pattern and our other controllers are just deficient, the problem might lie in the kubernetes_crudding.go logic.

---

Migrated from rstudio/ptd#2681

[ian-flores]

Root Cause Analysis

After investigating, the reconciliation loop is caused by a combination of three factors:

1. .Owns() declarations in SetupWithManager

FlightdeckReconciler is the only controller in this codebase that uses .Owns():

func (r *FlightdeckReconciler) SetupWithManager(mgr ctrl.Manager) error {
    return ctrl.NewControllerManagedBy(mgr).
        For(&positcov1beta1.Flightdeck{}).
        Owns(&appsv1.Deployment{}).
        Owns(&corev1.Service{}).
        Owns(&corev1.ServiceAccount{}).
        Owns(&rbacv1.Role{}).
        Owns(&rbacv1.RoleBinding{}).
        Owns(&networkingv1.Ingress{}).
        Complete(r)
}

All other controllers just use .For(Resource).Complete().

2. Unconditional updates in BasicCreateOrUpdate()

The function at internal/kubernetes_crudding.go:190 always calls r.Update() without checking if the object actually changed:

// Always updates, even if nothing changed
targetObj.SetResourceVersion(existingObj.GetResourceVersion())
if err := r.Update(ctx, targetObj); err != nil {
    // ...
}

3. The Loop Mechanism

1. FlightdeckReconciler.Reconcile() triggered
2. Builds fresh target objects (Deployment, Service, etc.)
3. BasicCreateOrUpdate() unconditionally calls r.Update()
4. Kubernetes updates ManagedFields/resourceVersion
5. Watch event fires (object modified)
6. .Owns() enqueues FlightdeckReconciler
7. GOTO 1 - infinite loop

Proposed Fix

This is related to #11 (SetControllerReference). The fix should:

1. Replace BasicCreateOrUpdate() with controller-runtime's controllerutil.CreateOrUpdate which uses a MutateFn pattern and only updates if changes occurred
2. Standardize owner reference handling using ctrl.SetControllerReference()
3. Keep .Owns() - it's actually the correct pattern once the update logic is fixed