Auth0 finalizers

[sebastianrosch]

Description

This adds a resource deletion protection finalizer to all resources created through Faros. This way, accidental deletion and garbage collection is prevented. We will have to remove the finalizer manually before being able to delete a resource manually. Also, GC will keep the resource around until the finalizer has been removed.

In case a resource is deleted in GitOps, Faros marks the GitTrackObject of the resource as deleted with an annotation. Once this GTO is reconciled, it removes the finalizer on the child (the actual resource) and then deletes itself, so that GC will cascade-delete the actual resource.

There is one disadvantage here: if resources with a finalizer ever get deleted by accident (GC or manually), they will get a deletion timestamp but won't be removed. If we then remove the finalizer, the resource would be deleted. If Faros is running and properly configured, and the resource still exists in GitOps, it would immediately recreate it. We might want to add a monitor for resource with a deletion timestamp, which would tell us that something happened in the cluster. Spinning up a new cluster could solve this.

This is no breaking change for our clusters, but we should roll this out slowly and monitor the impact.

References

This is a simplified implementation of pusher#101

Testing

• Unit tests
• Manual tests in minikube

[wimdec]

What will happen when we manually delete a GTO. Since there is a finalizer, it will keep existing with marked for deletion. If then the resource coupled to the GTO is updated in the git repository. Will the resource be updated?

[wimdec]

We indeed should have a monitor for objects which are not getting finalized in certain time. Not only for faros, but in general. Maybe there exists something similar.

[sebastianrosch]

What will happen when we manually delete a GTO. Since there is a finalizer, it will keep existing with marked for deletion. If then the resource coupled to the GTO is updated in the git repository. Will the resource be updated?

The GTOs don't have a Finalizer, so they can be deleted. But this would set the deletion timestamp on the child resource. If the resource is still in Git, GTO will be recreated, but the child would still have its deletion timestamp set. So yes, I think we need to monitor this as this can cause other issues down the line.

[sebastianrosch]

I'm not merging this yet as we have received feedback on the implementation: pusher#144

I want to work on this before merging, but we will use this build version until then.

[sebastianrosch]

We decided to no longer use Finalizers as the underlying problem is solved with ClusterGitTracks