Operator watched resources not getting created when deleted

[deva26]

Operator watched resources not getting created when deleted
Istio operator reconciler gets called when a watched resource is deleted. However the reconciler does not recreate the deleted resource as part of running the reconciliation.

Expected behavior
When a resource, say istiod deployment is deleted (possibly by a mistake), istio operator has to recreate the deleted resource as part of reconciiation

Steps to reproduce the bug

Version (include the output of istioctl version --remote and kubectl version and helm version if you used Helm)

How was Istio installed?

Environment where bug was observed (cloud vendor, OS, etc)

Recreate a watched resource when deleted was working several commits earlier. With the introduction of ObjectCache struct in the helmreconciler, reconciliation became a function of the delta between what was there in the cache and the new manifest created. In that process a change of what was running in the k8s cluster was missed out.

[morvencao]

The is because the new reconcile loop will just check the object cache for the watched objects, it if it is already in cache, it will do nothing.

The simplest way I can figure out is to delete object from cache when we get a delete event for the corresponding object, so that it will be recreated in the next reconcile loop.