tensorboard-controller: Fix tensorboard endless restarts

[oshep]

tensorboard-controller: Fix tensorboard endless restarts
tensorboards-web-app deletes objects using foreground deletion policy, Tensorboard CR will stay until all owned objects are deleted reconcile loop might keep on trying to recreate the resources that the API server tries to delete. so when Tensorboard CR is terminating, reconcile loop should do nothing

This commit allow manager to skip reconcile while Tensorboard CR is deleting.

[oshep]

Hi there!
I'm faced with the issue that Tensorboards web app stucks on deletion process.
I've investigate the problem and found that child resources are deleted and automatically generated by the controller in a loop. It leads to stucking of deletion process.
The solution is pretty similar like it done for the notebook-controller manager.
fc09bd1

[kimwnasptd]

Nice catch, thanks @oshep!

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CybervisionVolodymyrPogorelov, kimwnasptd

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• components/tensorboard-controller/OWNERS [kimwnasptd]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment