Add Metrics/Test for Metrics

[ANeumann82]

What would you like to be added:
As we had a pretty ugly regression with #1421 that should have been obvious if we checked the reconcile-loop length in our e2e tests, we should look into this. The controller-runtime adds some common metrics, we should use them.

• Add a test to validates a max-duration for a reconcile
• Expose KUDO metrics somehow useful
• Maybe add logging about slow reconciles?
• Can we have a check for the number of API calls made in each reconcile run?

Why is this needed:
Prevent performance regressions in the future.

[alenkacz]

A bit related to #629