isNamespaced check in Enhancer

[ANeumann82]

What happened:
The C* Tests started to fail on the step that expects a started plan to be IN_PROGRESS after 30 seconds.

What you expected to happen:
A started plan (after installation, update, etc.) should get into IN_PROGRESS a lot faster.

Anything else we need to know?:
The cause for this is the isNamespaced check in the enhancer:

With the code as-is, the timestamps show this:

2020/03/12 09:18:24 Engine: Executing Task 'node' of type Apply
2020/03/12 09:18:24 TaskApply(node): Start rendering resources
2020/03/12 09:18:24 TaskApply(node): Start enhancing resources
2020/03/12 09:18:40 TaskApply(node): Start applying resources
2020/03/12 09:18:42 TaskApply: Handle Resource sanity-test-namespace/cassandra-instance-node
2020/03/12 09:18:43 TaskApply: Handle Resource sanity-test-namespace/cassandra-instance-node-readiness-probe-sh
2020/03/12 09:18:45 TaskApply: Handle Resource sanity-test-namespace/cassandra-instance-generate-tls-artifacts-sh

If I manually change isNamespaced := false for testing, the timestamps look better:

2020/03/12 09:21:32 Engine: Executing Task 'node' of type Apply
2020/03/12 09:21:32 TaskApply(node): Start rendering resources
2020/03/12 09:21:32 TaskApply(node): Start enhancing resources
2020/03/12 09:21:32 TaskApply(node): Start applying resources
2020/03/12 09:21:33 TaskApply: Handle Resource sanity-test-namespace/cassandra-instance-generate-cqlshrc-sh
2020/03/12 09:21:35 TaskApply: Handle Resource sanity-test-namespace/cassandra-instance-node

We currently use the discovery.DiscoveryInterface for checking if a resource is namespaced, there is also a discovery.CachedDiscoveryInterface, which may be a good idea. Introduces caching though...

Another solution would be to set the IN_PROGRESS status faster, before the first task is executed. This may be required at some point anyway, if we get operators that apply > 50 resources the execution of the first task may take a long time without any feedback in the plan status.

How to reproduce
This is mostly an issue for operators that deploy a lot of resources, i.e. C*, Kafka, etc. It probably won't show up in an operator with just one or two resources.

Environment:

• Kudo version (use kubectl kudo version): 0.11

[porridge]

A variant of #1413? That one shows that this is not only slow but also racy. Closing as a duplicate of this one.

[gerred]

Assigning this to critical based on feedback. We need to ship a 0.11.1 with a fix for this in place.

[gerred]

made a 0.11.1 milestone to this, feel free to add any other issues to 0.11.1

[zen-dog]

Fixed by #1427