feat: Use custom cache function in Che Operator

[mmorhun]

What does this PR do?

This PR introduces custom cache function for the default k8s client of the Operator. This is required, because default cache function caches every object of the kind cluster-wide that the Operator ever reads. Such behavior ends up in a huge memory usage on a cluster with many resources. Consequently, the Operator is terminated by OOM.

However, Operator's controller runtime has own restrictions on the cache function: every kind of k8s object is limited to a single selector that is used to detect if specific object should be cached. In turn, a selector cannot have or condition in it. And that's the price we have to pay: all k8s objects that are handled by the default Operator client should have a label (or set of them) set. Otherwise, Che Operator will not see the objects at all (unless a separate slow client is used).

After this PR is merged all k8s resources that Che Operator interacts with must have app.kubernetes.io/part-of=che.eclipse.org label.

The requirement above is also applicable to the user defined configurations in Config Maps and Secrets (such as custom TLS certificates, etc.). To not to break existing installations, Che Operator will search for such objects on its start and add the required label. This way existing deployments of Che will not be broken.

Screenshot/screencast of this PR

N/A

What issues does this PR fix or reference?

Resolves: eclipse-che/che#20647
Depends on: che-dockerfiles/che-tls-secret-creator#6

How to test this PR?

1. Deploy Eclipse Che using operator installer.
2. Edit Che Operator deployment by setting memory limit to 256Mi
3. Create 500 namespaces: for i in {1..500}; do oc create namespace test$i; done
4. Wait and check no OOM error in the Operator pod
5. Restart Operator pod
6. Wait and check no OOM error in the Operator pod

PR Checklist

As the author of this Pull Request I made sure that:

• The Eclipse Contributor Agreement is valid
• Code produced is complete
• Code builds without errors
• Tests are covering the bugfix
• Custom resource definition file is up to date
• OLM bundles are up to date
• The repository devfile is up to date and works
• Sections What issues does this PR fix or reference and How to test this PR completed
• Relevant user documentation updated
• Relevant contributing documentation updated
• CI/CD changes implemented, documented and communicated

Reviewers

Reviewers, please comment how you tested the PR when approving it.

[tolusha]

What about this label?
app.kubernetes.io/part-of: che.eclipse.org

[amisevsk]

No real issues on my end but I don't have a deep insight into how the Che operator works.

[openshift-ci]

New changes are detected. LGTM label has been removed.

[openshift-ci]

@mmorhun: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/v7-devworkspace-happy-path	6e2b7f6	link	true	/test v7-devworkspace-happy-path

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.