[Bug] GrafanaDataSource in same namespace as operator not found when WATCH_NAMESPACE=""

[jsanda]

Describe the bug
grafana-operator can manage Grafana and GrafanaDashboard instances across multiple namespaces. This doc mentions that that there is multi-namespace support for GrafanaDataSource. A GrafanaDataSource has to be deployed in the same namespace as the operator.

If you configure the operator to watch multiple namespaces with WATCH_NAMESPACE="", the operator may still fail to discover a GrafanaDataSource created in the same namespace as the operator.

To Reproduce
Steps to reproduce the behavior:

1. Configure and deploy the operator with cluster-scoped permissions and with WATCH_NAMESPACE="".
2. Wait for the operator to become ready.
3. Deploy Grafana and GrafanaDataSource objects in same namepsace as the operator.
4. Wait for Grafana to become ready.
5. Log into Grafana. The data source is not listed.

Expected behavior
The operator should find the GrafanaDataSource and register it in the Grafana instance.

Suspect component/Location where the bug might be occuring
I need to review the code more, but I think the issue is here in datasource_controller.go.

Runtime (please complete the following information):

• Grafana Operator Version: v3.6.0
• Environment: Kind / GKE, Kubernetes 1.17, 1.18
• Deployment type: deployed

Additional context
I encountered this issue in the k8ssandra project I work on. It has Helm charts that can be used to easily reproduce the issue. I would be happy to provide steps if that would help.

[david-martin]

@jsanda Can you check if specifying the --scan-all flag is sufficient for your use case?
See https://github.com/integr8ly/grafana-operator/blob/master/documentation/multi_namespace_support.md#support-for-multiple-namespaces for more info on what it does.

It looks like setting the env var WATCH_NAMESPACE="" seems to cause unexpected behaviour, so would need to be investigated.
What way are you installing the operator and subsequently setting that var? Is it via helm?

[pb82]

We're going to document this behavior better for now. In the future we might support multi namespace support for data sources.

[pb82]

@jsanda does this work for you when you set the --scan-all operator flag?

[pb82]

@jsanda have you had a chance to try it with the --scan-all flag?

[jsanda]

Hey @pb82

Sorry for the late reply on this. I have not tested as we aren't using grafana-operator any more. Thanks for pinging me though!

[HVBE]

Thanks for the response, and sorry to see you've decided to move away from the grafana-operator, was there any specific reason that prompted you to switch to something else?