[Bug] Grafana dashboards seem to randomly jump to different folders

[ginokok1996]

Describe the bug
We are running the grafana v5 operator in Openshift.
Teams are able to create their own dashboards using the GrafanaDashboard objects.
This seemed to work fine but now we notice that from time to time dashboards are randomly jumping folders.
All the dashboards have unique names and uid's, they start off in the folder with the name of the namespace they are located in.
After a while they move to another random folder.
Removing the dashboard and wait for the operator to sync it again puts it back in the correct folder.

What dashboards jump to what folders also seem to differ from person to person.

Version
v5.4.1

To Reproduce
Steps to reproduce the behavior:
Not really know how to reproduce unfortunately.

Expected behavior
Grafana dashboards get synced and put in their correct folder and stay there.

Suspect component/Location where the bug might be occurring
Please provide this if you know where this bug might occur otherwise leave as unknown

Screenshots
If applicable, add screenshots to help explain your problem.

Runtime (please complete the following information):

• OS: Linux
• Grafana Operator Version v5.4.1
• Environment: Openshift 4.12.26
• Deployment type: Deployed via operatorhub

[ginokok1996]

My bad, i forgot to attach a screenshot as an example.

[weisdd]

@ginokok1996 could you prepare a lab in kind and share the instructions/manifests, so we can see this behaviour in action?

[pb82]

@ginokok1996 is this resolved now?

[ginokok1996]

@pb82 ,

Not yet.
I have yet to be able to reproduce it locally.
We have tried to define the folder name in the dashboard CR's, we thought that maybe it was getting the wrong namespace somehow when checking if a folder needs to be created.

But with no luck, Dashboards still end up in the wrong folder, when checking the CR in kubernetes the correct folder is named.

No errors in the operator whatsoever regarding anything about the folder..

[ginokok1996]

We are running 2 replica's of the grafana deployment.
I've just reverted to only 1 replica and the issue seems resolved.

So running multiple replica's seems to be the cause so far

[NissesSenap]

@ginokok1996 if you are running multiple replicas, you have to an external postgresql instance.
https://grafana.com/docs/grafana/latest/setup-grafana/set-up-for-high-availability/ else the sqlite3 database that is located in each pod will contain different information.

I will close this issue.