Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Empty Grafana dashboard #137

Closed
cmoulliard opened this issue Feb 14, 2020 · 4 comments
Closed

Empty Grafana dashboard #137

cmoulliard opened this issue Feb 14, 2020 · 4 comments

Comments

@cmoulliard
Copy link

cmoulliard commented Feb 14, 2020

Issue

I deployed successfully the Grafana Operator on kubernetes 1.15 using your instructions and the following files - https://github.com/halkyonio/r-d/tree/8b5961f6bfc2daeb0c8e41215b7b117490fe47d4/olm/resources/prometheus/grafana but the dashboard dont show the metrics collected from the prometheus server

In fact, when we are connected, grafana is requesting to add dashboards. See screenshots

Screenshot 2020-02-14 18 01 34

Screenshot 2020-02-14 18 23 07

What could be the issue as the plugin of prometheus is well configured and test to access it succeed !
Is it due to a RBAC issue : https://gist.github.com/cmoulliard/49aaeaaaf7ef9cc5c136db275be05e47 ?

@pb82
Copy link
Collaborator

pb82 commented Feb 14, 2020

So there's two distinct issues with two dashboards:

{"level":"info","ts":1581705027.170298,"logger":"controller_grafanadashboard","msg":"dashboard demo/keycloak-dashboard found but selectors do not match"}

and

{"level":"info","ts":1581705026.4836974,"logger":"controller_grafanadashboard","msg":"cannot submit dashboard demo/prometheus-dashboard"}

The first one means that a dashboard was found, but the dashboard selector in the Grafana CR does not match its labels.

The second one seems to be the real problem: there was an error submitting the dashboard to Grafana. A detailed error message can be found on the dashboard CR status:

$ kubectl get grafanadashboard prometheus-dashboard -n demo -oyaml

Can you please share the output of that command?
This often happens when basic auth is disabled in the Grafana CR. Currently the Operator relies on basic auth to talk to the Grafana API.

@cmoulliard
Copy link
Author

cmoulliard commented Feb 14, 2020

A detailed error message can be found on the dashboard CR status:

Here is the status

apiVersion: integreatly.org/v1alpha1
kind: GrafanaDashboard
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
...
  creationTimestamp: "2020-02-14T21:35:47Z"
  generation: 1
  labels:
    app: grafana
  name: prometheus-dashboard
  namespace: demo
  resourceVersion: "97246"
  selfLink: /apis/integreatly.org/v1alpha1/namespaces/demo/grafanadashboards/prometheus-dashboard
  uid: dc682f02-e171-4d30-9087-c79c9744bab4
spec:
  json: |
  name: prometheus-dashboard.json
status:
  hash: ""
  id: 0
  message: error creating dashboard, expected status 200 but got 404
  phase: failing
  slug: ""
  uid: ""

@cmoulliard
Copy link
Author

Here is the grafana config. Do you then recommend to disable auth.anonymous to false.
Remark: As this problem already happened for others, that should be documented !

apiVersion: integreatly.org/v1alpha1
kind: Grafana
metadata:
  name: grafana
spec:
  ingress:
    enabled: True
    hostname: "grafana-console.88.99.186.195.nip.io"
  config:
    log:
      mode: "console"
      level: "warn"
    security:
      admin_user: "root"
      admin_password: "secret"
    auth:
      disable_login_form: False
      disable_signout_menu: True
    auth.anonymous:
      enabled: True
  dashboardLabelSelector:
    - matchExpressions:
        - {key: app, operator: In, values: [grafana]}

@pb82
Copy link
Collaborator

pb82 commented Mar 2, 2020

Closing as duplicate od #136

@pb82 pb82 closed this as completed Mar 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants